AI search
Should I block AI crawlers like GPTBot?
For most businesses, no — blocking AI crawlers removes you from the answers your future customers are already reading. If ChatGPT and Perplexity can't read your site, they recommend competitors whose sites they can.
Blocking makes sense mainly for publishers whose content is the product being resold. For a business whose site exists to win customers, AI assistants are a referral channel, and robots.txt is the door.
The stronger play is the opposite: allow the crawlers, then make what they find unambiguous — plain-text services, service area, pricing posture, and schema they can quote accurately.