AI search for shop owners · Lesson 3 of 6 · 3 min
Blocking AI training is fine. Blocking AI search isn't.
AI companies use different crawlers for training their models and for answering questions. You can say no to training and still be found.
Two kinds of crawler
Training crawlers (such as GPTBot, ClaudeBot and Common Crawl's CCBot) collect pages to train future AI models. Search and answer crawlers (such as OAI-SearchBot, ChatGPT-User, Claude-SearchBot and PerplexityBot) fetch pages to answer a person's question right now, often with a link to you.
They are separate on purpose, so owners can choose. Blocking training is a reasonable choice and doesn't hide you from AI search.
The common mistake
Many shops added "block all AI" rules in 2024-25, or switched on a plugin that did it for them. Those rules often block the search crawlers too, so the shop disappears from AI answers while thinking it only opted out of training.
Do this
- Keep your training choice as it is, but make sure the search and answer crawlers are allowed. The free check lists both kinds separately and only flags the search ones.
Go deeper
Sources
- OpenAI: OAI-SearchBot, ChatGPT-User and GPTBot
- Anthropic: ClaudeBot, Claude-User and Claude-SearchBot