How AI engines find and cite websites
Each AI engine builds answers from a different crawler and index, so "AI SEO" is really five jobs. These guides take each engine in turn, from its own documentation: what it crawls with, where its answers come from, how it shows sources, and what you control.
| Engine | Crawlers to allow | Where answers come from | Measured in Pick Check |
|---|---|---|---|
| ChatGPTOpenAI | OAI-SearchBotChatGPT-User | ChatGPT search partners with third-party search providers, rewriting the question into targeted queries, and combines their results with OpenAI's own OAI-SearchBot crawl. | Yes |
| Google AI Overviews and AI ModeGoogle | Googlebot | Both features use the Google Search index. A page must be indexed and eligible to show with a snippet, with no additional requirements. They may run several related searches (query fan-out) and may use different models, so the links they show can differ. | Through Gemini (same Google index) |
| ClaudeAnthropic | Claude-SearchBotClaude-User | Anthropic's docs describe Claude grounding answers in content from the live web without naming a provider for text results. Its help center says image results are powered by Bing. | Yes |
| PerplexityPerplexity | PerplexityBotPerplexity-User | Perplexity searches the internet in real time across articles, websites and journals. Its Search API, launched in September 2025, exposes an index Perplexity describes as covering hundreds of billions of webpages, on the same infrastructure as its public answer engine. | Yes |
| GeminiGoogle | Googlebot | Gemini Apps grounding uses content from the Google Search index supplied to the model at prompt time. In the Gemini API, the Google Search tool lets the model decide whether to search, write the queries and return a grounded answer. | Yes |
Every Pick Check also asks Grok (xAI) with web search.