OpenAI · AI search
OAI-SearchBot: OpenAI's ChatGPT search crawler, and why to allow it
OAI-SearchBot is OpenAI's search crawler: it surfaces and links websites in ChatGPT's search features. Sites that opt out of it are not shown in ChatGPT search answers, apart from possible navigational links. It is separate from GPTBot, so you can allow search while blocking training.
- robots.txt token
OAI-SearchBot- User agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot- Operator
- OpenAI
- Type
- AI search
- What it feeds
- ChatGPT's search features, where it is used to surface and link websites in answers.
- Follows robots.txt
- Yes. OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. Robots.txt changes take about 24 hours to reach its systems.
- How to verify it
- OpenAI publishes the IP ranges OAI-SearchBot uses (list)
Should you block OAI-SearchBot?
Blocking OAI-SearchBot keeps your pages out of ChatGPT search answers and citations, apart from possible navigational links. For most businesses that want ChatGPT to recommend them, it should stay allowed, along with OpenAI's published IP ranges in your firewall.
User-agent: OAI-SearchBot
Disallow: /
User-agent: OAI-SearchBot
Allow: /
Add these lines to /robots.txt at your site root. A group naming OAI-SearchBot replaces your User-agent: * rules for it, so repeat any paths you also want it to skip.
Live check
Is OAI-SearchBot allowed on your site?
OAI-SearchBot and OpenAI's other crawlers
OpenAI documents its bots at developers.openai.com/api/docs/bots, which replaced the old platform.openai.com/docs/bots page (that address now redirects). The three you will see most:
| Token | What it does | Block it if… |
|---|---|---|
| OAI-SearchBot | Surfaces and links sites in ChatGPT search | You don't want to appear in ChatGPT search answers |
| GPTBot | Crawls content that may train OpenAI's foundation models | You don't want your content used for training |
| ChatGPT-User | Fetches a page during a user's ChatGPT request | You don't want ChatGPT opening pages users ask about |
OpenAI also lists OAI-AdsBot, which only visits landing pages submitted as ChatGPT ads. OpenAI says each robots.txt setting is independent, but if you allow both GPTBot and OAI-SearchBot it may reuse one crawl for both.
What blocking OAI-SearchBot does, and what it doesn't
This is the bot that decides whether ChatGPT search can show you. Opt out and your pages are not shown in ChatGPT search answers, though OpenAI says they can still appear as navigational links. Blocking it does not affect training (that is GPTBot) and is not the control for live fetches by ChatGPT-User, which OpenAI says is not used to decide search inclusion.
Changes are not instant: OpenAI says it can take about 24 hours after a robots.txt update for search to adjust. If you just unblocked the bot, give it a day before checking.
How to allow OAI-SearchBot and block training
User-agent: OAI-SearchBot
Allow: /
User-agent: GPTBot
Disallow: /
A named group replaces your User-agent: * rules for that bot, so OAI-SearchBot will ignore wildcard Disallow lines once it has its own group. If you want to keep some paths private from search, repeat them in the OAI-SearchBot group. robots.txt works per host, so each subdomain needs its own file.
OpenAI also recommends allowing requests from its published IP ranges. Many sites "allow" the bot in robots.txt while a CDN bot-fight setting challenges it.
Checking that a visit is really OAI-SearchBot
The current example string is Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot. OpenAI notes the version number may change. When fetching robots.txt it may add a marker, giving ...; compatible; OAI-SearchBot/1.4; robots.txt; +https://openai.com/searchbot, which helps when your logs omit paths. Confirm the source address against searchbot.json.
The live check above requests your page with OAI-SearchBot's token and compares the result with a browser's, so it catches a firewall or CDN block. It can't use OpenAI's real IP addresses, so IP-keyed rules may differ. The robots.txt checker shows how your rules resolve for each bot.
Questions
- If I block OAI-SearchBot, can ChatGPT still link to my site?
- Possibly, but only as a navigational link. OpenAI says opted-out sites will not be shown in ChatGPT search answers, though they can still appear as navigational links.
- How long until a robots.txt change for OAI-SearchBot takes effect?
- OpenAI says it can take about 24 hours from a robots.txt update for its search systems to adjust.
- Why does OAI-SearchBot's user agent look like Chrome on a Mac?
- Its current example string starts with a Chrome/131 browser prefix before the OAI-SearchBot/1.4 token. OpenAI says the version number may change, so match on the OAI-SearchBot token.
- Does allowing OAI-SearchBot mean OpenAI trains on my content?
- Not by itself. Training is controlled by GPTBot. OpenAI does say that if you allow both bots, it may use one crawl for both purposes, so disallow GPTBot if you want search without training.
- Is robots.txt enough, or do I need firewall changes too?
- OpenAI recommends both allowing OAI-SearchBot in robots.txt and allowing requests from its published IP ranges. A CDN bot-protection rule can block it even when robots.txt allows it.
Related crawlers
Sources
Checked against these pages on 2026-10-18. Operators change their crawlers; check the source before relying on a detail.