Search index · OpenAI
OAI-SearchBot
OAI-SearchBot is OpenAI's crawler for its AI search index. It honours robots.txt.
- Operator
- OpenAI
- Purpose
- Search index
- robots.txt
- Honoured
- Last checked
Documented user agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbotMatch on the token OAI-SearchBot rather than the whole string, since the version number inside the string can change. A user agent alone can be forged, so verify it as shown below.
robots.txt rules
To block it everywhere:
User-agent: OAI-SearchBot
Disallow: /To allow it explicitly, for example when a broader rule blocks other agents:
User-agent: OAI-SearchBot
Allow: /What blocking it changes
Opted-out sites are not shown in ChatGPT search answers, though OpenAI says they can still appear as navigational links. To keep a specific page out, OpenAI points to noindex, which the crawler must be allowed to fetch to read.
- OpenAI says search changes can take about 24 hours to take effect after a robots.txt update.
- OpenAI asks site owners to let the crawler’s published IP ranges through their host or CDN, not only robots.txt.
How to verify a request is genuine
Match the requesting IP against OpenAI’s published OAI-SearchBot ranges. Source: OpenAI: OAI-SearchBot IP ranges.
Other OpenAI agents
GPTBot: model trainingChatGPT-User: user-triggered fetch
Sources
OpenAI's documentation for OAI-SearchBot, read on 15 Oct 2026. Operators change these pages without notice; check before relying on a detail. Definition: OAI-SearchBot in our glossary.
Check how your site treats OAI-SearchBot
The AI crawler access check reads your robots.txt and fetches a URL as this agent, so you see what a CDN or firewall does as well.
Run the free check All AI crawlers