OAI-SearchBot

OAI-SearchBot is the OpenAI crawler that builds the index used by ChatGPT search. It identifies itself with the token OAI-SearchBot and honours robots.txt. OpenAI’s documentation separates it from GPTBot, which collects training data, and from ChatGPT-User, which fetches a page on a user’s request.

Why the distinction matters

Many sites blocked “OpenAI” in 2023 and 2024 with a single GPTBot rule. That rule does not affect OAI-SearchBot, so those sites can still be linked in ChatGPT search. The reverse mistake is more costly: a blanket rule that catches OAI-SearchBot removes the site from ChatGPT’s search results while leaving training use unchanged.

Recommended rule

If the goal is to appear in ChatGPT search, allow OAI-SearchBot explicitly, whatever the decision on GPTBot:

User-agent: OAI-SearchBot
Allow: /

Check the live behaviour rather than the file: a CDN or bot-protection layer can block an agent the robots.txt allows. Our AI crawler access tool fetches a URL as each documented agent and reports the status.

For the documented user-agent string, the robots.txt rules and how to verify a request, see the OAI-SearchBot reference page.