Model training · OpenAI
GPTBot
GPTBot is OpenAI's crawler for content that may be used in model training. It honours robots.txt.
- Operator
- OpenAI
- Purpose
- Model training
- robots.txt
- Honoured
- Last checked
Documented user agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbotMatch on the token GPTBot rather than the whole string, since the version number inside the string can change. A user agent alone can be forged, so verify it as shown below.
robots.txt rules
To block it everywhere:
User-agent: GPTBot
Disallow: /To allow it explicitly, for example when a broader rule blocks other agents:
User-agent: GPTBot
Allow: /What blocking it changes
OpenAI says disallowing GPTBot indicates the site’s content should not be used to train its generative AI foundation models. It is documented as independent of search: blocking it does not remove the site from ChatGPT search.
- OpenAI says that if both GPTBot and OAI-SearchBot are allowed, it may use the results of one crawl for both purposes.
How to verify a request is genuine
Match the requesting IP against OpenAI’s published GPTBot ranges. Source: OpenAI: GPTBot IP ranges.
Other OpenAI agents
OAI-SearchBot: search indexChatGPT-User: user-triggered fetch
Sources
OpenAI's documentation for GPTBot, read on 15 Oct 2026. Operators change these pages without notice; check before relying on a detail. Definition: GPTBot in our glossary.
Check how your site treats GPTBot
The AI crawler access check reads your robots.txt and fetches a URL as this agent, so you see what a CDN or firewall does as well.
Run the free check All AI crawlers