User-triggered fetch · Anthropic
Claude-User
Claude-User is Anthropic's fetcher for user-requested pages. It honours robots.txt.
- Operator
- Anthropic
- Purpose
- User-triggered fetch
- robots.txt
- Honoured
- Last checked
Documented user agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Claude-User/1.0; +Claude-User@anthropic.comMatch on the token Claude-User rather than the whole string, since the version number inside the string can change. A user agent alone can be forged, so verify it as shown below.
robots.txt rules
To block it everywhere:
User-agent: Claude-User
Disallow: /To allow it explicitly, for example when a broader rule blocks other agents:
User-agent: Claude-User
Allow: /What blocking it changes
Anthropic says blocking it prevents its system from retrieving your content when a person asks Claude a question that needs it, which may reduce visibility in user-directed web search.
- It fetches individual pages on a user’s request rather than crawling on a schedule.
How to verify a request is genuine
Match the requesting IP against the crawler addresses Anthropic publishes. Source: Anthropic: crawler IP addresses.
Other Anthropic agents
ClaudeBot: model trainingClaude-SearchBot: search index
Sources
Anthropic's documentation for Claude-User, read on 15 Oct 2026. Operators change these pages without notice; check before relying on a detail.
Check how your site treats Claude-User
The AI crawler access check reads your robots.txt and fetches a URL as this agent, so you see what a CDN or firewall does as well.
Run the free check All AI crawlers