AI search optimization
Be found and described accurately by assistants
- Access
Stage 1: Access
AI content signals inspector
Every machine-readable usage signal a site publishes for AI systems: present, absent or conflicting, with who documents honouring it.
Free to use. No login, nothing stored.
What it checks
Access- robots.txt Content-Signal lines (search, ai-input, ai-train) per user-agent group
- Snippet controls in meta robots and X-Robots-Tag: nosnippet, max-snippet, max-image-preview, noarchive, noimageindex
- data-nosnippet sections and the words inside them
- noai and noimageai values, and whether any vendor documents them
- TDM Reservation Protocol: the rules in /.well-known/tdmrep.json and which one applies to the page
- ai.txt at the site root, and conflicts between signals and robots.txt blocks
Method question for this stage: Can crawlers reach and render every key page?
Run the check
Results
No check run yet
- Info
Five groups: Content-Signal, snippet controls, TDM Reservation Protocol, ai.txt, consistency
Each signal is reported as present, absent or conflicting, with the rule it tests, the documentation it comes from and who documents honouring it. Absent files are Info rows, not failures: none of these signals is required.
How we check
The rules this tool tests, and where they come from
A result is a check against a published rule, not a prediction. Every row shows its working.
Rules and sources
- Cloudflare: Content Signals Policy (blog, September 2025)
- Cloudflare: managed robots.txt and content signals
- Google: robots meta tag, data-nosnippet and X-Robots-Tag
- W3C Community Group: TDM Reservation Protocol (final report, May 2024)
Links go to the operator's or standards body's own documentation. We do not link to third-party summaries.
Read more
Next step
Fix what it finds
This check belongs to the Access stage of the Crawl-to-Citation Method. The service below covers the fix; the Diagnostic ranks it against everything else.
Most engagements start with the Search Visibility Diagnostic ($2,500).
It covers all six method stages, so what this tool finds is ranked against everything else before you commit to any work.
Book the diagnosticWhat is the difference between blocking a user agent and signalling a policy?
A robots.txt Disallow line tells a named crawler not to fetch a path; the operators this site links (Google, OpenAI, Anthropic, Perplexity) document that their crawlers read it, so it is the one control with documented effect. A signal such as Content-Signal, tdmrep.json or ai.txt says how content may be used after it is fetched, and leaves access open. Cloudflare writes that content signals "are not technical countermeasures against scraping"; no AI crawler documentation we link mentions reading them yet. Use a Disallow for access you want refused and a signal for a preference you want on record.
Are these signals legally binding?
This tool reports whether a signal is present, absent or in conflict with another. It gives no legal advice. The Content Signals Policy text and the TDM Reservation Protocol both refer to Article 4 of EU Directive 2019/790, which lets rights holders reserve text and data mining in a machine-readable way; what that means for your site, in your jurisdiction, is a question for a lawyer.
Should I add nosnippet to keep my content out of AI answers?
Only if you accept the trade-off. Google documents one directive for both: nosnippet removes the text snippet and video preview from your results in web search, Images, Discover, AI Overviews and AI Mode, and it also stops the content being used as direct input for AI Overviews and AI Mode. There is no documented value that keeps the search snippet and drops the AI use. data-nosnippet lets you exclude a section instead of the whole page, which is often the better tool for a specific passage.
Last reviewed