Content Signals

Content Signals is a draft convention, published by Cloudflare in 2025, for stating in robots.txt how a site’s content may be used. It adds a Content-Signal line with three yes/no values:

• search: building a search index and showing links and snippets.
• ai-input: using the content as input to a generated answer, such as an AI Overview or a chatbot response.
• ai-train: using the content to train or fine-tune models.

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=no

Status and limits

It is a policy statement, not an access control. No crawler vendor documents honouring it yet, and robots.txt Allow and Disallow rules still decide what can be fetched. Cloudflare can insert the line through its managed robots.txt feature, which is worth knowing because a CDN default can differ from the policy the site owner intended.

A practical note

Some validators, including Lighthouse’s robots.txt audit, report the directive as unknown. Our own robots.txt states the policy as a comment for that reason (search, AI input and AI training all allowed), and we keep Cloudflare’s managed robots.txt off so nothing at the CDN overrides it. The AI content signals tool reads a site’s robots.txt and reports what it declares.