Insights

Articles on SEO, AI search and measurement. Each one states its sources and when it was last updated.

Technical SEO

How to find AI crawlers in your server logs, and verify them

What an access log line contains, how to match the user agent tokens OpenAI, Anthropic, Perplexity, Google, Bing, Apple and Common Crawl document, how to confirm a request is genuine with published IP ranges or reverse DNS, what to measure, and what logs cannot tell you.

11 min read

AI search visibility

How Claude chooses sources: what Anthropic documents, and what it does not

What Anthropic publishes about its three crawlers, robots.txt, its IP list, web search in the Claude apps and the web search tool in the Claude API, what it leaves open about how sources are chosen, and a checklist for being eligible as a cited source.

7 min read

AI search visibility

How Gemini chooses sources: what Google documents, and what it does not

What Google publishes about the sources behind answers in the Gemini app: grounding in Google Search, the Google-Extended token and what it does not control, the Sources button, and the questions its documentation leaves open. With a checklist.

8 min read

Measuring AI visibility

Bing Webmaster Tools' AI Performance report, explained

What Microsoft's AI Performance report counts across Copilot and Bing's AI summaries, the exact definition of each metric, which part is a sample, and how to turn it into decisions about specific pages.

4 min read