How AI assistants and AI Overviews choose, describe and cite sources, and what you can change.
What Anthropic publishes about its three crawlers, robots.txt, its IP list, web search in the Claude apps and the web search tool in the Claude API, what it leaves open about how sources are chosen, and a checklist for being eligible as a cited source.
What Google publishes about the sources behind answers in the Gemini app: grounding in Google Search, the Google-Extended token and what it does not control, the Sources button, and the questions its documentation leaves open. With a checklist.
What Microsoft publishes about Bingbot, grounding, robots meta tags and IndexNow for Copilot, what it leaves open about source selection, and a checklist.
What Perplexity publishes about its crawlers, robots.txt, firewall allowlisting and its new source labels, what it leaves open about how sources are chosen, and a checklist for being eligible as a cited source.
Original data from eight young sites we measure: how much of their Google visibility happened inside AI features, which pages were shown, and how often Bing and Google moved in opposite directions. With the monthly table to download and the limits stated.
What OpenAI publishes about getting a site into ChatGPT search: the crawler that matters, the robots.txt and CDN settings, noindex, referral tracking, and the questions its documentation leaves open. With a checklist and data from eight sites.
Google has published direct guidance on its generative AI features. Here is what it says you need, what it says you do not, and a checklist built only from those pages.