Can AI find your site?
We check robots.txt, sitemap discovery, and whether your CDN/WAF is letting documented AI crawlers reach your pages.
Every audit ends with a link to this page. It shows what we checked, where each rule comes from, and how confident we are in it. Forward this when you want to send the framework — not a specific run — to a client or developer.
Methodology last reviewed: 2026-05-16.
Three rollups, ten check categories. Each finding maps to a source and a confidence level you'll see on the right of every report.
We check robots.txt, sitemap discovery, and whether your CDN/WAF is letting documented AI crawlers reach your pages.
We check headings and page structure, structured data, and whether the main content is reachable without running JavaScript.
We check trust signals (HTTPS, author markup, outbound citations, freshness) and content shape that nudge an AI assistant toward picking you when it answers.
We don't make up rules. Every check is grounded in one of four source types. Findings link back to the exact doc they came from.
Crawler names, user agents, and opt-out mechanisms come straight from each provider's own documentation.
Checks like robots.txt parsing, structured data, and sitemap discovery follow the documented protocol — not heuristics.
Worth knowing about, but not yet officially adopted. We surface these as low-effort possible-future wins, not as critical fixes.
Not yet officially adopted by major AI companies. The audit reports this as a low-effort, possible-future-benefit recommendation, not as critical.
Content-structure recommendations and trust-signal heuristics lean on published research and reporting from infrastructure providers, not provider docs.
Every finding carries one of three confidence levels so you can tell at a glance which recommendations are based on documented standards and which are based on observed behaviour or emerging practice.
We record exact provenance with every report so anyone receiving it can re-run, verify, or just trust where the numbers came from.