Bot Appétit the free AEO + SEO readiness check
Scan
Research

Astro AI readiness statistics

In November 2026, we could only tell which sites run on Astro where our scanner could read the page, so this page covers those sites and says nothing about how many turned our scanner away.

Updated 21:07 UTC · November 2026

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

All sites on the State of the Web

How Astro sites land

Every page we read gets a score out of 100, the number at the top of its report (how scoring works). Each row is a range of scores. Where too few sites score in the lowest ranges, they are joined into one row.

  • 91–100 56%
  • 0–90 44%
Show as a table
ScoreShare of sites
91–10056%
0–9044%

A platform is only detectable on a page our scanner could read, so these shares are of the Astro sites we could read.

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

Better or worse

Not enough history yet.

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

AI access

  • robots.txt disallows some AI crawlers 33%
  • robots.txt disallows major AI crawlers 2%
Show as a table
What we foundShare of sites
robots.txt disallows some AI crawlers33%
robots.txt disallows major AI crawlers2%

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

What Astro sites get right and wrong

Passed more often than on the average site: “Open Graph tags for sharing” (69%, against 26% of all sites); “Images are optimised” (51%, against 14% of all sites); “Semantic HTML landmarks” (87%, against 52% of all sites). Least often passed: several checks that no Astro site we could read passed.

Every check, Astro against all sites

Share of sites that pass each check, over the sites the check applies to. The Gap column is the difference in percentage points between the two shares; a dash means not enough sites yet.

Check Astro sites All sites Gap (points)
Served over a secure HTTPS connection 100% >99% Level
HTTP redirects to HTTPS 97% 94% +3 points
No mixed (insecure) content 100% 99% +1 point
Core security headers are set 33% 11% +22 points
Not flagged for malware or phishing 100% 100% Level
No server version leakage 96% 93% +3 points
security.txt is published 15% 5% +10 points
Reached without a redirect chain 99% 98% +1 point
Caching or validation headers are set 92% 94% −2 points
The site settles on one host (www vs non-www) 88% 87% +1 point
robots.txt is present and not over-blocking 97% 90% +7 points
Not accidentally set to noindex 100% 99% +1 point
This page declares its canonical URL 84% 60% +24 points
Mobile viewport is set 98% 93% +5 points
Has a title tag 100% >99% Level
Title fits a search snippet 61% 57% +4 points
Has a meta description 100% 91% +9 points
Description fits a search snippet 69% 64% +5 points
The page has an H1 76% 84% −8 points
Logical heading order 74% 63% +11 points
Topic is consistent across title, H1 and description 64% 62% +2 points
Images have descriptive alt text 91% 94% −3 points
Your page's HTML isn't oversized 99% 99% Level
Open Graph tags for sharing 69% 26% +43 points
Twitter/X card tags 79% 73% +6 points
XML sitemap is present and referenced 83% 74% +9 points
Missing pages return a real 404 88% 91% −3 points
Images are optimised 51% 14% +37 points
Language and charset declared 100% 87% +13 points
hreflang for language variants 100% 99% +1 point
Analytics is installed 67% 76% −9 points
Search Console / Bing verification 92% 82% +10 points
A favicon is set 99% 98% +1 point
No meta-refresh redirect 99% >99% −1 point
Your content survives without JavaScript 98% 96% +2 points
Your answer is in the HTML 98% 95% +3 points
Buttons, links and fields have names 65% 32% +33 points
Semantic HTML landmarks 87% 52% +35 points
Content is scannable in self-contained chunks 91% 93% −2 points
Claims come with figures, quotes or sources 58% 56% +2 points
Section headings are phrased as questions 28% 21% +7 points
Reading the page is affordable for a machine 68% 89% −21 points
No hidden text written for an AI model 100% >99% Level
Sitemap dates tell a machine what changed 52% 67% −15 points
The links in llms.txt resolve 94% 85% +9 points
Author / publisher is clear 56% 49% +7 points
A machine-readable published / updated date 25% 12% +13 points
llms.txt is present and valid 37% 27% +10 points
Clean content for agents (Markdown) 10% 4% +6 points
Structured data is present 70% 55% +15 points
Structured data is valid 98% 97% +1 point
FAQ / Q&A schema — 47% —
Organization schema with sameAs 46% 32% +14 points
Right schema type for the page 82% 93% −11 points
Schema facts also appear in the page text — 56% —
Breadcrumb structured data — 27% —
AI crawlers aren't blocked 65% 90% −25 points
Automated visitors can reach your pages — — —
Engines are allowed to quote the page 92% 99% −7 points
Your page arrives quickly 98% 95% +3 points
AI bot rules are explicit and intentional 42% 14% +28 points
Content usage preferences declared 22% 4% +18 points
Resource-discovery Link headers 11% 2% +9 points
No broken links 92% 86% +6 points
Links don't point at redirects 84% 91% −7 points
Descriptive anchor text 100% 99% +1 point
Sensible internal/external link mix 99% 96% +3 points
Links are real links 91% 89% +2 points
Largest Contentful Paint (LCP) — — —
Cumulative Layout Shift (CLS) — — —
Interaction to Next Paint (INP) — — —
Page weight and render-blocking — — —
MCP Server Card 7% 1% +6 points
AI catalog 2% 1% +1 point
A2A Agent Card 3% <1% +3 points
WebMCP tools in the page 1% 1% Level
AI usage preferences (Content-Usage) 0% <1% Level
Licensing terms for AI (RSL) 0% <1% Level
Text-and-data-mining reservation (TDMRep) 0% <1% Level
Markdown version linked from the page 6% 1% +5 points
Content signals agree everywhere — — —
Agent Skills index 8% 1% +7 points
API catalog 8% 1% +7 points
OAuth authorization-server metadata 6% 3% +3 points
OAuth protected-resource metadata 4% 2% +2 points
OpenID Connect discovery 2% 2% Level
Web Bot Auth (operates a signed bot) 0% <1% Level
Universal Commerce Protocol manifest — 6% —
Auth.md agent registration 4% <1% +4 points
AI discovery via DNS 0% <1% Level

Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.

See how your site compares

A free scan checks all of this on your page in seconds, with no signup.

Scan your site free

Want it handled? We do that too. No pitch, just a plain-English chat.

Book a call