How much does it cost an AI to read my page?
Every page costs a machine tokens to read, and a page that costs a great deal risks being cut short or skipped for a cheaper source. We estimate that cost from the page as served and, where your site hands a Markdown version to agents that ask, from that. It is a rough guide, and it can only warn.
What it actually is
We take the HTML document we fetched and estimate how many tokens a language model would spend reading it, using the same rule of thumb the model vendors publish for ordinary text. Where the site also serves a Markdown version to an agent that asks for one, we use that version too and judge the cheaper of the two, because an agent can choose it. The result is an estimate, not a count from any particular model.
Why it matters
A tool that fetches a page for an AI normally sets a ceiling on how much it will take in, and what lies beyond the ceiling is dropped. Anthropic documents exactly that for its own fetch tool. A page that is mostly markup, script and repeated menus spends that allowance before the useful text arrives, while the same content as clean Markdown can cost a fraction of the tokens. Cloudflare publishes one such comparison for its own blog post.
What good looks like
A page whose substance sits in a modest amount of text, with little markup wrapped around it, or a Markdown version that agents can ask for. Long reference documents are fine when they are split into pages that each cover one thing, so no single fetch has to carry the whole.
Being honest about it
This is an estimate from the size of the document, not a measurement of any model, and models count differently, especially for code, numbers and languages that are not written in the Latin alphabet. The line we warn at is a marker for a very large page, not a limit that any tool publishes. A long page is not a broken one, so the check only ever passes or warns, and because it is new it sits under measured, not scored: it is shown in your report and never moves your score.
Where most sites go wrong
Pages weighed down by inline scripts, styles and data that a machine has to wade through to reach the words. The other is one enormous page that tries to hold an entire manual, which a fetching tool is most likely to truncate at the point where the answer was.
On the web
State of the Web · November 2026In November 2026, 76% of the sites where it applies (3% of the sites we could read) passed the “Reading the page is affordable for a machine” check.
Not enough history yet.
Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.
Not sure where yours stands?
A free scan checks this and everything else in about 20 seconds — no signup. Or if it's already flagged and you'd rather it were simply handled, we do that too.
Flagged for this one? We handle it. No pitch, just a plain-English chat.
Book a call