Do I still need an XML sitemap?
Yes, though it does less than people think. A sitemap does not make pages rank — it makes them findable, which matters most for pages nothing links to and for sites large enough that crawling everything is not automatic.
What it actually is
It is a machine-readable list of the pages you would like known about, with a note of when each last changed. Crawlers normally discover pages by following links, which works well for anything well-linked and badly for anything new, deep or recently added. The sitemap is the shortcut past that.
Why it matters
The gap it closes is time. Without one, a new page waits to be discovered through whatever links happen to point at it; with one, it is on a list that gets checked. For a site publishing regularly, or one where important pages sit several layers down, that difference compounds — and the pages that benefit most are usually the newest, which are also the ones you most want seen.
What good looks like
A valid sitemap at a predictable address, referenced from robots.txt so nothing has to guess where it lives, listing the canonical version of every page you want found — and only those. Every URL in it resolves successfully, and pages you have excluded from indexing are not listed in it.
Being honest about it
For a small, well-linked site this is genuinely close to optional, and anyone presenting it as urgent is overselling. Its real value scales with size and publishing frequency. The one case where it matters regardless of size is a site with pages that nothing links to internally, which is more common than people expect — landing pages and seasonal content especially.
Where most sites go wrong
The most common failure is staleness: a sitemap generated once, never regenerated, now listing pages that no longer exist while omitting everything published since. The second is contradiction — the sitemap lists a page while the page itself asks not to be indexed, or lists one host variant while the site redirects to another. Mixed signals get resolved by the engine ignoring the file, which wastes the whole thing.
On the web
State of the Web · November 2026In November 2026, 75% of the sites we could read passed the “XML sitemap is present and referenced” check.
Shopify sites pass it most often, at 100%. WordPress sites pass it least often, at 77%.
On the sites scanned in both August 2026 and November 2026, it went from 68% to 66%.
The month-by-month line counts the sites scanned in each calendar month, so it can differ from the figure above, which uses each site’s latest scan from the last three months.
Goes with
-
Sites that pass this are 3 times as likely to pass the “AI bot rules are explicit and intentional” check too. 15% of the sites that pass the first check pass this one, against 5% of the sites that do not.
-
Sites that pass this are twice as likely to pass the “Images are optimised” check too. 14% of the sites that pass the first check pass this one, against 7% of the sites that do not.
-
Sites that pass this are twice as likely to pass the “Open Graph tags for sharing” check too. 24% of the sites that pass the first check pass this one, against 13% of the sites that do not.
Not enough history yet.
Based on the sites Bot Appétit has scanned. Most of these sites come from lists we scan for research. Not a random sample of the web.
Not sure where yours stands?
A free scan checks this and everything else in about 20 seconds — no signup. Or if it's already flagged and you'd rather it were simply handled, we do that too.
Flagged for this one? We handle it. No pitch, just a plain-English chat.
Book a call