AI search glossary
XML sitemap: format, limits and how search engines use it
An XML sitemap is a file, usually at /sitemap.xml, that lists the URLs on a website a search engine should crawl, optionally with the date each page last changed, following the protocol published at sitemaps.org.
Also called: sitemap, sitemap.xml, sitemap index
Why it matters for AI search
AI answers that search the web can only cite pages that a search index has found and stored. A sitemap is the most direct way to hand crawlers your full list of important URLs, including new pages that have few internal links yet. An accurate lastmod date also tells crawlers which pages changed, which helps updated prices and facts reach the indexes AI engines draw from sooner.
What a sitemap looks like
The sitemaps.org protocol defines a short XML file. Three tags are required: urlset wraps the file, each page gets a url entry, and loc holds its full address. lastmod, changefreq and priority are optional.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://www.example.com/grinder-g2</loc>
<lastmod>2026-10-12</lastmod>
</url>
</urlset>
The file must be UTF-8, and characters such as ampersands must be escaped. URLs must be under 2,048 characters.
Limits and sitemap index files
Each sitemap can hold up to 50,000 URLs and 50 MB uncompressed, and gzip compression is allowed. Past that, split the list and create a sitemap index: a file with a sitemapindex root and one sitemap entry (with its own loc) per child file. An index can reference up to 50,000 sitemaps, but only sitemaps on the same site.
What Google actually reads
Google's documentation is direct about this:
- lastmod is used if it is consistently and verifiably accurate. Changes to main content, structured data or links count; a new copyright year does not.
- priority and changefreq are ignored.
- Submission is a hint. Google does not promise to download or use every sitemap.
So the useful work is keeping the list clean: only canonical, indexable URLs that return 200. Listing a URL in a sitemap is itself a weak canonical signal, so leave out duplicates and redirected URLs (see canonical tag).
What to include and leave out
Include every page you want found and cited: product and category pages, service and pricing pages, guides, location pages and your about page. Leave out URLs that redirect, return errors, carry noindex, or are filtered and sorted versions of another page. A sitemap full of those wastes crawl requests and blurs your canonical signals.
Update lastmod only when the page's main content changes. If every URL shows today's date on every build, the field stops meaning anything and Google says it uses lastmod only when it is verifiably accurate. For sites with tens of thousands of pages, one sitemap per section (products, articles, locations) makes errors easier to find in Search Console.
How to submit a sitemap
-
robots.txt. Add a line anywhere in the file. Every crawler that reads robots.txt, including AI crawlers, can find it:
Sitemap: https://www.example.com/sitemap.xml -
Search Console. Submit it in the Sitemaps report to see when Google fetched it and any errors.
-
For faster updates to Bing and others, pair the sitemap with IndexNow, which pushes changed URLs as they happen.
Most platforms generate a sitemap for you. Where it lives differs by platform: see WordPress, Shopify or Next.js. You can confirm your robots.txt references it with the robots.txt checker.
Questions
- How do I find a site's sitemap?
- Try /sitemap.xml, then look for a Sitemap line in /robots.txt. Many platforms use other paths, such as /wp-sitemap.xml on WordPress or a sitemap index that links to several child sitemaps.
- How many URLs can a sitemap contain?
- Up to 50,000 URLs and 50 MB uncompressed per file. Larger sites split their URLs across several sitemaps and list them in a sitemap index file.
- Does Google use priority and changefreq?
- No. Google says it ignores both values. It uses lastmod only when the dates are consistently and verifiably accurate.
- Does submitting a sitemap guarantee indexing?
- No. Google calls a sitemap a hint. It helps discovery, but each URL still has to be crawlable, indexable and worth indexing.
- What is the difference between an XML sitemap and an HTML sitemap?
- An XML sitemap is written for crawlers and follows the sitemaps.org protocol. An HTML sitemap is an ordinary page of links for visitors. Search engines can follow links on both, but only the XML version can be submitted in Search Console.