Amazon · Model training
Amazonbot: Amazon's general-purpose crawler, and how to block it
Amazonbot is Amazon's general crawler: Amazon says it is used to improve its products and services and may be used to train Amazon AI models, so it is mixed-use rather than purely a training bot. It follows robots.txt Allow and Disallow rules but not Crawl-delay, may use a robots.txt cached for up to 30 days, and takes about 24 hours to reflect changes. Search eligibility, such as Alexa answers, is handled by a separate bot, Amzn-SearchBot.
- robots.txt token
Amazonbot- User agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36- Operator
- Amazon
- Type
- Model training
- What it feeds
- Improvement of Amazon's products and services, including more accurate information for customers, and possibly training of Amazon AI models.
- Follows robots.txt
- Yes. Amazon says Amazonbot honors robots.txt user-agent and allow/disallow rules, using a copy cached for up to 30 days, plus rel=nofollow and the noarchive, noindex and none meta tags. It does not support crawl-delay.
- How to verify it
- Match the request IP against Amazon's published Amazonbot IP addresses (list)
Should you block Amazonbot?
Blocking Amazonbot keeps your content out of Amazon's general product improvement and possible AI model training. Amazon's page suggests allowing it can make you eligible for Amazon Content Partners benefits, so a block may cost that. It does not affect Amzn-SearchBot, which decides whether you can appear in Amazon search experiences such as Alexa.
User-agent: Amazonbot
Disallow: /
User-agent: Amazonbot
Allow: /
Add these lines to /robots.txt at your site root. A group naming Amazonbot replaces your User-agent: * rules for it, so repeat any paths you also want it to skip.
Live check
Is Amazonbot allowed on your site?
Amazonbot, Amzn-SearchBot and Amzn-User
Amazon documents three user agents, and each robots.txt setting is independent of the others:
| Token | What it does | Block it if… |
|---|---|---|
| Amazonbot | Crawls to improve Amazon products and services; may train Amazon AI models | You don't want general Amazon use or possible AI training |
| Amzn-SearchBot | Crawls for search experiences in Amazon products, such as Alexa; not for generative AI training | You don't want to appear in Amazon search experiences |
| Amzn-User | Fetches live pages for user actions, such as an Alexa question needing current information | You don't want Alexa reading pages on a user's behalf (it may not follow every robots.txt rule) |
Amazon says neither Amzn-SearchBot nor Amzn-User crawls content for generative AI model training. That leaves Amazonbot as the only one of the three whose output may train Amazon's models.
What blocking Amazonbot does, and what it doesn't
A Disallow for Amazonbot keeps your pages out of whatever Amazonbot feeds: Amazon's general product improvement and any AI model training that draws on it. Amazon's page doesn't list the specific products, so the exact downstream effect is not documented.
It doesn't change Amzn-SearchBot's access, so Alexa search eligibility is decided by your Amzn-SearchBot rules. It also doesn't bind Amzn-User fully, since Amazon says user-initiated fetches may not follow all robots.txt directives. One more trade-off: Amazon advertises Amazon Content Partners benefits (an affiliate commission boost, hosting credits and traffic tools) for sites that allow Amazonbot.
If training is your only concern, Amazon offers a page-level alternative: the noarchive robots meta tag, which Amazon defines as "do not use the page for model training."
How to block Amazonbot
At the root of each host:
User-agent: Amazonbot
Disallow: /
Quirks from Amazon's page:
- No crawl-delay. Amazon's bots ignore
Crawl-delay. - Caching. They fetch your robots.txt or use a cached copy from the last 30 days, and changes take about 24 hours to show up.
- Per host. Amazon reads robots.txt at each host, so
site.example.comneeds its own file. - Unreachable file. If robots.txt can't be fetched, they behave as if it doesn't exist, which means no restrictions.
Verifying a real Amazonbot visit
Amazon publishes the IP addresses Amazonbot uses; compare the request's source IP with that list. Amazon's page names no reverse DNS method. The documented user agent is labeled an example string, and W.X.Y.Z is a version placeholder, so match on Amazonbot rather than the full string. You can test your rules for all three Amazon agents with the robots.txt checker.
Questions
- Is Amazonbot a training crawler?
- Partly. Amazon says it is used to improve Amazon's products and services and may be used to train Amazon AI models. Its stated purpose is broader than training, so treat it as mixed-use.
- Will blocking Amazonbot keep me out of Alexa answers?
- Not by itself. Amazon says Amzn-SearchBot is what makes content eligible for search experiences such as Alexa, and each Amazon user agent setting is independent.
- How long until Amazonbot sees my robots.txt change?
- Amazon says changes may take about 24 hours to be reflected, and its bots may use a cached robots.txt from the last 30 days, so allow for a delay before judging a new rule.
- Can I opt a single page out of model training without blocking Amazonbot?
- Yes. Amazon says its bots respect the noarchive robots meta tag, which it defines as do not use the page for model training. noindex and none are honored too.
- Does Amazonbot support Crawl-delay?
- No. Amazon states its user agents do not support the crawl-delay directive, so use Allow and Disallow rules or contact amazonbot@amazon.com about crawl issues.
Related crawlers
Sources
Checked against these pages on 2026-10-18. Operators change their crawlers; check the source before relying on a detail.