Does Perplexity train AI models on pages PerplexityBot crawls?

Perplexity says PerplexityBot is not used to crawl content for AI foundation models. Its role is to surface and link websites in Perplexity's search results.

Will a robots.txt block stop Perplexity reading my page when a user asks about it?

Not reliably. Those fetches come from Perplexity-User, which Perplexity says generally ignores robots.txt because a user requested them. Only an IP or WAF rule using perplexity-user.json blocks it.

Why does the PerplexityBot IP list URL redirect?

The documented address on perplexity.com currently redirects to the same file on perplexity.ai. Either URL leads to the published ranges.

Do PerplexityBot and Perplexity-User use the same IP list?

No. Perplexity publishes perplexitybot.json for the crawler and perplexity-user.json for the user fetcher, so you can treat them differently at your firewall.

What happens if I block PerplexityBot?

Your site may not be surfaced or linked in Perplexity's search results. Since Perplexity says it isn't a training crawler, blocking it costs visibility without opting you out of anything else.

Perplexity · AI search

PerplexityBot: Perplexity's search crawler, and why to allow it

PerplexityBot is Perplexity's search crawler: it surfaces and links websites in Perplexity's search results, and Perplexity says it is not used to crawl content for AI foundation models. Perplexity recommends allowing it in robots.txt so your site can appear. Its sibling, Perplexity-User, fetches pages for individual questions and generally ignores robots.txt.

robots.txt token: PerplexityBot
User agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)
Operator: Perplexity
Type: AI search
What it feeds: Perplexity's search results, where it surfaces and links websites; Perplexity says it is not used for foundation-model training.
Follows robots.txt: Yes. Perplexity recommends allowing PerplexityBot in robots.txt so your site can appear in results, which implies it follows robots.txt; its docs make no more explicit statement than that.
How to verify it: Perplexity publishes the IP ranges PerplexityBot uses (list)

Should you block PerplexityBot?

Blocking PerplexityBot means your site may not be surfaced or linked in Perplexity's search results, and since it isn't a training crawler, there is no training opt-out to gain. Most businesses should allow it. Blocking it does not stop Perplexity-User, which only an IP or WAF rule controls.

Block PerplexityBot

User-agent: PerplexityBot
Disallow: /

Allow PerplexityBot

User-agent: PerplexityBot
Allow: /

Add these lines to /robots.txt at your site root. A group naming PerplexityBot replaces your User-agent: * rules for it, so repeat any paths you also want it to skip.

Live check

Is PerplexityBot allowed on your site?

Perplexity-User

Perplexity-User fetches a page when a Perplexity user asks a question, so Perplexity can answer and cite it; Perplexity says it is not used for general crawling or AI training. Because a user requested the fetch, Perplexity's docs say it generally ignores robots.txt, so blocking it only works at the IP or WAF level. Its IP ranges are published at https://www.perplexity.com/perplexity-user.json.

PerplexityBot and Perplexity-User

Perplexity documents two agents, and they behave very differently toward robots.txt:

Token	What it does	Honors robots.txt	IP list
PerplexityBot	Surfaces and links sites in Perplexity's search results	Implied yes	perplexitybot.json
Perplexity-User	Fetches a page to answer and cite it when a user asks	Generally ignores it	perplexity-user.json

Neither is described as a training crawler. Perplexity says PerplexityBot is not used to crawl content for AI foundation models, and that Perplexity-User is not used for general crawling or AI training.

What blocking PerplexityBot does, and what it doesn't

Blocking PerplexityBot means your site may not be surfaced or linked in Perplexity's search results. Unlike blocking GPTBot or ClaudeBot, there is no training opt-out on the other side of the trade, so for a business that wants to be recommended it is mostly a loss.

Perplexity's docs recommend allowing the bot so your site can appear, which implies it follows your rules, but they stop short of a plain "we honor robots.txt" statement. We have not assessed third-party reports about undeclared crawling, since they are not primary sources.

Perplexity-User: the fetcher that ignores robots.txt

When someone asks Perplexity a question, it may visit a page to answer and cite it, using Perplexity-User. Perplexity's docs say that since a user requested the fetch, this fetcher generally ignores robots.txt rules. The same docs also describe the token as a way to control which sites these requests can reach, which sits awkwardly with that; we go by the "generally ignores" wording.

So a robots.txt group for Perplexity-User is not a dependable block. If you need to refuse it, deny the ranges in perplexity-user.json at your firewall or WAF. Doing so stops Perplexity fetching and citing your pages in answers to user questions. Compare ChatGPT-User, where OpenAI says robots.txt "may not apply", and Claude-User, where Anthropic says it does.

How to allow PerplexityBot

User-agent: PerplexityBot
Allow: /

A named group overrides your User-agent: * rules for PerplexityBot, so copy any private paths into it. robots.txt is per host, so each subdomain needs its own file. Also check that your CDN's bot protection lets the published IPs through.

Checking that a visit is really PerplexityBot

The documented string is Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot); Perplexity-User's is the same with Perplexity-User/1.0 and a perplexity-user link. Compare the source IP with the matching list. The documented perplexity.com URLs currently redirect to the same files on perplexity.ai.

The live check above sends a request with PerplexityBot's token and compares it with a browser request, which catches a WAF or CDN block. It doesn't come from Perplexity's IPs, so IP-keyed rules may differ. The robots.txt checker shows how your rules resolve for each token.

Questions

Does Perplexity train AI models on pages PerplexityBot crawls?: Perplexity says PerplexityBot is not used to crawl content for AI foundation models. Its role is to surface and link websites in Perplexity's search results.
Will a robots.txt block stop Perplexity reading my page when a user asks about it?: Not reliably. Those fetches come from Perplexity-User, which Perplexity says generally ignores robots.txt because a user requested them. Only an IP or WAF rule using perplexity-user.json blocks it.
Why does the PerplexityBot IP list URL redirect?: The documented address on perplexity.com currently redirects to the same file on perplexity.ai. Either URL leads to the published ranges.
Do PerplexityBot and Perplexity-User use the same IP list?: No. Perplexity publishes perplexitybot.json for the crawler and perplexity-user.json for the user fetcher, so you can treat them differently at your firewall.
What happens if I block PerplexityBot?: Your site may not be surfaced or linked in Perplexity's search results. Since Perplexity says it isn't a training crawler, blocking it costs visibility without opting you out of anything else.

Related crawlers

Sources

Checked against these pages on 2026-10-18. Operators change their crawlers; check the source before relying on a detail.