What is my user agent?

It is the string your browser sends with every request, something like Mozilla/5.0 followed by your operating system and browser version. Modern browsers send a reduced version that fixes the platform version and zeroes minor version numbers for privacy.

What is the difference between a user agent string and a user agent token?

The string is the full header value. The token is the short product name inside it, such as GPTBot or Googlebot, which is what you write in a robots.txt User-agent line.

Can a bot fake its user agent?

Yes. Anyone can send any User-Agent value. Google, OpenAI and others publish IP ranges or DNS checks so you can verify that a request claiming to be their crawler really came from them.

Should I serve different content based on user agent?

Generally no. MDN warns that serving different content based on the user agent is usually a bad idea. Use it for logging, rate limits and robots.txt rules instead.

AI search glossary

User agent: what the string means, and how AI crawlers identify themselves

A user agent is the software that makes a web request on someone's behalf, such as a browser or a crawler, and the User-Agent string is the HTTP request header in which that software names itself, its version and often its platform.

Also called: User-Agent header, user agent string, UA string

Why it matters for AI search

Every AI crawler announces itself through its user agent, and robots.txt rules are written against the product token inside it (GPTBot, OAI-SearchBot, ClaudeBot and so on). Knowing which token belongs to which job lets you block model training while still allowing the search crawlers that decide whether AI answers can cite you. Because the string can be faked, serious decisions should also check the operator's published IP ranges.

How a User-Agent string is built

MDN gives the general syntax as a product name and version followed by comments:

User-Agent: <product>/<product-version> <comment>

Browsers start with Mozilla/5.0 for historical compatibility, then add platform details and engine names. A desktop Chrome string looks like this:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36

Browsers now send a reduced string: the platform version and device model are fixed and minor version numbers are zeroed. Sites that need more detail can request it through User-Agent Client Hints (Sec-CH-UA-* headers).

How crawlers use it

Crawlers put a product token and a contact URL in their string. Examples from OpenAI's own documentation:

Token	Job	String ends with
GPTBot	Collects content that may train OpenAI models	`GPTBot/1.4; +https://openai.com/gptbot`
OAI-SearchBot	Surfaces sites in ChatGPT search	`OAI-SearchBot/1.4; +https://openai.com/searchbot`
ChatGPT-User	Fetches a page when a user asks ChatGPT	`ChatGPT-User/1.0; +https://openai.com/bot`

robots.txt matches the token, not the full string:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

That pair blocks training while allowing ChatGPT search. OpenAI notes that ChatGPT-User visits are user-initiated, so robots.txt rules may not apply to them. The full list of AI tokens and what each one does is in our AI crawler directory, and the robots.txt checker shows how your file treats each one.

Verifying a crawler is real

Because any client can claim to be Googlebot, Google says its crawlers identify themselves three ways: the user agent, the source IP and the reverse DNS hostname. To verify:

Run a reverse DNS lookup on the IP from your logs.
Check the hostname ends in googlebot.com, google.com or googleusercontent.com.
Run a forward lookup on that hostname and confirm it returns the same IP.

For large volumes, compare IPs against Google's published JSON range files. OpenAI publishes similar lists, such as searchbot.json and chatgpt-user.json. A request that claims an AI crawler's user agent but comes from an unlisted IP should be treated as an ordinary unknown bot.

Where this shows up in practice

Server logs, CDN bot rules and analytics filters all key off the user agent. When AI referrals or crawl activity look odd, the first check is which user agents actually hit your pages, and whether the ones that matter for agentic search and AI citations are allowed through.

Questions

What is my user agent?: It is the string your browser sends with every request, something like Mozilla/5.0 followed by your operating system and browser version. Modern browsers send a reduced version that fixes the platform version and zeroes minor version numbers for privacy.
What is the difference between a user agent string and a user agent token?: The string is the full header value. The token is the short product name inside it, such as GPTBot or Googlebot, which is what you write in a robots.txt User-agent line.
Can a bot fake its user agent?: Yes. Anyone can send any User-Agent value. Google, OpenAI and others publish IP ranges or DNS checks so you can verify that a request claiming to be their crawler really came from them.
Should I serve different content based on user agent?: Generally no. MDN warns that serving different content based on the user agent is usually a bad idea. Use it for logging, rate limits and robots.txt rules instead.