Microsoft · AI search
Bingbot: Microsoft's search crawler, and how to control it
Bingbot is Microsoft's standard web crawler for Bing: it handles most of Bing's crawling each day and builds the Bing search index. It follows robots.txt, and Bing Webmaster Tools also lets you set its crawl rate by the hour. Blocking it takes your site out of Bing search. Bing's crawler documentation does not say whether Bingbot's data feeds Copilot or other Microsoft AI answers.
- robots.txt token
Bingbot- User agent
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36- Operator
- Microsoft
- Type
- AI search
- What it feeds
- The Bing search index: Bing calls Bingbot its "standard crawler" that "handles most of our crawling needs each day".
- Follows robots.txt
- Yes. Bing says robots.txt files "can be configured to tell Bing crawlers how to interact with your website." You can also set crawl rate by the hour with the Crawl Control tool in Bing Webmaster Tools.
- How to verify it
- Bing's Verify Bingbot tool, reverse DNS to a search.msn.com hostname plus a forward lookup, or Bing's published IP list (list)
Should you block Bingbot?
Blocking Bingbot removes your site from Bing's index, so you lose Bing search visibility and whatever else is built on that index. Bing's docs don't say which Microsoft AI products use the index, so treat any effect beyond Bing search as unknown. If crawl load is the problem, use Crawl Control in Bing Webmaster Tools before reaching for a block.
User-agent: Bingbot
Disallow: /
User-agent: Bingbot
Allow: /
Add these lines to /robots.txt at your site root. A group naming Bingbot replaces your User-agent: * rules for it, so repeat any paths you also want it to skip.
Live check
Is Bingbot allowed on your site?
Bingbot and Bing's other crawlers
Bing lists several crawlers. Bingbot does most of the work; the others have narrow jobs:
| Crawler | What it does | Token in its user agent |
|---|---|---|
| Bingbot | Bing's standard crawler, building the search index | bingbot/2.0 |
| AdIdxBot | Crawls ads and the sites they link to for Bing Ads quality control | adidxbot/2.0 |
| BingPreview | Generates page snapshots for Bing | bingbot/2.0 (same strings as Bingbot) |
| MicrosoftPreview | Generates page snapshots for Microsoft products | MicrosoftPreview/2.0 |
| BingVideoPreview | Builds video previews shown only in Bing | BingVideoPreview/1.0 |
Nothing in this list is described as an AI-training or AI-answer crawler. Bing's crawler docs simply don't address whether Bingbot's index feeds Copilot, and this page doesn't assume it does.
User agent strings
Bing documents three Bingbot strings: the desktop one in the frontmatter, a short form Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) W.X.Y.Z Safari/537.36, and a mobile one:
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
W.X.Y.Z is replaced by whatever Microsoft Edge version Bing's renderer currently runs, so it changes regularly.
What blocking Bingbot does, and what it doesn't
A full Disallow for Bingbot tells Bing not to crawl your site, which drops it from Bing's search index and from anything else that reads that index. It is a heavy step: Bing's docs offer a lighter one, the Crawl Control tool in Bing Webmaster Tools, which sets crawl rate by the hour. AdIdxBot and MicrosoftPreview carry their own names in their user agents, and Bing's overview doesn't spell out which robots.txt group each of them reads, so don't assume a Bingbot rule covers them.
How to block or allow Bingbot
At the root of each host:
User-agent: Bingbot
Disallow: /
The frontmatter uses Bingbot, while Bing's docs write it lowercase. robots.txt matching on the user-agent name is case-insensitive, so bingbot and Bingbot behave the same. As with any named group, a Bingbot group replaces your User-agent: * rules for Bingbot, so copy shared Disallow lines into it if you are only blocking part of the site.
Verifying a real Bingbot visit
User agents are easy to fake, and Bing says so. It offers three checks:
- The Verify Bingbot tool: paste an IP and Bing tells you whether it belongs to Bingbot. It is public, and also available inside Bing Webmaster Tools.
- A reverse DNS lookup that should return a
search.msn.comhostname, followed by a forward lookup to confirm it resolves to the same IP. - Matching the IP against bingbot.json, Bing's published list.
Note that Bing's help pages render in the browser with JavaScript, so a plain fetch of the URL may come back nearly empty. You can test how your robots.txt treats Bingbot with the robots.txt checker.
Questions
- Should I write Bingbot or bingbot in robots.txt?
- Either works. robots.txt user-agent matching is case-insensitive, and Bing's own docs write the token in lowercase (bingbot/2.0 in the user agent string).
- Does blocking Bingbot remove me from Copilot?
- Bing's crawler documentation doesn't say. It describes Bingbot as the crawler for Bing's index and does not mention Copilot or other AI answers, so the effect of a block on Microsoft's AI products is not documented.
- What does W.X.Y.Z mean in the Bingbot user agent?
- Bing updates its rendering engine to the latest stable Microsoft Edge, and W.X.Y.Z stands for that Edge version (Bing's example is 80.0.345.0). Match on bingbot/2.0 rather than a full string.
- Can I slow Bingbot down without blocking it?
- Yes. Bing Webmaster Tools has a Crawl Control tool that lets you set Bingbot's crawl rate by the hour, so you can push crawling to quiet periods.
- Why do BingPreview requests look exactly like Bingbot?
- Bing documents BingPreview, which makes page snapshots, as using the same bingbot/2.0 user agent strings. The user agent alone can't tell the two apart in your logs.
Related crawlers
Sources
Checked against these pages on 2026-10-18. Operators change their crawlers; check the source before relying on a detail.