Where is the robots.txt setting in Webflow?

Site settings > SEO > Indexing. Add your rules, save and publish. It needs a Site plan or paid Workspace plan, and once a robots.txt exists you can only replace its rules, not delete it.

Does Webflow add my sitemap to robots.txt?

Yes, automatically. If you also type a Sitemap line yourself, the file lists the sitemap twice. A Remove sitemap.xml from robots.txt toggle under Indexing removes the automatic line, which helps reverse-proxy setups; it requires a paid Site plan.

Can I add an llms.txt file to a Webflow site?

Yes. Upload it at Site settings > SEO > LLMs.txt. It must be UTF-8 and under 100 KB, and it is published to your custom domain only. Webflow does not support the per-page .md files the llms.txt proposal describes.

Does Webflow add schema markup automatically?

No. You add it per page in Page settings under Schema markup, either by pasting JSON-LD or by clicking Generate schema markup to let Webflow AI write it. Put schema in one place only: the field or custom code, not both.

AI search readiness · Website builders and CMSs

Webflow AEO setup: robots.txt, sitemap, llms.txt and schema markup

Webflow puts nearly every AI search setting in Site settings > SEO. You can write robots.txt rules and a Content-Signal header under Indexing, auto-generate /sitemap.xml under Sitemap, upload an llms.txt under LLMs.txt, and add JSON-LD per page in a Schema markup field that Webflow AI can fill. Most of these need a paid Site plan or paid Workspace.

Edit robots.txt: YesSite settings > SEO > Indexing. Needs a Site plan or paid Workspace. Once created it can't be deleted, only replaced.
XML sitemap: YesSite settings > SEO > Sitemap > Auto-generate sitemap. Updates on each publish, with lastmod and hreflang for Localize sites. /sitemap.xml
llms.txt: YesSite settings > SEO > LLMs.txt > Upload file. UTF-8, under 100 KB, served on the custom domain only, not on webflow.io staging.
AI crawlers: No AI block toggle documented. You write robots.txt groups for AI bots under Site settings > SEO > Indexing, and can set a Content-Signal header (for example ai-train=no, search=yes) there. Enterprise sites on Webflow's Cloudflare hosting can serve Markdown to AI agents.
Structured data: No schema added by default. Each page's settings have a Schema markup field where you paste JSON-LD or click Generate schema markup (Webflow AI); Collection pages can bind CMS fields. Requires a paid Site plan or Workspace.

Webflow robots.txt and AI crawlers

Webflow builds robots.txt from rules you type in Site settings > SEO > Indexing. Save, then publish, and the file appears at yourdomain.com/robots.txt. You need a Site plan or a paid Workspace plan, and once the file exists it cannot be removed, only replaced (an Allow: / rule for all user agents is the reset).

Webflow documents no switch that blocks AI bots for you. To control them, write groups by user agent. For example, to keep GPTBot out of training while letting ChatGPT search find you:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

The same Indexing screen lets you set a Content-Signal header, such as Content-Signal: ai-train=no, search=yes, ai-input=no, which states separately whether content may be used for training, search and AI answers. Webflow notes it is a proposed extension to RFC 9309, not an accepted standard, so treat it as a signal on top of robots.txt rather than a replacement. Enterprise sites on Webflow's Cloudflare infrastructure can also switch on Markdown for agents (Site settings > Publishing > Advanced publishing options), which returns Markdown instead of HTML when an AI agent requests a page.

Two things to avoid: don't use robots.txt to hide a page that is already indexed (the Sitemap indexing toggle in Page settings adds noindex, which works better), and turn off Staging indexing so the webflow.io copy of your site does not compete with the real one.

Webflow sitemap

Turn on Site settings > SEO tab > Sitemap section > Auto-generate sitemap, save and publish. The sitemap at /sitemap.xml regenerates on every publish, each URL carries a lastmod date, and Localize sites get hreflang tags for every locale. You can paste a custom sitemap instead, but then you maintain hreflang yourself. Either way Webflow adds the sitemap to robots.txt automatically. Submit sitemap.xml in Google Search Console once the site is verified. More on how sitemaps help discovery: XML sitemap.

llms.txt on Webflow

Upload the file at Site settings > SEO > LLMs.txt > Upload file, then Save changes and publish. Webflow requires UTF-8 and under 100 KB, serves it only on your custom domain (not the staging domain), and does not support the per-page .html.md files the proposal describes. Webflow itself calls llms.txt experimental with unclear impact, so write it plainly, keep internal URLs out, and run it through the llms.txt checker.

Schema markup in Webflow

Nothing is added by default. Open the Pages panel, hover a page, click Settings, and scroll to Schema markup. Paste JSON-LD or click Generate schema markup to let Webflow AI draft it from the page content (Webflow AI must be on in Workspace settings). On Collection pages, bind CMS fields so each item gets its own values; Reference, Multi-reference and Multi-image fields can't be used. Markup in this field goes into the page head. If older schema already lives in custom code, keep it there and don't duplicate it. Validate with the Schema Markup Validator or Google's Rich Results Test, and see JSON-LD for the format.

Check your Webflow site

Paste your domain to see which AI crawlers your live robots.txt lets in.

robots.txt checker

Test your robots.txt against every AI crawler

robots.txt generator

Allow AI search, block trainingStay citable in ChatGPT, Claude, Perplexity, Gemini and Copilot answers while opting out of model training.Allow everythingEvery crawler may read every public page. The default when there is no robots.txt.Block all AI crawlersKeep Google and Bing search, block every AI crawler. AI answers will rarely cite or recommend the site.

Sitemap URLPrivate paths (one per line)

robots.txt

# robots.txt generated with https://www.getpicked.app/tools/robots-txt-checker
# Preset: Allow AI search, block training

User-agent: *
Allow: /

# Model training crawlers
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: Meta-ExternalAgent
User-agent: Amazonbot
User-agent: CCBot
User-agent: Bytespider
Disallow: /

Questions

Where is the robots.txt setting in Webflow?: Site settings > SEO > Indexing. Add your rules, save and publish. It needs a Site plan or paid Workspace plan, and once a robots.txt exists you can only replace its rules, not delete it.
Does Webflow add my sitemap to robots.txt?: Yes, automatically. If you also type a Sitemap line yourself, the file lists the sitemap twice. A Remove sitemap.xml from robots.txt toggle under Indexing removes the automatic line, which helps reverse-proxy setups; it requires a paid Site plan.
Can I add an llms.txt file to a Webflow site?: Yes. Upload it at Site settings > SEO > LLMs.txt. It must be UTF-8 and under 100 KB, and it is published to your custom domain only. Webflow does not support the per-page .md files the llms.txt proposal describes.
Does Webflow add schema markup automatically?: No. You add it per page in Page settings under Schema markup, either by pasting JSON-LD or by clicking Generate schema markup to let Webflow AI write it. Put schema in one place only: the field or custom code, not both.

Other website builders and cmss

Sources

Checked against these pages on 2026-10-19. Operators change their crawlers; check the source before relying on a detail.