Is the Yoast llms.txt feature free?

Yes. Yoast says generating your llms.txt file is included for free. It arrived in Yoast SEO 25.3 on June 10, 2025, and is off until you turn it on under Settings > Site features.

What is the Yoast sitemap URL?

/sitemap_index.xml. Requests to /sitemap.xml redirect there. Submit the index once in Google Search Console; it links to the post, page, category and other sitemaps.

Why can't I find the Yoast robots.txt file editor?

Yoast says the File editor menu does not appear when WordPress file editing is disabled, and saving fails when robots.txt is not writable. In those cases, edit or upload robots.txt over FTP or your host's file manager.

Can Yoast block AI crawlers like GPTBot?

Only in Premium. Yoast SEO Premium has Block unwanted bots toggles for GPTBot, CCBot and Google-Extended under Settings > Advanced > Crawl optimization. The free plugin can only block Google AdsBot.

AI search readiness · SEO plugins

Yoast SEO for AI search: robots.txt, sitemap_index.xml and llms.txt

Yoast SEO is a WordPress plugin with 10+ million active installs. It edits robots.txt under Tools > File editor, serves an XML sitemap index at /sitemap_index.xml, and has generated llms.txt for free since version 25.3 (June 10, 2025). Premium adds toggles that block GPTBot, CCBot and Google-Extended, and every page gets a linked JSON-LD schema graph.

Edit robots.txt: YesYoast SEO > Tools > File editor creates or edits robots.txt. It needs a writable file, and the menu disappears if WordPress file editing is disabled.
XML sitemap: YesYoast's XML sitemaps serve an index with up to 1,000 items per sitemap, leave out noindex pages, and redirect /sitemap.xml to the index. /sitemap_index.xml
llms.txt: YesFree since 25.3: turn it on under Settings > Site features > AI tools. Automatic selection refreshes weekly, or pick pages by hand. Not available on multisite.
AI crawlers: Settings > Advanced > Crawl optimization > Block unwanted bots. The free plugin can only block Google AdsBot; Premium adds toggles for Google-Extended (Gemini and Vertex AI), GPTBot and CCBot, which Yoast writes into robots.txt.
Structured data: Outputs one JSON-LD @graph per page: WebPage, WebSite and Organization (or Person), linked by @id, plus Article on posts and FAQPage or HowTo when you use Yoast's blocks. Developers extend the graph through Yoast's Schema API.

Editing robots.txt with Yoast SEO

WordPress serves a virtual robots.txt when there is no file in the site root. Yoast lets you replace it from the dashboard:

Go to Yoast SEO > Tools > File editor.
Click Create robots.txt file if none exists.
Edit the rules and click Save changes to robots.txt.

Yoast's starting file is User-agent: * with an empty Disallow: (allow everything) plus a Sitemap: line pointing to your sitemap_index.xml. If your server or WordPress setup disables file editing, the File editor menu does not appear, and if robots.txt is not writable, saving fails. The fix is to edit the file over FTP or your host's file manager. See the WordPress page for how the virtual file works.

Yoast sitemap: /sitemap_index.xml

Yoast's XML sitemaps feature serves an index at /sitemap_index.xml, and requests for /sitemap.xml redirect to it. The index links to separate sitemaps per post type, taxonomy and author. Each holds up to 1,000 items by default, overflowing into more files. Pages with a noindex directive, private or password-protected content, and paginated archives are left out, and the sitemap files themselves carry x-robots-tag: noindex, follow.

Submit only the index URL to Google Search Console and Bing.

Yoast llms.txt

Yoast SEO 25.3 (June 10, 2025) introduced automatic llms.txt generation, and Yoast says it is free. To enable it:

Go to Yoast SEO > Settings > Site features.
Under AI tools, open llms.txt and turn the toggle on.
Click Save changes.

By default Yoast picks pages automatically, favoring cornerstone content, and refreshes the file weekly. A July 29, 2025 update added Manual page selection under Customize llms.txt file. If you see the notice that the file couldn't be auto-generated, Yoast says to check the web server's filesystem permissions. The feature is not available on multisite. Check the output with the llms.txt checker.

AI crawlers and the schema graph

Under Settings > Advanced > Crawl optimization > Block unwanted bots, Premium adds toggles for Google-Extended, GPTBot and CCBot; the free plugin only blocks Google AdsBot. Bot Blocker works by updating robots.txt, so it only affects bots that honor it. Blocking GPTBot does not remove you from ChatGPT search, which OAI-SearchBot handles.

For schema markup, Yoast prints a single JSON-LD @graph on every page with WebPage, WebSite and Organization (or Person for a personal site), linked by @id. Posts add an Article; pages using Yoast's FAQ or HowTo blocks add FAQPage or HowTo. Plugins can add pieces through Yoast's Schema API instead of printing separate, disconnected blocks.

Check your Yoast SEO site

Paste your domain to see which AI crawlers your live robots.txt lets in.

robots.txt checker

Test your robots.txt against every AI crawler

robots.txt generator

Allow AI search, block trainingStay citable in ChatGPT, Claude, Perplexity, Gemini and Copilot answers while opting out of model training.Allow everythingEvery crawler may read every public page. The default when there is no robots.txt.Block all AI crawlersKeep Google and Bing search, block every AI crawler. AI answers will rarely cite or recommend the site.

Sitemap URLPrivate paths (one per line)

robots.txt

# robots.txt generated with https://www.getpicked.app/tools/robots-txt-checker
# Preset: Allow AI search, block training

User-agent: *
Allow: /

# Model training crawlers
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: Meta-ExternalAgent
User-agent: Amazonbot
User-agent: CCBot
User-agent: Bytespider
Disallow: /

Questions

Is the Yoast llms.txt feature free?: Yes. Yoast says generating your llms.txt file is included for free. It arrived in Yoast SEO 25.3 on June 10, 2025, and is off until you turn it on under Settings > Site features.
What is the Yoast sitemap URL?: /sitemap_index.xml. Requests to /sitemap.xml redirect there. Submit the index once in Google Search Console; it links to the post, page, category and other sitemaps.
Why can't I find the Yoast robots.txt file editor?: Yoast says the File editor menu does not appear when WordPress file editing is disabled, and saving fails when robots.txt is not writable. In those cases, edit or upload robots.txt over FTP or your host's file manager.
Can Yoast block AI crawlers like GPTBot?: Only in Premium. Yoast SEO Premium has Block unwanted bots toggles for GPTBot, CCBot and Google-Extended under Settings > Advanced > Crawl optimization. The free plugin can only block Google AdsBot.

Other seo plugins

Rank Math

Sources

Checked against these pages on 2026-10-19. Operators change their crawlers; check the source before relying on a detail.