/llms.txt
The llms.txt File, Explained
A single Markdown file at your site root that hands AI tools a clean, curated map of your most important pages. Here is the exact format from the spec, a copy-ready example, and a free generator.
Short answer
llms.txt is a Markdown file at /llms.txt that gives AI agents a short, curated list of your most useful pages. It starts with an # Name (the only required line), then a > summary, then ## Section lists of - [Title](url): note links. It is a proposal from Answer.AI, revised as v2 in August 2026, not an official standard. Coding agents use it to find docs; Google Search says it does not use it at all.
The big idea
A curated, Markdown index of your site, written for AI agents
It is just Markdown
A plain llms.txt file at your site root, or under a path like /docs/llms.txt: an H1 name, a short summary, then H2 sections of curated links. No new syntax to learn.
Agents read it on demand
An agent opens it when it needs your content, most often a coding agent looking up docs, then follows the links. It is not a ranking signal: Google Search ignores it.
It is curated, not exhaustive
Unlike a sitemap, llms.txt lists only the pages that genuinely help a model understand you, small enough to fit in a context window.
What llms.txt Is — and Why It Exists
Large language models increasingly rely on website content to answer questions, but they hit a wall: a context window is finite, and a real web page is mostly navigation, ads, and scripts wrapped around the actual information. Converting all of that to clean text on the fly is slow and lossy.
llms.txt solves this by letting you publish the clean version. It is a Markdown file at /llms.txt that names your project, summarizes it in a sentence, and links to your most useful pages — exactly the high-signal context a model needs, and nothing else. The proposal comes from Jeremy Howard at Answer.AI (September 2024, revised as v2 in August 2026), and its name deliberately mirrors the well-known /robots.txt and /sitemap.xml conventions.
It pairs naturally with the broader idea of Markdown for AI. The spec chose Markdown because it is the format language models understand most readily, while still being regular enough to parse with ordinary code.
The Format, Part by Part
The spec is precise but short. The sections must appear in this order — and only the first one is required.
| Part | Status | What it is |
|---|---|---|
| H1 — project name | Required | A single # heading with the name of your site or project. This is the only required line in the entire file. |
| Blockquote summary | Optional | A > blockquote with a short description carrying the key information a model needs to understand the rest of the file. |
| Body sections | Optional | Zero or more Markdown sections (paragraphs, lists) of any type except headings, with extra context on how to read the files below. |
| H2 file lists | Optional | Zero or more ## sections, each a list of links. Each item is a required [name](url) hyperlink, then optionally a colon and a note. |
| ## Optional section | Convention | A section named “Optional” holds secondary links an agent can skip when it needs a shorter context. Since v2 this is a convention, not a rule tools must apply. |
A Complete Example
Here is a full llms.txt with every part of the spec. Because it is plain Markdown, it renders cleanly too — the source is on the left, the way a model reads its structure is on the right.
# Acme Docs
> Acme is an open-source toolkit for building
> data pipelines in Python.
Important: Acme is not a hosted service. Everything
runs locally or in your own infrastructure.
## Docs
- [Quick start](https://acme.dev/start.md): Install and run your first pipeline
- [API reference](https://acme.dev/api.md): Every public class and method
## Examples
- [ETL walkthrough](https://acme.dev/etl.md): A complete end-to-end pipeline
## Optional
- [Changelog](https://acme.dev/changelog.md): Full release historyAcme Docs
Acme is an open-source toolkit for building data pipelines in Python.
Important: Acme is not a hosted service. Everything runs locally or in your own infrastructure.
Docs
- Quick start: Install and run your first pipeline
- API reference: Every public class and method
Examples
- ETL walkthrough: A complete end-to-end pipeline
Optional
- Changelog: Full release history
Free llms.txt Generator
Fill in your project name, a one-line summary, and the links you want to expose. The valid, spec-formatted file builds live on the right — copy it or download it as llms.txt. Everything runs in your browser.
# Acme Docs
> Acme is an open-source toolkit for building
> data pipelines in Python.
## Docs
- [Quick start](https://acme.dev/start.md): Install and run your first pipeline
- [API reference](https://acme.dev/api.md): Every public class and method
## Optional
- [Changelog](https://acme.dev/changelog.md): Full release history
Generated entirely in your browser — nothing is uploaded. Save the result as llms.txt at the root of your site so it resolves at /llms.txt.
llms.txt vs sitemap.xml vs robots.txt
These live at similar well-known paths, so they get confused. They solve different problems: llms.txt is the only one written for a model to read and small enough to fit in context.
| File | Read by | Scope | Format |
|---|---|---|---|
| llms.txt | AI agents, on demand (mostly coding agents reading docs) | Curated — only your most useful pages | Markdown |
| llms-full.txt | Tools that want a whole docs site in one file | The full text of those pages, inlined. A convention, not part of the spec | Markdown (large) |
| sitemap.xml | Search engine crawlers | Exhaustive — every indexable page | XML |
| robots.txt | All crawlers & bots | Access rules — what may be crawled | Plain text directives |
Who Actually Reads llms.txt?
Less than the hype suggests, and in a different way than robots.txt. Agents fetch llms.txt when they need your content; there is no evidence it changes how search engines rank or cite you.
Coding agents and docs sites: yes
This is where it works. Agents use a docs site's llms.txt to find the right API page, and OpenAI, Anthropic, and Google publish one for their own developer docs.
Google Search: no
Google's AI optimization guide says Search, including AI Overviews and AI Mode, does not use llms.txt, and that having one neither helps nor hurts your rankings.
AI crawlers: rarely
A 12-week log study of 83 sites (EZY.ai, April to July 2026) counted 7 llms.txt fetches from OpenAI's crawlers, 9 from ClaudeBot, and none from PerplexityBot, against thousands of robots.txt fetches. Check your own logs for /llms.txt by user agent.
Chrome Lighthouse: checks it
Lighthouse's agentic browsing audits request /llms.txt. A server error fails the audit; a 404 is marked not applicable, since the file is optional.
The Companion: .md Versions of Your Pages
The proposal has a second half that is easy to miss. Wherever a page would help an agent, serve a clean Markdown copy at the same URL, with .md appended or, since v2, with the extension swapped for .md. The links inside a good llms.txt point at these.
| Human page | Markdown version (either form) |
|---|---|
| /docs/intro.html | /docs/intro.html.md or /docs/intro.md |
| /guide/ | /guide/index.html.md or /guide/index.md |
v2 also makes these files discoverable. Point from each page to its Markdown copy with rel="alternate", and to the llms.txt that covers it with rel="describedby", as HTML tags or as one HTTP header you can add at the server or CDN:
<!-- In the <head> of /docs/page.html -->
<link rel="alternate" type="text/markdown" href="/docs/page.html.md">
<link rel="describedby" href="/docs/llms.txt">
# Or as one HTTP response header
Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"Accept: text/markdown header, which services like Cloudflare can answer by converting the page on the fly.Ways to Generate It
Pick whatever matches your stack. For most small sites, writing it by hand is the right call: the curation is the whole point.
By hand (recommended for small sites)
Use the generator above or just open a text editor. You know which pages matter most; a crawler does not.
Docs framework plugins
VitePress (vitepress-plugin-llms), Docusaurus (docusaurus-plugin-llms), and Drupal can emit llms.txt, and often the .md pages, at build time.
Your docs platform or CMS
Mintlify and GitBook serve one for hosted docs, Wix generates one for every site, and Yoast SEO and AIOSEO add a generator to WordPress. Review the output and trim it to the pages that matter.
llms-full.txt (optional)
Docs platforms like Mintlify also publish an llms-full.txt with every page's full text in one file. It is a popular convention, not part of the spec; v2 dropped the old llms_txt2ctx expansion tool.
Common Mistakes
The errors that turn a useful llms.txt into noise.
Avoid
Listing every URL like a sitemap
Do
Hand-pick the pages that explain your site
Avoid
Expecting /docs/llms.txt to describe the whole site
Do
Use /llms.txt for the site, /docs/llms.txt for just the docs
Avoid
Linking to HTML pages full of nav and ads
Do
Link to clean .md versions where you can
Avoid
Skipping the H1 or the summary
Do
Always include the # name; add the > summary
Avoid
Adding llms.txt to rank in Google
Do
Add it for agents; do normal SEO for Google
Frequently Asked Questions
What is llms.txt?
llms.txt is a Markdown file at the root of a website (yourdomain.com/llms.txt) that gives AI agents a short, curated map of your most useful pages. It has an H1 with the site or project name, an optional blockquote summary, and H2 sections listing links with one-line notes. Jeremy Howard of Answer.AI proposed it in September 2024 and revised it as v2 in August 2026. Agents read it on demand, for example when a coding agent looks up a library's docs, because full HTML pages are noisy and a whole site will not fit in a context window.
What is the exact format of an llms.txt file?
In this order: (1) an H1 with the project or site name, the only required part; (2) a blockquote with a short summary; (3) zero or more Markdown sections of any type except headings, for extra detail; and (4) zero or more H2 sections containing 'file lists', Markdown lists where each item is a [name](url) hyperlink optionally followed by a colon and a note. A section named 'Optional' is used by convention for secondary links an agent can skip. v2 also allows an optional byte-order mark at the start of the file.
Where do I put the llms.txt file?
Usually at the root of your domain, so it resolves at https://yourdomain.com/llms.txt, next to robots.txt and sitemap.xml. Since v2 you can also place one at any path, such as /docs/llms.txt, and it covers the pages under that path. When more than one file applies, agents should use the most specific one. That lets a site that only controls a path, like a GitHub Pages project site, publish its own.
What is the difference between llms.txt and sitemap.xml?
A sitemap.xml lists every indexable page on your site for search engine crawlers, in XML, and is usually far too large to fit in an AI context window. llms.txt is the opposite: a short, hand-curated Markdown overview of only your most useful pages, meant to be read when an agent needs information about your site. A sitemap aims for completeness; llms.txt aims for a high-signal summary. They serve different purposes and can coexist.
What is llms-full.txt?
llms-full.txt is a companion file that puts the full Markdown text of your pages into one large document, instead of just linking to them. It is a popular convention, not part of the llms.txt spec. Docs platforms such as Mintlify generate both files automatically. The original proposal described a tool, llms_txt2ctx, for expanding llms.txt into a context file; v2 dropped it and instead expects agents to read llms.txt and follow the links they need.
Is llms.txt an official standard?
No. It is a community proposal from Answer.AI, maintained on GitHub, not a W3C or IETF standard. Adoption is real, though: the v2 proposal says thousands of sites publish one, docs platforms like Mintlify and GitBook generate one automatically, OpenAI, Anthropic, and Google publish llms.txt files for their developer docs, and Chrome's Lighthouse checks for it in its agentic browsing audits.
Does Google use llms.txt?
No. Google's guide to optimizing for its generative AI features says Google Search, including AI Overviews and AI Mode, does not use llms.txt or other AI text files, and that you do not need to create them to appear there. It also says creating one for other services is fine and will neither help nor hurt your visibility in Google Search.
Do AI tools like ChatGPT and Claude actually read llms.txt?
Mostly on demand, not by crawling. The common use is a coding agent or assistant fetching a docs site's llms.txt, or being pointed at one, to find the right page. Published 2026 server-log studies found the big AI crawlers rarely request it: one 12-week study of 83 sites counted 7 llms.txt fetches from OpenAI's crawlers and 9 from ClaudeBot, against thousands of robots.txt fetches. It is cheap to add and does no harm, but do not expect it to change how often AI search cites you.
How do I generate an llms.txt file?
Write it by hand, since it is just Markdown, or use the free generator on this page to fill in your project name, summary, and links and copy the result. Many platforms generate one for you: Mintlify and GitBook for hosted docs, Wix for every Wix site, Yoast SEO and AIOSEO on WordPress, plus plugins for VitePress, Docusaurus, and Drupal. Review generated files and cut them down to the pages that matter.
What are the .md page versions the spec mentions?
Alongside llms.txt, the proposal suggests serving a clean Markdown copy of any page an agent might need, at the same URL as the page. v2 allows two forms: append .md (/docs/intro.html.md) or replace the extension (/docs/intro.md). For URLs without a file name, use index.html.md or index.md. To make them discoverable, link them with rel="alternate" type="text/markdown", and point to the covering llms.txt with rel="describedby", either as HTML link tags or an HTTP Link header.
What changed in llms.txt v2?
v2, published in August 2026, adds discovery through standard link relations (rel="alternate" type="text/markdown" for a page's Markdown copy, rel="describedby" for its llms.txt); allows .md copies at page.md as well as page.html.md; defines that an llms.txt in a subpath covers the pages under it, with the most specific file winning; drops the llms_txt2ctx expansion tool; and turns the 'Optional' section from a rule into a convention.