How to write an llms.txt file

llms.txt is a plain Markdown file, proposed in 2024, intended to give an AI system a short, structured summary of a site — what it is, and which pages matter — without having to crawl and interpret the whole thing from scratch. It is worth being direct about what this actually is before writing one: it is a proposed convention put forward by one part of the industry, not an established web standard, and no search engine or AI system is obliged to read it or act on it.

Where it sits next to robots.txt and sitemap.xml

The comparison to robots.txt is useful for understanding the idea, but the two are not equivalent in status. Robots.txt is a decades-old, near-universally respected convention with a published specification that essentially every crawler honours. Sitemap.xml has a formal protocol maintained at sitemaps.org and is explicitly supported by every major search engine. llms.txt has neither the history nor the confirmed, universal support of either — it is a proposal that some AI tools have said they look for, while plenty of others do not mention it at all. Treat it as a low-cost, plausible bet on where a corner of the web might be heading, not as infrastructure you can rely on.

The format

The convention as proposed is deliberately simple: standard Markdown, structured consistently enough that a parser can extract it reliably.

# Your Site Name

> A one- or two-sentence summary of what the site does and who it is for.

Any additional short context worth including as plain paragraphs.

## Section Name
- [Page Title](https://yourdomain.com/page): a short, accurate description of the page.
- [Another Page](https://yourdomain.com/other-page): what this page actually covers.

## Optional
- [Less important page](https://yourdomain.com/misc): included but lower priority.

• An H1 with the site's name, once, at the top.
• A blockquote directly underneath giving a concise summary — this is the part most likely to be quoted directly if the file is used at all.
• H2 sections grouping related pages, each listed as a Markdown link followed by a colon and a short, accurate description.
• An optional final section, conventionally named "Optional", for pages worth including but of lower priority than the rest.

Write it the way you would write anything else that gets published

The same rule that governs the rest of this knowledgebase applies here without exception: describe only what is actually true. Do not list a page that does not exist, describe a feature the site does not have, or claim a capability that has not shipped. An AI system summarising or quoting from this file has no way to verify it against reality beyond what it can separately confirm by crawling the site — and a mismatch between what llms.txt claims and what the site actually contains is the kind of inconsistency that undermines trust in everything else the file says.

Publishing it

Upload the finished file to your site's root through your control panel's file manager, so it is reachable at yourdomain.com/llms.txt — the same convention robots.txt and sitemap.xml both use. There is no submission process and no tool to notify, unlike Search Console for a sitemap; a crawler that looks for llms.txt simply requests it directly by URL when it visits the site.

Maintaining it once it exists

Treat llms.txt the same way you would treat any other page describing your site: revisit it when the site itself changes meaningfully, rather than writing it once and forgetting it. A file that still lists a discontinued service or an old page months after it changed is worse than not publishing one at all, for the same reason any stale, inaccurate summary undermines trust in the source it came from.

Keep expectations realistic

Publishing an llms.txt file costs little and cannot hurt anything, provided it stays accurate and the rest of the site remains genuinely crawlable. What it cannot do is guarantee that any particular AI system uses it, cites your site, or represents your content the way you intended. Treat it as one small, low-effort part of being easy to find and accurately understood on the modern web — not as a control panel for how AI systems talk about your business.