llms.txt: a sitemap-style index for AI consumers
llms.txt is a proposed convention (analogous to robots.txt and sitemap.xml) for a human-curated index of a site's most cite-worthy content, designed for AI engines to ingest. Shipping one is cheap and signals you take AI visibility seriously.
llms.txt is a proposed convention (analogous to robots.txt and sitemap.xml) for a human-curated index of a site's most cite-worthy content, designed for AI engines to ingest. Shipping one is cheap and signals you take AI visibility seriously. A markdown file at /llms.txt that lists your most authoritative pages with one-line descriptions and links. Often paired with /llms-full.txt which concatenates the full text of those pages. Adoption is early but momentum is real.
Last updated·
What it is
A markdown file at /llms.txt that lists your most authoritative pages with one-line descriptions and links. Often paired with /llms-full.txt which concatenates the full text of those pages. Adoption is early but momentum is real.
Why it matters
A curated index is more efficient for AI engines than crawling the whole sitemap. Sites that ship llms.txt get cited disproportionately in early experiments. The cost is one build-time concatenation; the upside is being early on a likely-standard signal.
How to fix it
- Generate /llms.txt at build time. A markdown list grouped by section (Product, Tools, Research) with the title and one-line description of each.
- Generate /llms-full.txt with the full body. For each cite-worthy page, append its main content. AI engines that respect the convention can ingest it directly.
- Reference llms.txt from robots.txt. Discoverability — until well-known fully, point AI engines at it explicitly.
Authoritative sources
- Google Search Central documentation — Google
- Schema.org vocabulary — schema.org
- SEO Starter Guide — Google Search Central
- MDN — HTML meta and link elements — Mozilla MDN