What is the difference between robots.txt and llms.txt?

robots.txt tells crawlers what they may or may not fetch. llms.txt is a human-curated markdown index meant to help LLMs pick the right pages for context at inference time. They solve different problems and work together — robots.txt controls crawling; llms.txt guides discovery.

Do I need to host llms.txt at my domain root?

The convention is https://yoursite.com/llms.txt at the site root, the same way robots.txt lives at /. Some teams also publish a fuller companion file (llms-full.txt) with more content, but the index file should be at the root path for tools and crawlers that look for it.

Does ChatGPT or Claude actually read llms.txt today?

Support is still emerging and inconsistent across AI products and crawlers as of 2026. Treat llms.txt as a best-effort signal and curated index — not a guaranteed ranking or citation mechanism. It is still worth publishing because it is lightweight, spec-defined, and under your control.

What's the Optional section for?

In the llms.txt spec, a section titled Optional holds links an LLM may skip first when context is limited — changelog, blog archives, comparisons, and similar pages that are useful but not core. It keeps your primary sections focused.

Is this free, and does it store my data?

Yes, it is free. Everything runs client-side in your browser — your sitemap, llms.txt content, and edits are never uploaded to a server or stored by DesignRevision.

llms.txt Generator

Build, validate, and export a spec-compliant llms.txt — no crawl, no signup.

Free llms.txt generator with a built-in llms.txt validator. Create a file from scratch, import URLs from a pasted sitemap.xml, or paste an existing llms.txt to edit. Everything runs in your browser — nothing is uploaded.

Site header

H1 title *

Summary (blockquote)

Notes (optional paragraph)

Paste a sitemap.xml urlset or upload a file. Sitemap indexes are not fetched — paste a child sitemap instead.

Upload .xml

Paste an existing llms.txt to load it into the editor for tweaking.

Sections & links

Live preview

llms.txt validator

No validation issues — your llms.txt looks good.

What is an llms.txt file?

llms.txt is a markdown convention for helping AI systems discover your site. Unlike a sitemap dump, it is a curated index: one H1 title, an optional blockquote summary, optional notes, then H2 sections of links formatted as - [Title](url): description. The spec recognizes an ## Optional section for pages an LLM may skip when context is tight.

How to create llms.txt

Enter your site or project name as the H1.
Add a one-sentence blockquote summary (recommended).
Create sections like Docs, Tools, or Optional.
Add your most important links with short descriptions.
Use the built-in llms.txt validator to catch broken URLs, duplicates, and formatting issues.
Copy or download llms.txt and host it at your domain root.

You can also paste a sitemap.xml to seed link candidates (titles are guessed from URLs — edit them for accuracy) or paste an existing file to round-trip edit.

Real llms.txt examples

These are live production files you can inspect:

designrevision.com/llms.txt — DesignRevision's own index (we ship this tool and practice what we preach).
designrevision.com/llms-full.txt — full component source concatenation companion file.
llmstxt.org/llms.txt — the spec author's own file.

# DesignRevision

> Production-ready shadcn/ui components you own…

## Components

- [Button](https://designrevision.com/components/button.md): Free shadcn/ui button…

llms.txt vs robots.txt

robots.txt controls what automated crawlers may fetch. llms.txt is a human-maintained index meant to guide LLMs toward the right pages at inference time. They are complementary — not replacements for each other.

Why this generator is different

Most online llms.txt generators ask for your homepage URL and crawl your site server-side, often with AI-written descriptions. This tool is 100% client-side: no URL handoff, no signup, no per-request cost. You stay in control of what goes into the file, with spec-correct output and live validation built in.

Frequently asked questions

Enter your site title and optional summary, add H2 sections (like Docs or Optional), then list links as title, URL, and description. You can also paste a sitemap.xml or an existing llms.txt to import structure, use the live validator to fix issues, then copy or download the finished file.

llms.txt is a markdown file proposed at llmstxt.org that sits at your site root. It gives AI systems a curated index of your most important pages — a title, short summary, and grouped links with descriptions.

If you care about how AI tools represent your product or documentation, yes — a well-curated llms.txt is low effort and gives you control over what gets highlighted. Adoption by crawlers is still early, but the file is cheap to ship.

robots.txt tells crawlers what they may or may not fetch. llms.txt is a human-curated markdown index meant to help LLMs pick the right pages for context. They solve different problems and work together.

The convention is https://yoursite.com/llms.txt at the site root, similar to robots.txt. Some teams also publish a fuller companion file, but the index should live at the root path.

Support is still emerging and inconsistent across AI products as of 2026. Treat llms.txt as a best-effort signal — not a guaranteed citation mechanism — but it is lightweight and under your control.

In the llms.txt spec, Optional holds links an LLM may skip first when context is limited — changelog, blog archives, and similar useful-but-secondary pages.

Yes, it is free. Everything runs client-side in your browser — your content is never uploaded or stored by DesignRevision.