How the llms.txt validator works
- Paste your llms.txt content or pass a URL to fetch.
- The validator parses the file line by line and checks every rule in the AnswerDotAI llms.txt spec: exactly one H1 at the top, optional blockquote summary, H2 sections, and markdown link-list items.
- Every issue is listed with a line number, a one-line explanation, and a fix hint you can copy straight into your file.
What is llms.txt?
llms.txt is a plain-text Markdown file at the root of a site (/llms.txt) that points AI agents at the pages that matter. It is modeled on robots.txt and sitemap.xml but designed for readers, not indexers. Agents fetching a site for context can pull llms.txt first, skim the curated list of links, then decide which pages to actually fetch.
The llms.txt spec in one screen
# Site name ← single H1, required > One short sentence summary. ← optional blockquote Optional prose paragraphs between ← free-form markdown the summary and the first section. ## Section name ← one or more H2 sections - [Name](https://example.com): desc ← markdown links, one per line ## Optional ← "Optional" section is a convention - [Less important](https://example.com): skip if tight on context
Which tokenizer the stats use
The token numbers use OpenAI's o200k_base encoding, the same tokenizer as the Token counter tool. Counts track closely with Claude and Gemini on English prose (within about 10 to 15 percent), which is the same caveat as every other tokenizer that runs in a serverless environment.
What this validator does not do
- It does not crawl the URLs inside your llms.txt. That would turn the tool into a crawler and defeat the point. The validator trusts that an agent is capable of fetching those URLs itself.
- It does not require an API key, log in, or save anything about the content you paste.
- It does not validate the structure of llms-full.txt. The spec allows arbitrary long-form content there, so the tool reports size and token count and leaves content validation to you.
Frequently asked questions
What is llms.txt?
llms.txt is a small Markdown file at the root of a site (/llms.txt) that helps AI agents find the most useful pages without having to crawl the whole site. It has a single H1 with the site name, an optional blockquote summary, and one or more H2 sections, each with a list of markdown links. The spec was proposed by AnswerDotAI and is backed by Anthropic, Shopify, Vercel, and others.
Where should my llms.txt live?
At the root of your origin, served at /llms.txt. The spec calls for the exact path. Do not put it in a subdirectory, do not rename it, and do not put it behind auth.
What is llms-full.txt and do I need both?
llms-full.txt is the companion "dump everything here" file. Where llms.txt is a curated list of links, llms-full.txt is the full text of the content you want agents to read. Most sites start with llms.txt only. Add llms-full.txt when you have meaningful long-form content you want ingested directly. This validator reports size and token count for llms-full.txt if it exists, but does not validate its structure, because the spec allows arbitrary full-text content there.
Does the validator check that the URLs inside my llms.txt resolve?
No. Every URL inside your file would be a separate network request, which turns this tool into a crawler. Intentionally out of scope. The validator checks that each URL parses, warns on http://, and flags duplicates.
How many tokens is "too many" for llms.txt?
The spec has no hard cap, but the point of llms.txt is that an agent can read the whole thing before deciding what else to fetch. Keep it under a few thousand tokens for the main file. llms-full.txt can be larger since agents will only pull it when they need it, but every token in either file competes with the actual answer for context window space.
Can you generate llms.txt from my Keep bookmarks?
Keep stores every link you bookmark as clean Markdown with a title and description. That is most of what llms.txt needs. A bookmark-to-llms.txt export is the natural next step for this tool, and is on the roadmap. In the meantime, this validator is the fastest way to check a hand-written or framework-generated file against the spec.
What counts as an error vs a warning?
Errors block the file from passing: a missing H1, a second H1, a list item that does not match the spec shape, or an unparseable URL. Warnings flag things that work but are probably not what you meant: a non-https URL, an empty blockquote, a duplicate URL, a section with no links. Infos are small nudges, like switching a http:// to https://.
Do you store the content I paste?
No. The content is sent once to the validator, parsed in memory, and the result is sent back. Nothing is written to the database or tied to an account.
Generate llms.txt from the bookmarks you already save
Keep captures every URL you save as clean Markdown with a title and description. That is most of what llms.txt needs. A bookmark-to-llms.txt export is the natural next step for this tool. In the meantime, the validator is the fastest way to check a hand-written or framework-generated file against the spec.