How the llms.txt validator works
- Paste your llms.txt content or pass a URL to fetch.
- The validator parses the file line by line and checks every rule in the AnswerDotAI llms.txt spec: exactly one H1 at the top, optional blockquote summary, H2 sections, and Markdown link-list items.
- Every issue is listed with a line number, a one-line explanation, and a fix hint you can copy straight into your file.
What is llms.txt?
llms.txt is a plain-text Markdown file at the root of a site at /llms.txt that points AI agents at the pages that matter. It is modeled on robots.txt and sitemap.xml but designed for readers, not indexers. Agents can pull llms.txt first, skim the curated list of links, then decide which pages to fetch.
The llms.txt spec in one screen
H1: Site name
> One short sentence summary.
Optional prose before the first section.
H2: Section name
- [Name](https://example.com): description
H2: Optional
- [Less important](https://example.com): skip if tight on contextWhich tokenizer the stats use
The token numbers use OpenAI's o200k_base encoding, the same tokenizer as the Token counter tool. The count is exact for OpenAI models. For Claude and Gemini it is a close estimate, usually within 10 to 15 percent on English prose.
What this validator does not do
- It does not crawl the URLs inside your llms.txt.
- It does not require an API key, log in, or save anything about the content you paste.
- It does not validate the structure of llms-full.txt. The tool reports its size and token count.
Frequently asked questions
llms.txt is a small Markdown file at the root of a site (/llms.txt) that helps AI agents find the most useful pages without having to crawl the whole site. It has a single H1 with the site name, an optional blockquote summary, and one or more H2 sections, each with a list of markdown links. The spec was proposed by AnswerDotAI and is backed by Anthropic, Shopify, Vercel, and others.
At the root of your origin, served at /llms.txt. The spec calls for the exact path. Do not put it in a subdirectory, do not rename it, and do not put it behind auth.
llms-full.txt is the companion "dump everything here" file. Where llms.txt is a curated list of links, llms-full.txt is the full text of the content you want agents to read. Most sites start with llms.txt only. Add llms-full.txt when you have meaningful long-form content you want ingested directly. This validator reports size and token count for llms-full.txt if it exists, but does not validate its structure, because the spec allows arbitrary full-text content there.
No. Every URL inside your file would be a separate network request, which turns this tool into a crawler. Intentionally out of scope. The validator checks that each URL parses, warns on http://, and flags duplicates.
The spec has no hard cap, but the point of llms.txt is that an agent can read the whole thing before deciding what else to fetch. Keep it under a few thousand tokens for the main file. llms-full.txt can be larger since agents will only pull it when they need it, but every token in either file competes with the actual answer for context window space.
Keep stores every link you bookmark as clean Markdown with a title and description. That is most of what llms.txt needs. A bookmark-to-llms.txt export is the natural next step for this tool, and is on the roadmap. In the meantime, this validator is the fastest way to check a hand-written or framework-generated file against the spec.
Errors block the file from passing: a missing H1, a second H1, a list item that does not match the spec shape, or an unparseable URL. Warnings flag things that work but are probably not what you meant: a non-https URL, an empty blockquote, a duplicate URL, a section with no links. Infos are small nudges, like switching a http:// to https://.
No. The content is sent once to the validator, parsed in memory, and the result is sent back. Nothing is written to the database or tied to an account.