llms.txt Checker

What the llms.txt Checker checks

llms.txt is a proposal from 2024 for a Markdown file at /llms.txt that gives language models a short, curated overview of a site: what it is, and links to the pages that matter most, ideally in clean text form. It is a community convention, not a web standard, and it is separate from robots.txt, which controls crawling.

This checker requests /llms.txt on the domain you enter and reports whether the file exists, whether it returns a successful status, and whether it is served as text rather than an HTML error page or a redirect to your home page. A soft 404, where a missing file returns your normal page with status 200, is reported as missing.

Why it matters

To be clear about expectations: no major search engine has said it uses llms.txt for ranking, and having one will not move your position in Google. Some AI tools and developer-facing assistants do read it, mainly for documentation sites, so it can help those tools find the right pages.

The result is therefore informational. It is useful to know whether you publish the file and that it is served correctly, but it matters far less than whether AI crawlers may access your site at all and whether your content is readable in the HTML. Our guides to AI crawlers and AI-readable content cover those.

How to fix common problems

If you decide to publish one, keep it short and factual. Start with # Site name, a one-line summary in a > blockquote, then sections such as ## Docs or ## Products with Markdown links to your key pages and a few words on each. Do not paste your whole site into it.

Serve it at the root as text/plain or text/markdown with status 200, and make sure your server does not redirect unknown paths to the home page. Keep it updated when the linked pages move; a stale file is worse than none.

Make sure your robots.txt rules match your intent, because a model that is blocked from crawling will not benefit from llms.txt. The full PerfoMetrix audit scores AI readiness, covering crawler access, content in the HTML, schema and authorship, among 170+ checks.

Questions

Does llms.txt improve SEO?

There is no evidence that search engines use it for ranking. Treat it as optional documentation for AI tools, not an SEO lever.

Is llms.txt the same as robots.txt for AI?

No. robots.txt tells crawlers what they may fetch. llms.txt only suggests which content is useful. To allow or block AI crawlers, use robots.txt.

What is llms-full.txt?

Some sites also publish a larger file with the full text of their documentation. It is a convention used by some documentation tools; this checker only looks at /llms.txt.