What is llms.txt, and do you need one?

Every few months a new file promises to get you into AI answers. Right now it's llms.txt. Some people say it's essential. Google says it ignores it. Both can be true, depending on who you want to read your site.

Here's what llms.txt actually is, who uses it, and an honest call on whether you should spend an hour on one.

Short answer: llms.txt is a markdown file at your domain root that summarizes your site and links to your key pages for AI agents. It won't help Google rankings or AI Overviews. It's cheap, harmless, and useful mainly for docs, developer tools and agent-driven browsing.

Where llms.txt came from

The format was proposed by Jeremy Howard, co-founder of Answer.AI and fast.ai, on September 3, 2024, at llmstxt.org. The problem it targets is real. As the proposal puts it: "Web pages are built for people. An HTML page wraps its information in navigation, ads, and JavaScript, and converting it back into clean text is difficult and imprecise."

Language models also have limited context. The proposal notes that context windows "are still too small for most websites in their entirety, and every wasted token costs time and money." So instead of making an agent crawl your whole site, you hand it a short map.

In August 2026 the proposal was updated to a v2, based on two years of adoption. The changes page lists what moved: standard link relations so agents can discover markdown versions of pages, more flexible markdown URLs, and a clear rule that a file covers the pages under its path. The old context-expansion tooling, and the special meaning of an "Optional" section, were dropped.

What goes in the file

It's plain markdown with a light structure:

1. An H1 with your site or project name. This is the only required part.
2. A blockquote with a one-paragraph summary.
3. Optional detail: a few paragraphs or bullets of context an agent should know.
4. H2 sections with link lists: markdown links to your key pages, each with a short note.

A minimal example for a SaaS product:

# Acme Invoices

> Acme Invoices is invoicing software for freelancers in the EU.
> It handles VAT, recurring invoices and payment reminders.

## Product
- [Pricing](https://acme.example/pricing): Plans and what each includes
- [Features](https://acme.example/features): Full feature list

## Docs
- [API reference](https://acme.example/docs/api): REST API, auth and webhooks
- [Import guide](https://acme.example/docs/import): Moving from other tools

The proposal also suggests offering clean markdown versions of pages, at the same URL with .md appended or with the extension swapped for .md. v2 adds rel="alternate" type="text/markdown" and rel="describedby" link relations so agents can find those versions and the llms.txt that covers them.

Who actually reads it

This is where the hype and the facts part ways.

Google Search doesn't. In June 2026 Google added a note to its AI optimization guide: "It's completely fine if you decide to create and maintain LLMS.txt files (or other similar files) for other services or systems that use these files. Doing so won't harm (nor help) your visibility or rankings in Google Search, as Google Search ignores them." Google's AI features documentation is just as direct: "You don't need to create new machine readable files, AI text files, or markup to appear in these features."

This isn't new. In April 2025, Google's John Mueller wrote on Reddit that "none of the AI services have said they're using LLMs.TXT (and you can tell when you look at your server logs that they don't even check for it)," and compared it to the keywords meta tag. (Search Engine Journal)

Chrome's Lighthouse checks for it, lightly. Lighthouse's experimental agentic browsing audits include an llms.txt check. It flags a server error when fetching the file. If the file simply doesn't exist (a 404), the audit is marked not applicable, "as providing the file is optional." So Lighthouse won't penalize you for skipping it. It only cares that you don't serve a broken one.

Documentation platforms and AI labs publish it. The v2 proposal notes that documentation platforms now generate llms.txt automatically and that major AI labs publish their own. That's the clearest use case: developer docs, where a person or a coding agent is pointed at a specific site and needs the right page fast.

Agents that are told to use it. The honest version of "who reads llms.txt" is: agents and tools that a person points at your site, or that look for it while browsing on someone's behalf. That's a small audience today. It may grow. It isn't search traffic.

Should you make one?

Our recommendation, by situation:

• You run developer docs or an API product: yes. Agents and developers do use these files to find the right page, and the cost is tiny.
• You're a SaaS or content site: it's fine to add one, and it takes minutes if you generate it. Just don't expect traffic from it. Spend the real effort elsewhere.
• Someone is selling you "llms.txt optimization": don't pay for it. There's nothing to optimize that Google or the major assistants have said they reward.

What matters far more for showing up in AI answers:

1. Crawlable HTML with the answer in the page, not hidden behind JavaScript or a login.
2. A robots.txt that lets in the AI crawlers you want. Many sites block them by accident through a CDN rule or an old blanket disallow. Check yours with our free AI crawler checker.
3. Clear, quotable answers. A direct answer near the top of a page, like the short answer above, is easier for any system to extract.
4. Being mentioned in places people trust. Reviews, directories, comparisons, and communities where your category is discussed.

How to make one in five minutes

If you decide to add one:

1. List the 10–30 pages that best explain what you do: pricing, key features, docs, comparisons, your best guides.
2. Write a one-paragraph summary a stranger would understand.
3. Generate the file with our free llms.txt generator, or write it by hand using the structure above.
4. Serve it at /llms.txt as plain text. Make sure it returns a 200, not an error or a redirect to your homepage. A server error is the one thing Lighthouse flags.
5. Keep it in sync. The best llms.txt files are generated from the same source as your sitemap, so they never go stale.

That last point is how we do it. Our own llms.txt is generated from the same page registry that builds our sitemap and footer. When a page is added, it shows up everywhere at once. We did the same on Versely, our own product, as part of its technical base.

The bottom line

llms.txt is a reasonable idea with a small audience. It won't move your Google rankings, and Google says so in writing. It costs almost nothing to add, it helps agents that look for it, and it's a good forcing function to decide which of your pages actually matter.

Make one if it takes you five minutes. Then go fix your titles, your robots.txt and your content, because that's where the traffic is. If you want that work done on a schedule, it's what we're building Autopilot to do.