llms.txt: Do You Need One in 2026? What Google, AI Labs and the Data Say

Somebody on your team read that every website now needs an llms.txt file to show up in ChatGPT. An agency then quoted you a fee to "implement" one. Before you pay for anything, it is worth knowing what the file actually does, and who actually reads it.

The short version is reassuring. An llms.txt file is a small, optional text file that takes about 15 minutes to write. It is neither a ranking factor nor a shortcut into AI answers, and Google has now said so in writing.

This guide explains what the llms.txt proposal is, what Google and the major AI companies have and have not said about it, what real usage data shows, and when adding one is still sensible. You will also get a minimal example you can copy.

Key Takeaways

• llms.txt is a proposal, not a standard. Jeremy Howard proposed it in September 2024, and the spec reached version 2 in August 2026.
• Google Search ignores it. Google's AI optimisation guide says adding the file will neither harm nor help your visibility in Google Search.
• No major AI search crawler documents reading it. OpenAI, Anthropic and Perplexity publish llms.txt files for their own developer docs, yet their crawler documentation does not promise to read yours.
• Most files are never opened. Ahrefs found that 28% of sites in its sample publish one, and 97% of those files received zero traffic in May 2026.
• It is cheap, so add it if you like. A basic file takes minutes, and free generators exist in Yoast SEO and other tools.
• It should never be a paid core deliverable. Any agency selling llms.txt as the centre of an AI visibility plan is selling decoration.

Do You Need an llms.txt File?

Quick answer: No, you do not need an llms.txt file to appear in Google's AI Overviews, ChatGPT or Perplexity. Google says its Search systems ignore the file, and no major AI search engine documents using it for ranking or citation. It is still cheap to add, and it can help coding agents and developer tools read documentation sites, so treat it as a low-priority extra rather than a strategy.

That answer changes only in one situation: if your site is software documentation that developers feed into AI coding tools. For almost every other business, crawler access, clear content and genuine brand mentions matter far more.

What Is llms.txt?

llms.txt is a proposed convention for giving large language models a short, curated map of a website. According to the llms.txt proposal on llmstxt.org, it was written by Jeremy Howard, published on 3 September 2024, and last modified on 10 August 2026. The current text calls itself v2, "updated based on what I learned from two years of adoption".

The idea is simple. A web page is built for people, with navigation, scripts, ads and cookie banners. A language model working inside a limited context window benefits from a clean summary instead. So the proposal suggests a Markdown file at /llms.txt that lists your most useful pages, each with a one-line description.

The proposal also suggests offering clean Markdown copies of key pages, either by appending .md to the URL or replacing the extension with .md. That part is separate, and it is far more work than the file itself.

Crucially, the spec describes the file as something used on demand, when an agent needs information while helping a user. It is not a crawling instruction like robots.txt, and it is not a list of every URL like a sitemap.

llms.txt vs robots.txt vs sitemap.xml

These three files are often lumped together, but they do very different jobs.

In other words, robots.txt controls access, a sitemap aids discovery, and llms.txt is an optional reading list. Only the first two affect whether search engines can find you.

The llms.txt Format, With a Minimal Example

The llms.txt format is plain Markdown in a fixed order. The spec says the H1 with your project or site name "is the only required section". After that come, in order:

1. An optional blockquote with a short summary of the site.
2. Optional paragraphs or lists with extra detail.
3. One or more H2 sections containing "file lists", each a Markdown link plus an optional description.
4. By convention, a section headed ## Optional for secondary links an agent can skip when short on space.

Here is a minimal llms.txt file for a service business:

# Acme Solar

> Acme Solar designs and installs rooftop solar systems for homes and
> factories in Chennai, with net metering support and five-year maintenance.

## Services
- [Residential solar](https://example.com/residential-solar): System sizes, subsidy eligibility and installation timeline
- [Commercial solar](https://example.com/commercial-solar): Rooftop and ground-mount projects for factories and offices

## Pricing and support
- [Solar pricing](https://example.com/pricing): Price ranges by system size, updated each quarter
- [Contact](https://example.com/contact): Phone, email and site visit booking

## Optional
- [Blog](https://example.com/blog): Guides on net metering, subsidies and maintenance

Save it as a UTF-8 text file named llms.txt and upload it to your site root, so it loads at yourdomain.com/llms.txt. That is the whole job.

Does Google Use llms.txt?

No. This is now settled in Google's own documentation. Google's AI optimisation guide, last updated on 10 July 2026, lists llms.txt under its mythbusting section on what you do not need to do. It says you do not need new machine-readable files, AI text files, markup or Markdown to appear in Google Search, "as Google Search itself doesn't use them".

The same passage adds that creating the file for other services is "completely fine", and that doing so "will neither harm nor help your site's visibility or rankings in Google Search". Google's separate page on AI features makes the same point: there are no special optimisations needed to appear in AI Overviews or AI Mode.

There is one wrinkle worth knowing. Chrome's Lighthouse tool now includes an llms.txt check in its Agentic Browsing audits. According to Chrome's Lighthouse documentation, the audit only flags a server error when fetching the file, marks a missing file (a 404) as not applicable, and states that "providing the file is optional at the moment". So a Lighthouse audit is not a sign that Google Search wants the file. It checks that your server behaves sensibly for browser agents.

Do OpenAI, Anthropic or Perplexity Use llms.txt?

This is where most articles overstate things. Here is what each company actually documents, based on their official crawler pages.

The pattern is clear. AI labs use llms.txt as documentation publishers, not as crawlers of your site. The llms.txt spec itself notes that OpenAI, Anthropic and Gemini publish llms.txt files for their own developer docs. That helps developers paste those docs into coding tools. It says nothing about whether ChatGPT search or Claude reads the file on a plumber's website.

OpenAI's crawler documentation is a good example. The page links to OpenAI's own llms.txt as a documentation index, yet its guidance for site owners is entirely about robots.txt: allow OAI-SearchBot if you want to appear in ChatGPT search results. What really controls your AI visibility is covered in our guide to why a website is not showing in ChatGPT.

What the Usage Data Shows

The most useful evidence comes from server logs rather than opinions. In June 2026, Ahrefs published an llms.txt study covering 137,210 domains in Ahrefs Web Analytics that received traffic in May 2026.

According to that study:

• 28% of the domains publish an llms.txt file, so adoption is already widespread.
• 97% of those files received zero traffic during May 2026.
• 96% of requests to the files that were read came from bots rather than people.
• 19.5% of fetches came from named AI tools, with agentic infrastructure (10.5% of requests) the largest AI category.
• 12% of requests came from tools studying llms.txt, such as checkers and audit tools, rather than systems consuming it.

Ahrefs' own conclusion is blunt: if your goal is showing up in ChatGPT, Perplexity or AI Overviews, the file is "largely decoration". That matches what the AI companies document. Retrieval bots, the kind that fetch pages for live AI answers, made up just 1.1% of AI requests in the Ahrefs data. The readers that do exist lean towards agents and developer tools, which fits the file's origins in software documentation.

When Adding llms.txt Still Makes Sense

Given all that, why add one at all? Because it is nearly free, and there are a few cases where it has real value.

• Developer documentation and APIs. Coding agents are the heaviest readers, so SaaS products, SDKs and API docs benefit most.
• Browser agents. If you care about Lighthouse's Agentic Browsing checks, a clean file avoids a server error and gives agents a summary.
• Your CMS already does it. If your SEO plugin can generate the file in one click, switching it on costs nothing.

For a local clinic, a manufacturer or a law firm, however, the honest priority is low. It will not move your AI citations, so do it only after the work that does.

llms.txt Generators and Plugins

You rarely need to write the file by hand. The llms.txt spec lists several platforms that generate it automatically, including Mintlify, GitBook, Yoast SEO, AIOSEO and Wix, plus plugins for VitePress and Docusaurus.

On WordPress, Yoast's llms.txt feature is available in the free plugin, according to Yoast's own feature page. It can build the file automatically, prioritising cornerstone content, or let you pick the pages yourself, and it refreshes the file weekly.

Before trusting any llms.txt generator, check three things:

1. The links resolve. Every URL in the file should return a 200 status, not a redirect or a 404.
2. The descriptions are accurate. Auto-generated summaries sometimes describe pages vaguely or wrongly.
3. It stays short. A curated list of 10 to 30 key pages is the point. A dump of every URL is just a worse sitemap.

How to Add llms.txt in 15 Minutes

If you decide to add one, follow these steps:

1. List the 10 to 30 pages that best explain your business: services, pricing, key guides, contact.
2. Write a one-line description for each, stating what the page answers.
3. Arrange them under a few H2 headings, with an ## Optional section for secondary links.
4. Add an H1 with your business name and a blockquote summary of what you do.
5. Upload the file to your root as /llms.txt, or switch on your SEO plugin's generator.
6. Open yourdomain.com/llms.txt in a browser and confirm it loads as plain text.
7. Check that your robots.txt and firewall do not block the crawlers you care about. Our free robots.txt tester shows how specific bots are treated.
8. Set a reminder to update the file when you add or retire key pages.

Why llms.txt Should Never Be a Paid Core Deliverable

Here is the practical risk. Because the file sounds technical and new, some agencies sell it as the centrepiece of an "AI optimisation" package. That is a warning sign we cover in our list of AEO agency red flags.

A reasonable agency may add the file as a 15-minute housekeeping task. It should not be a line item that justifies a monthly fee, and nobody can honestly promise that it will get you cited. If a proposal leads with llms.txt, ask what else is in it.

The work that actually influences AI answers looks different:

Google's own guide explains why ranking matters: its AI features are "rooted in our core Search ranking and quality systems". That is why our guide to ranking in Google AI Overviews starts with ordinary SEO fundamentals rather than special files.

If you want a specialist team to handle the work that moves AI answers, our AEO services cover crawler access, content, entity consistency and measurement, with llms.txt included as a small extra rather than sold on its own.

Frequently Asked Questions

Final Thoughts

The llms.txt debate has generated far more noise than the file deserves. Google ignores it for Search, the major AI search crawlers do not document reading it, and the log data shows most files sit unopened.

That does not make it harmful. It makes it small. Add the file if your plugin offers it, or if you publish documentation for developers, then spend your real effort on crawler access, clear answers and genuine authority. Treat llms.txt as a cheap extra, never as the reason you hire anyone.

About the author

Ganesh Shanmugam

Ganesh Shanmugam is an SEO and AEO consultant based in Chennai, India, and the founder of Billion Game — a boutique agency serving clients across the US, UK, Australia, Canada, and Ireland. He previously held SEO roles at 7 Eagles and Digital Scholar before going independent, and brings 4+ years of hands-on experience

More from Billion Game →