If you have read anything about AI SEO this year you have been told to add an llms.txt file to your website. Some agencies sell it as a deliverable. Some tools grade you on it. We check for it in our own scan. So it is fair to ask the question plainly: does the file do anything? Here is what Google says, what the AI companies’ own documentation says, what the largest study to date found, and what we do about it on client sites.
What llms.txt is
llms.txt is a proposal, published at llmstxt.org in September 2024, for a plain-text file at the root of a website (yourdomain.com/llms.txt) that describes the site to large language models in Markdown: a one-line summary, a short description, and a curated list of the pages that matter, each with a sentence about what it contains (llmstxt.org). The idea borrows the shape of robots.txt and the intent of a sitemap, and it was aimed first at developer documentation, where a model reading a library’s docs benefits from a clean index.
# Example Plumbing > Licensed plumbing and water heater service in Greensboro, NC since 2009. ## Services - [Water heaters](https://www.example.com/water-heaters/): repair and replacement, same-day - [Drain cleaning](https://www.example.com/drain-cleaning/): residential and commercial ## About - [Our team](https://www.example.com/about/): licensed NC plumbing contractor, license number on the page
What Google says
Google is unambiguous. Its guide to optimizing for generative AI features, published in May 2026, states: “You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn’t use them” (Google Search Central). Google’s search team has said the same thing informally for a year: no Google Search system reads llms.txt. If your goal is AI Overviews or AI Mode, the file is irrelevant.
What OpenAI, Anthropic and Perplexity say
Nothing. Each company publishes documentation for its crawlers: OpenAI for GPTBot, OAI-SearchBot and ChatGPT-User (OpenAI), Anthropic for ClaudeBot, Claude-User and Claude-SearchBot (Anthropic), Perplexity for PerplexityBot and Perplexity-User (Perplexity). All three describe how the agents honour robots.txt. None mentions llms.txt. As of this writing no major AI search provider has documented reading the file in production.
What the data says
In June 2026 Ahrefs published the largest measurement so far. Using server logs from 137,210 domains on its web analytics product in May 2026, it found that 28 percent of them had published an llms.txt file, and that 97 percent of those files received zero requests during the month (Ahrefs). Of the 3 percent that were fetched at all, most requests came from ordinary tools and people, not AI systems. And one line in the study settles the question of discovery: AI bots made zero requests for llms.txt files that did not exist. They never go looking.
28 percent of the domains had published one. AI bots made no requests for files that did not exist.
So why do we still publish one?
Three reasons, none of them “because it ranks.”
- It costs ten minutes. A file that might help a minority of tools and cannot hurt is a reasonable hedge at that price. We would not pay for it as a line item, and we do not charge for it as one.
- Writing it is useful. To write llms.txt you have to write a one-paragraph description of the business and a list of the five pages that matter. Every other step of an AI SEO install needs exactly that: the schema description, the Google Business Profile description, the About page. The file is a forcing function.
- Some smaller tools read it. Coding assistants and a handful of research tools do fetch it for documentation sites. For a local business that is close to nobody, and we say so.
What we do not do is weight it heavily, sell it as a strategy, or let it substitute for the things that measurably matter.
What actually matters instead
The same guidance from Google that dismisses AI text files lists what helps: content that is unique and useful, pages that are crawlable and render their text in HTML, structured data that matches what the page shows, a good page experience, and for a local business, a complete Business Profile (Google Search Central). Our own data agrees. Across 79 Piedmont Triad businesses we scanned in August 2026, 94 percent had no FAQ markup, 47 percent no local business schema, and on 97 percent of sites the reviews were invisible to a crawler (DiamondBack Advertising). None of those gaps is fixed by a text file.
| What it does | Who reads it | |
|---|---|---|
| robots.txt | Tells crawlers what they may fetch; every AI company’s search and training agents honour it | Every documented crawler |
| sitemap.xml | Lists the pages you want indexed; declared in robots.txt | Search engines, and through them the AI features built on their indexes |
| llms.txt | Describes the site to language models in Markdown; a 2024 proposal | No major AI search provider has documented reading it; 97 percent of files got no requests |
The honest recommendation
Publish one if it takes you ten minutes, then spend the rest of the afternoon on the four layers in the AEO guide, which is where the work actually is. If someone quotes you a price for an llms.txt file on its own, ask them which engine reads it. Ask them for the documentation.
Frequently asked questions
Does Google use llms.txt?
No. Google’s May 2026 guide to optimizing for generative AI features states that you do not need AI text files or Markdown to appear in Google Search, including its AI features, because Google Search does not use them.
Does ChatGPT read llms.txt?
OpenAI’s crawler documentation for GPTBot, OAI-SearchBot and ChatGPT-User does not mention the file, and Ahrefs found AI bots made zero requests for llms.txt files that did not exist. There is no documented production use.
Is there any harm in publishing llms.txt?
No. It is a plain-text file that costs nothing to host and cannot affect rankings. The only harm is treating it as a strategy or paying for it as one.
What should I do instead of llms.txt?
Make sure your pages are crawlable and render their text in HTML, add structured data that matches the page (LocalBusiness, Service, FAQPage), answer real customer questions on your pages, keep business details consistent everywhere, and make reviews readable to crawlers.
Why does DiamondBack’s scan check for llms.txt?
Because the check costs nothing and the file is harmless, we include it and weight it lightly. The score is driven by structured data, crawler access, rendering, answerability and consistency, which are the checks that move visibility.