llms.txt Implementation: What It Does and Does Not Do

Description

Service llms.txt implementation: what the file does, and what it does not

llms.txt is a Markdown file at the root of a site that lists its most important pages for AI agents. It is cheap to publish. Google states that Search ignores it, and a 2026 log study of 137,210 domains found that 97% of published files were never requested. WavX adds one as a small item within an audit, and does not sell it as a result.

Discuss your SEO/GEO/AEO strategy Last updated 2 October 2026

What llms.txt is

llms.txt is a text file, written in Markdown, placed at the root of a website ( yourdomain.com/llms.txt ). It gives an AI agent a short description of the site and a curated list of links to the pages that matter, so that the agent does not have to work the site out from its navigation.

It was proposed by Jeremy Howard in September 2024. The current text, on llmstxt.org, is version 2, last modified in August 2026, and it still describes itself as a proposal. No standards body has adopted it.

The format is deliberately small:

one H1 with the name of the site, which is the only required part

a blockquote with a short summary

optional plain paragraphs or lists of notes

sections under H2 headings, each a list of links with a one-line description

a section named "Optional" for links an agent can skip when it has little room

An example for an invented clinic:

# Example Dental Clinic

> A dental clinic in Sector 56, Gurgaon. Treatments, prices and booking

> details are on the pages below. Prices are checked monthly.

## Treatments

- [Root canal treatment](https://example.com/treatments/root-canal): what is done, number of visits, price

- [Braces and aligners](https://example.com/treatments/braces): options, duration, price

## Practical

- [Fees](https://example.com/fees): full price list with the date last checked

- [Book an appointment](https://example.com/book): hours, address, phone

## Optional

- [About the dentists](https://example.com/team): names and qualifications

Two things it is often confused with:

It is not robots.txt. It permits nothing and blocks nothing. Crawler access is controlled in robots.txt and at the CDN or firewall. The AI crawler directory lists the crawlers and their names.

It is not a sitemap. A sitemap lists every indexable page for search engines. llms.txt is a short, selected list with descriptions.

The plain-language version of all this is in llms.txt: what it does and what it does not.

What the evidence says

The file is widely recommended as a way to be found by AI search. The published evidence does not support that.

Question

Finding

Source

Does Google use it?

No. "Google Search itself doesn't use them." Publishing one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them"

Google Search Central, guide to generative AI features; clarified 15 June 2026

Do other engines say they use it?

The crawler documentation of OpenAI, Perplexity and Anthropic, read on 2 October 2026, does not say that any of their search crawlers reads llms.txt. Ahrefs puts it this way: no major AI platform has committed to reading it

Vendor documentation; Ahrefs

Is the file requested at all?

Of 137,210 domains, 28% published an llms.txt. 97% of those files received no requests of any kind in May 2026

Ahrefs log study, June 2026

Who requests the rest?

Among the files that were fetched: SEO audit tools 21.7% of requests, AI agents 10.5%, AI training crawlers 5.3%, AI assistants 2.5%, AI search crawlers 1.1%. About 12% came from tools that score or catalogue llms.txt files

Ahrefs

Do AI crawlers look for a file that is missing?

No. Requests for missing llms.txt files came almost entirely from people, and none from AI bots

Where is it used?

Mostly software documentation. The proposal says the files are "used most heavily for software documentation, where coding agents follow them to find API references and tutorials"

llmstxt.org

Ahrefs adds two cautions about its own numbers. Its sample leans towards technical, SEO-aware sites, so 28% adoption is an upper bound. And a request is not a reading: the study counts fetches, so real use is at or below the figures above.

The conclusion is narrow and clear. If the aim is to appear in ChatGPT, Perplexity or Google's AI features, an llms.txt file does not move the result today. If the site is developer documentation that coding agents consult, the file has a real audience.

Who should have one

Kind of site

Worth doing?

Why

Developer documentation, an API or an SDK

Yes

This is the use the proposal describes and the one the log data supports. The proposal notes that OpenAI, Anthropic and Gemini publish llms.txt files for their own developer docs

Software product with a help centre

Reasonable

Customers who use AI assistants to work out your product may point them at it

Business website: a clinic, a firm, a store

Harmless, low value

Fine if your platform produces it for nothing. Do not expect an effect

Any site, for the purpose of Google AI Overviews

No

Google has said in writing that Search ignores the file

How WavX handles it

WavX treats llms.txt as housekeeping inside an AI visibility audit or a wider GEO engagement . The steps are short.

Check whether the platform already makes one. The proposal's own list of integrations includes Wix, which generates the file for every site, and the WordPress plugins Yoast SEO and AIOSEO. If yours does, the job is to review what it produced.

Select the pages. A good file is short. It links to the pages that explain what the business does, what it charges and how to reach it, and leaves the rest out.

Write the summary and the one-line descriptions as facts. No slogans. Nothing in the file should say more than the visible pages do.

Generate it from the site's own data where possible. On a Next.js site, WavX builds the file from the same source as the sitemap, so that it cannot fall out of step with the pages.

Make it findable. The current proposal recommends a rel="describedby" link pointing to the file. Ahrefs found that agents fetch the file when something directs them to it, and do not go looking.

Treat it like code. Keep it in version control, restrict who can change it, and keep its content to links and descriptions.

Check the logs after a month. If nothing has requested the file, the report says so.

llms-full.txt , a single file with the full text of the key pages, is added on request. The proposal does not define it.

Doing it yourself

For most business sites this is the right route.

Use the free llms.txt generator on this site and upload the result to the root of your domain.

Or switch on the option in your platform or SEO plugin, then read what it generated and remove anything you would not want quoted.

Recheck it whenever prices, services or addresses change. A file that contradicts the site is worse than no file.

For a small site this takes under an hour.

What it does not do

It does not improve rankings or AI citations. There is no published evidence that it does, and Google has stated that it does not for Search.

It does not let crawlers in or keep them out. A site that blocks OAI-SearchBot or PerplexityBot in robots.txt is still blocked, whatever llms.txt says.

It does not fix a site that engines cannot read. If the text of your pages is missing from the HTML, or the pages are not indexed, that is the fault to repair.

It does not replace the pages. An engine that cites you links to a page. The page still has to state the fact clearly.

When you should not pay for this

Nearly always, if it is the only thing being bought. A vendor selling llms.txt as a standalone route into AI answers is selling something the evidence does not support. The work that is associated with AI visibility is elsewhere: being reachable by each engine's search crawler, being indexed, stating sourced facts clearly on your own pages, and being listed, reviewed and mentioned on other sites. That is what the audit examines, and what AI citation and referral tracking then measures.

WavX includes the file because it is cheap, because the documentation use is real, and because the position may change. If it does, the engines will say so in their documentation, and this page will be updated with the date.

Frequently asked questions

Will llms.txt help my site appear in ChatGPT, Perplexity or Google AI Overviews?

The evidence says no, as of October 2026. Google's guide states that Search ignores the file and that it will neither harm nor help. In Ahrefs' study of 137,210 domains, the search crawlers of OpenAI, Perplexity and Anthropic together made only a couple of hundred requests for llms.txt files in a month. We tell every client this before adding one.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers what they may fetch. llms.txt is a reading list: a short description of the site and links to its key pages. It grants nothing and blocks nothing. If you want to control which AI crawlers can read your site, that is done in robots.txt and at your CDN or firewall.

Is there any harm in having one?

Little, if it is kept accurate. The risks are a stale file that describes pages or prices you have since changed, and a file that someone can edit without review. Ahrefs found research crawlers already probing llms.txt files as a route for prompt injection, so the file should contain plain links and descriptions and be changed only through the same review as the rest of the site.

Should I also publish llms-full.txt?

It is optional and the published proposal does not define it. Some sites publish one as a single file holding the full text of their main pages. The proposal itself suggests a different route: a Markdown version of each important page at the same address with .md added. Either is reasonable for documentation. Neither has evidence of affecting search visibility.

What does WavX charge for llms.txt?

There is no separate price. It is a small item inside an audit or a wider GEO engagement, and GEO work is quoted after an audit. If the file is all you want, use the free generator on this site or your platform's built-in option.

Sources

llmstxt.org: The /llms.txt file, v2 (Jeremy Howard; published 3 September 2024, modified 10 August 2026) · read 2 October 2026

Google Search Central: Optimizing your website for generative AI features on Google Search · read 2 October 2026

Google Search Central: latest documentation updates (llms.txt clarification, 15 June 2026) · read 2 October 2026

Ahrefs: We analyzed 137K sites, 97% of llms.txt files never get read (15 June 2026) · read 2 October 2026

OpenAI: overview of OpenAI crawlers · read 2 October 2026

Perplexity: Perplexity crawlers · read 2 October 2026

Anthropic: does Anthropic crawl data from the web, and how can site owners block the crawler? · read 2 October 2026

Related

Free llms.txt generator

AI visibility audit

Generative engine optimisation (GEO) services

Build your own software — your way, your pricing.

WavX Solutions is here to create your own software in a fully custom way, built exactly how you work — with a pricing model that fits your business. Connect now and let's build it.

Contact Now helpwavx@gmail.com