Origin.
llms.txt is a low-complexity file proposal, but low implementation effort does not prove retrieval or citation value. Define the intended consumer and evidence before treating it as visibility work.
Marketing Atlas · Reference · AI Search
llms.txt.
Updated May 2026 · Reference page · Written marketing plan
A proposed plain-text file at a domain root for listing selected site resources for language-model use. A voluntary resource-list proposal, separate from robots.txt.
The numbers underneath
What does this concept change in AI search?
•Plain text · URL list + summary
•Located at /llms
•Llmstxt
Section 01 · Quick definition
Definition.
In one pass
The llms.txt proposal defines a plain-text file at a domain root that can list selected resources for systems that choose to use it. The format is markdown-flavored: a heading with the site name, a short summary, optional metadata, and a list of important URLs grouped by section.
The structural assessment
The proposal is published at llmstxt.org. Use is voluntary, and support must be verified from current vendor documentation and server evidence.
Section 02 · Why it matters
Why it matters.
Mechanic.
A curated resource list can describe which pages the publisher considers important. Whether a crawler discovers, retrieves, prioritizes, or scores those pages from llms.txt depends on that crawler's documented behavior.
The practical stake is test design, not assumed adoption. Keep the proposed resource list factual and current, then verify whether the intended consumer requests or documents support for it.
Section 03 · How it runs
How to assess an llms.txt implementation.
The llms.txt proposal describes a markdown file at /llms.txt that can list selected site resources. It does not replace robots.txt or sitemap.xml. No crawler's discovery, ingestion, prioritization, or scoring behavior is guaranteed by the proposal; verify current vendor documentation and server logs before relying on it.
01
Step one · root fetch
If publishing the proposed file, serve it at /llms.txt with an appropriate text content type and confirm the response. Do not assume a crawler requests it; verify requests in server logs.
02
Step two · header and summary
Follow the proposal's heading and summary format if you publish the file. Treat the summary as publisher-supplied text, not proof that a model uses it as an editorial frame.
03
Step three · sectioned URL list
Group selected URLs clearly and keep them current. The list expresses the publisher's selection; it does not establish a crawler's discovery or priority order.
04
Step four · retrieval and citation
Publish the file only as a documented or measured experiment. Check vendor support and server logs; do not assume listed pages receive retrieval or citation preference.
The shift this concept names
The llms.txt proposal defines a plain-text file at a domain root that can list selected resources for systems that choose to use it.
Before applying this concept
“llms.txt is robots.txt for AI. Same job.”
After applying this concept
Vendor documentation and server evidence decide whether the file is used. The proposal itself does not establish retrieval or citation preference.
Section 04 · Common misunderstandings
What people get wrong.
Misunderstanding 01
“llms.txt is robots.txt for AI. Same job.”
robots.txt carries crawler directives. llms.txt proposes a resource list and does not replace those directives. Support and use vary by vendor.
Misunderstanding 02
“If the file is voluntary, it doesn't help.”
Voluntary in the sense that the spec is community-maintained, not imposed by a regulator. Do not infer llms.txt support from the presence of a vendor crawler. Verify each vendor's current documentation and inspect server logs before claiming the file is used.
Misunderstanding 03
“We have a sitemap, so we don't need llms.txt.”
A sitemap can list crawlable URLs for search systems. llms.txt is a proposed curated resource list. Whether any system uses that list for retrieval or citation must be verified. They have different proposed roles, and neither file proves that a specific system uses it.
Misunderstanding 04
“Listing every page in llms.txt makes us more visible.”
Follow the proposal's format and keep any resource list concise and current. Do not claim a URL count changes model weighting without evidence.
Misunderstanding 05
“The summary should match our home-page hero copy.”
Keep the summary plain, concrete, and factual. Do not assume a model will quote it; treat it as publisher-supplied text for systems that document or demonstrate support.
Section 05 · Questions to ask
Questions a Stan Consulting marketing review asks.
Does the domain serve a valid /llms.txt at the root, returning 200 OK with text/plain or text/markdown content type?
Does the domain serve a valid /llms.txt at the root, returning 200 OK with text/plain or text/markdown content type?
Does the heading and summary state the publisher's identity and purpose accurately, without promotional claims?
Are the listed URLs the canonical answers for the questions the brand wants to be cited on, or is the list a dump of every page on the site?
Do the section headings describe the listed resources accurately for any reader or documented consumer?
Does an /llms-full.txt exist with the same URLs and rendered content, or is there a reason to keep the longer file out of scope?
Has the file been updated in the last 90 days to reflect current canonical pages, or is it pointing at a 2024 site map?
Does the AI-Generated-Content section (if used) accurately label which pages on the site are AI-assisted versus human-written?
Stan's take · four points
llms.txt does not replace robots.txt and should not be presented as a crawler directive.
The proposal defines a resource-list format, not a technical directive or guaranteed model input.
The job is to keep the proposed file accurate, concise, and useful without assuming a model reads or quotes it.
A useful resource list should read like clear documentation, not a hero banner. Whether any model uses it must be established separately.
Stan Tscherenkow · Principal · Stan Consulting LLC
Section 06 · Adjacent concepts
Related Atlas entries.
Section 07 · Sources
Sources.
llmstxt.org · The /llms.txt specification
The community proposal for a plain-text llms.txt resource list. It describes a heading, summary, and URL-list format; vendor support must be verified separately.
Anthropic · llms.txt documentation
Anthropic's current llms.txt documentation. Use only the behavior documented on the linked page; do not infer ClaudeBot or citation effects.
OpenAI · OAI-SearchBot and crawler conventions
OpenAI's reference on crawler user agents and robots.txt controls. It does not by itself establish llms.txt support.
Search Engine Land · GEO and llms.txt coverage
Practitioner reference on how llms.txt fits into a broader generative engine optimization program, and operator playbooks for writing the file well.
Search Engine Journal · llms.txt explained
Practitioner reference covering the spec, common implementation mistakes, and the editorial framing distinction between llms.txt and robots.txt.