Websites have carried a file for machines for decades: robots.txt tells search-engine crawlers where they're welcome. More recently there's a proposed counterpart aimed at AI systems: llms.txt — a small text file at a fixed address in which a website introduces itself briefly and machine-readably.
The idea
The format was proposed by developer Jeremy Howard in September 2024. The reasoning: language models reading a website struggle with the same problem their users do — limited context. A typical web page is mostly navigation, ads and chrome; extracting the substance is expensive. llms.txt inverts that: one lean Markdown file stating what this site is and where its key content lives.
The format is deliberately simple: a heading with the name, a blockquote with the one-sentence summary, link lists to the core content, optionally a section for the skippable parts. No new protocol, no registration — just a text file.
What ours looks like
All the websites in our little family serve llms.txt from one shared implementation. This site's version — live at lostcontext.ai/llms.txt — reads (in German):
# lostcontext.ai
> Aufklärung über künstliche Intelligenz – Nutzen und Risiken,
> Erfahrungen und Einordnung von einem Entwickler, der damit
> Anwendungen baut. Ohne Hype, ohne Panik.
## Seiten
- [Startseite](https://lostcontext.ai/)
## Optional
- [Sitemap – alle URLs der Site](https://lostcontext.ai/sitemap.xml)
That's all of it. The effort was a few lines of code, written once, shared by every site.
What it buys — honestly
Here comes the cold shower: llms.txt is a proposal, not a standard. No standards body has ratified it, and whether the major AI providers actually read the file while crawling is not publicly established — even the project page makes no such claims (as of July 2026). Anyone selling you llms.txt as a "secret AI-SEO weapon" is selling hope.
Why do we serve it anyway? The arithmetic is simple. Cost: one static text file, close to zero upkeep. Potential benefit: tools and agents that know the convention instantly find a clean self-description — some documentation sites already use the format actively, for example the MCP protocol docs, which point their AI readers straight at their own llms.txt. And should the convention take off, we're already there. A friendly door that costs little: put it up, don't expect much of it.
One clarification, because it gets muddled a lot: llms.txt invites and describes — it is not a mechanism for keeping AI crawlers out. Controlling which bots may harvest your content remains the job of robots.txt and your terms of use.
What to take away
If you run a website yourself: llms.txt is an afternoon project and does no harm — as long as you treat it as what it is: a polite business card for machine readers with uncertain reach, not a ranking lever. And next time you read that one simple trick makes websites "visible to AI": the standard this site holds itself to applies here as well — check what's substantiated before you invest.
/compact — the essentials, if context is running low:
llms.txt is a small text file at a fixed address, proposed in 2024, in which a website introduces itself briefly and machine-readably to AI systems — a counterpart to robots.txt. It is a proposal, not a standard: whether the major AI providers actually read the file is not publicly established, and it cannot keep crawlers out — that remains the job of robots.txt. Since it costs next to nothing, it is worth serving as a friendly door for tools and agents that know the convention — put it up, don't expect much of it.