Right now, while you're reading this, AI tools are crawling websites trying to understand what businesses do, what they offer, and whether they're worth mentioning to the person asking.
The problem? Most websites are like messy closets. Everything's there, but good luck finding what you need quickly.
That's where LLMS.txt files come in. Think of them as your website's personal concierge for AI visitors.
A lot has changed since we first published this post in June 2025, so we've rewritten it with what the file is, who actually reads it in 2026, and why we still think it's worth 15 minutes of your time.
What is an LLMS.txt file?
An LLMS.txt file is a welcome mat for AI systems visiting your website. It's a simple text file that sits in your website's root folder and tells AI, “Here's the most important stuff about us, and here's where to find it.”
Imagine you're hosting a dinner party. Instead of letting guests wander around your house looking for the bathroom, you put up a little sign that says “Bathroom upstairs, snacks in the kitchen, good conversation in the living room.” That's what llms.txt does for AI visitors.
It's written in plain markdown, no coding degree required. A short description of your business, your key pages, and links to the content you most want AI tools to read.
The idea came from Jeremy Howard, co-founder of Answer.AI, who proposed it in September 2024. It's a proposal rather than an official web standard, and Howard revised it in August 2026 based on 2 years of real-world use. You'll also see companion files named llms-full.txt around the web, which hold a site's full content in one document; that's a convention started by documentation platforms, and most small businesses only need the main file.
What is an LLM file?
Before we go further, let's clear up some confusion. When people say “LLM file,” they usually mean LLMS.txt. It's just shorthand.
LLM stands for “large language model,” the technical term for AI systems like ChatGPT, Claude, and Google's Gemini. So an “LLM file” is any file designed to help those systems understand your website, and LLMS.txt is the version with a published format.
What is robots.txt used for?
You might have heard of robots.txt. It's been around since the early days of the internet. Think of it as the bouncer at your website's door: it tells search engine crawlers what they can and can't access.
Robots.txt is all about saying “no”:
- “Don't index my admin pages.”
- “Stay out of my private folders.”
- “Don't crawl my duplicate content.”
It's a protective tool, and it still matters. AI crawlers like GPTBot and ClaudeBot respect robots.txt rules, so it's also where you control which AI companies can use your content at all.
What's the difference between robots.txt and LLMS.txt?
Robots.txt is defensive. It says, “Don't go here, stay away from this.” It controls access.
LLMS.txt is a guide. It says, “Here's exactly what you should know about us, and here's our best content.” It gives an AI system the map.
Think about it this way:
- Robots.txt is like putting up “Do not enter” signs
- LLMS.txt is like hiring a tour guide who walks visitors past your best features
The two files do different jobs, and having one says nothing about the other.
Okay, but does anything actually read this file?
Time for straight talk, because you'll run into this the second you search the topic.
Google's systems don't use llms.txt. John Mueller said in June 2025 that no AI system was reading the file, and compared it to the old keywords meta tag. Google's AI optimization guide (May 2026) now says it plainly: creating AI text files “will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them.”
The server logs back him up. Ahrefs analyzed 137,210 domains with May 2026 data and found that 97% of the sites with a valid llms.txt file got zero requests for it that month, from anything at all. Of the requests that did happen, AI retrieval bots made about 1%. ChatGPT's crawler, Claude's crawler, and Perplexity's crawler read your regular HTML pages instead.
Someone even ran the control experiment. SEO consultant Mark Williams-Cook invented cats.txt, a joke file listing his agency's office cats, and it passed every test llms.txt advocates cite as proof: the AI bots fetched it, Google indexed it, and Google's AI Overview confidently described a fictional cat. “A bot fetched our file” turns out to prove nothing at all.
So if anyone promises that an llms.txt file will lift your rankings or get you cited in AI Overviews, walk away. Google has said in writing that it won't.
So why are we still telling you to make one?
Because the file found a different job.
While AI search ignored it, the agentic web picked it up. AI agents are tools that visit your site to do something on a user's behalf: gather your pricing, compare your services, fill out your contact form. Those agents work better when a site hands them a map, and llms.txt has become the usual place to put one. In the Ahrefs data, the AI tools fetching these files most were coding agents, which is exactly that use case.
The clearest signal came in May 2026, when Google's own Lighthouse tool (version 13.3.0) added llms.txt to its default agent-readiness audits. To be precise about what it checks: the audit flags a file that errors when fetched, and treats a missing file as optional. So Google Search ignores the file while Google's developer tools test for it. Welcome to 2026.
The companies building AI publish these files for their own properties too. Anthropic maintains one for its developer docs, OpenAI does the same, and WordPress.org has one covering the software that runs 43% of the web. Adoption keeps climbing, mostly because platforms switched it on for everyone: an HTTP Archive analysis by Casey Burridge (June 2026) found 5.07% of the top million sites publish a valid file, with the top 10,000 tier up about 5.4x in a year. Shopify and Wix now generate the file automatically, and Yoast has shipped it since June 2025.
And the math hasn't changed: this takes about 15 minutes, costs nothing, carries no risk, and your human visitors never see it. We think that's a sensible bet on where the web is heading, as long as nobody's selling it to you as an SEO trick.
Your LLMS.txt questions answered
Wait, how is LLMS.txt actually different from robots.txt again?
Robots.txt is a security guard who tells visitors which rooms they can't enter. LLMS.txt is a friendly receptionist who says, “Welcome! Here's what we do best, and here's where you'll find our most helpful information.” One controls access, the other gives directions.
Does Google use llms.txt?
No. Google has confirmed that llms.txt plays no role in Search, AI Overviews, or AI Mode, and its crawlers don't read the file. The reason to have one is AI agents and assistants that visit your site directly, plus Lighthouse's agent-readiness checks.
Do I need to be technical to set this up?
Not at all. An LLMS.txt file is a text file that you save with a specific name and place in your website's main folder. If you can create a Word document and save it, you can create one. We show you exactly how in the next post in this series.
Will this actually help my business get found online?
In Google? No, and we'd rather tell you that plainly than sell you magic. Where it can help is with AI agents and assistants that visit your site directly, a small slice of traffic today that keeps growing. Treat it like buying the domain name of a product you might launch: cheap insurance, quick to do, useful if the bet pays off.
How often do I need to mess with this file once it's set up?
Review it when something big changes: new services, new contact info, a site restructure. A quarterly check-in is plenty. This lives on the same list as your plugin updates and backup checks, which is to say, the boring maintenance that quietly keeps a site healthy.
What if AI systems don't support this yet?
Most don't, and that's the honest answer. But the file takes 15 minutes, the format is simple, and the tools being built for AI agents (including Google's Lighthouse) already check for it. If broad support arrives, you're done. If it doesn't, you spent 15 minutes.
You didn't start your business to become a WordPress technician, yet here you are treating your most critical business asset like a hobby project. While you're spending hours on updates and crossing your fingers that nothing breaks, your competitors are focused on what actually makes money.
👋 I'm Jeane Sumner, and Website HQ doesn't just host your site – we armor it, optimize it, and monitor it so you can get back to running your business instead of fixing it. Book your Site Care strategy call and never worry about your website again.
Ready to make one? The step-by-step guide walks you through creating and publishing your first LLMS.txt file, no technical experience required. And the industry leaders post shows what Yoast, Anthropic, and Shopify put in theirs.
In this series:
- Why Your Website Needs an LLMS.txt File (And What the Heck That Even Means) –This article!
- What Industry Leaders Put in Their LLMS.txt Files (And How to Steal Their Strategies)
- How to Create Your First LLMS.txt File: A Step-by-Step Guide
- The LLMS.txt Domain Dilemma: Where Should You Actually Put Your File?
- How to Measure AI Traffic (And Whether Your LLMS.txt File Is Doing Anything at All)
Google now lets you pick which sites get priority in your Search and AI results. If this series helped, add us. Two clicks, and you can undo it anytime.
