"A competitor's site added a file called llms.txt. It seems to be a new standard for getting cited by AI, so we want to add it immediately as well"—over the past few months, requests like this have arrived in succession. Just recently, a web manager at a B2B manufacturer with over a billion yen in annual revenue (name withheld) asked us, "A consultant quoted 300,000 yen to implement llms.txt; is that reasonable?" Before looking at the price, there is no way to evaluate it without clarifying what that 300,000 yen actually delivers. As a team undertaking implementation in custom development, we believe decisions should be based on cost-effectiveness using verified facts, rather than adopting something just because it is trending or rejecting it outright as suspicious.
In this article, we examine the llms.txt file itself without overestimating or underestimating its value. To state our conclusion upfront: adding it causes little harm, but dedicating extensive engineering hours to it is unjustified. The places where you should truly focus your budget lie elsewhere.
What llms.txt Is, and What It Is Not
The llms.txt file is a Markdown file designed to summarize and guide AI crawlers and large language models (LLMs) through a website's content. Placed in the site's root directory (/llms.txt), the concept is to compile what the site is about and what is written on each page in an easily readable format for AI. While robots.txt tells crawlers what they may or may not view, llms.txt is often described as proactively guiding AI on how you want your company understood. It gained rapid traction in late 2025, billed as a new standard for LLM Optimization (LLMO) and Generative Engine Optimization (GEO).
What you need to understand here is that llms.txt is not an official standard, but merely a proposed specification. It is at the stage where an idea put forward by someone has spread; search engines and AI vendors have made no commitments to reference it whenever present. Much commentary hypes it as mandatory without sharing this premise, so it is necessary to examine it with a cool head.
Google Is Distinctly Skeptical
The weightiest piece of evidence is Google's official stance. Google's John Mueller remarked that llms.txt "amounts to the keywords meta tag." The keywords meta tag refers to <meta name="keywords">, once heavily weighted in SEO but now virtually ignored in search rankings. In other words, the metaphor implies that it is merely a self-reported hint that search engines do not take at face value.
Furthermore, in explanations from May 2026, Google indicated that allocating substantial engineering hours to implementing llms.txt yields minimal return, at least within the context of Google Search. Google's consistent stance is that special measures to be discovered by AI (AIO/LLMO) are unnecessary, and that executing standard SEO properly is sufficient. The broader perspective on preparing for AI search is covered in our SEO Survival Strategy in the Age of AI Overviews (GH Media), which arrives at the same conclusion: solid execution of foundational practices works, rather than special tricks.
Additionally, observers note that no crawlers have been confirmed to actually ingest llms.txt at this time. Relatedly, in December 2025, when an llms-related file was discovered on the Google Search Central site, rumors spread that Google might be adopting it. However, the file was later deleted, and Google explained that it appeared during an overall site CMS migration and was not placed intentionally by the search team. At the very least, there has been no official commitment to use it.
Does That Mean Zero Effect? Not Necessarily
Dismissing it as entirely pointless would not be intellectually honest either. Reports affirming positive effects from llms.txt have emerged within the industry.
One observation is that changes in how AI describes a company were seen around four to eight weeks after implementation. For example, answers became more accurate or up-to-date when asking ChatGPT or Perplexity what kind of company a business is. Another claim is that, distinct from Google Search, llms.txt may be referenced in the context of conversational AI and generative search engines.
However, these remain observational reports of perceived effectiveness; rigorous verification that isolates whether llms.txt alone was responsible, as opposed to concurrent improvements in content or structured data, remains insufficient. From a custom development perspective, it is reasonable to treat it with the posture that "while positive reports exist, it cannot yet be called an established tactic with proven reproducibility." Viewing it by domain—skeptical in Google Search, possible in conversational AI—helps avoid confusion.
Our Conclusion and Implementation Approach in Custom Development
Taking both sides into account, our conclusion is as follows: Adding llms.txt causes little harm, but overestimating it is a mistake. Treat it as a lightweight measure taking tens of minutes to deploy, and judge its effects through measurement. It is not an initiative warranting heavy engineering hours or high costs.
For an existing site, all it takes is organizing key pages and summaries into Markdown and placing a single file in the root directory. If you already understand your site architecture, the actual work takes anywhere from tens of minutes to half a day at most. Selling this as a "300,000 yen AI optimization package" simply does not match the actual deliverable. We advised the aforementioned manufacturer: "On its own, this can be deployed at no extra charge during an audit. You should direct your budget elsewhere instead."
Once deployed, do not abandon it—measure it. Specifically: (1) periodically query ChatGPT, Perplexity, and others about your company to observe whether description accuracy improves over time; and (2) inspect server logs to verify whether AI crawlers are fetching /llms.txt. Maintain it if effects are observed, but avoid over-investing if nothing changes—knowing when to hold the line is the most important discipline when dealing with trends.
| Dimension | Realistic Positioning of llms.txt |
|---|---|
| Effort | Tens of minutes to half a day; lightweight for existing sites |
| Impact on Google Search | Do not expect impact (officially skeptical) |
| Impact on conversational AI | Possible, but requires measurement and verification |
| Investment decision | Fine to add opportunistically; never make it a flagship initiative |
Allocate Budget to Preparing Information That AI Can Easily Understand
While llms.txt remains merely a self-reported hint, what AI and search engines actually value is far more fundamental: quality content, E-E-A-T (Experience, Expertise, Authoritativeness, and Trustworthiness), clearly articulated service details and value propositions, and third-party mentions (backlinks and citations). AI relies on this substantiated information to understand your company and determine whether to cite it.
Where custom development proves particularly effective is structuring data (schema.org / JSON-LD). Marking up details like company names, business descriptions, and operators accurately in machine-readable format mitigates the risk of AI misidentifying your business. This boasts far more established implementation practices and proven efficacy than llms.txt, which merely asks AI to describe you a certain way. Specific type selections and implementation pitfalls are detailed in our articles Restoring Search Appearance with Structured Data (JSON-LD) (GH Media) and Structured Data and AI Search (GH Media), which covers its impact in AI search.
Another vital aspect is defensive preparation to avoid misrepresentation by AI. When authoritative primary information is sparse on your official website, AI fills the gaps with inaccurate third-party descriptions. Ensuring that company information and business descriptions on your official site are accurate and comprehensive takes higher priority than placing an llms.txt file. This topic is explored in our article The Risk of AI Misrepresenting Your Company and Building Sites That Get Accurately Cited (GH Media).
Pitfalls Clients Often Fall Into
Finally, we share common failure patterns we have observed across client inquiries.
First and most common is clients assuming AI readiness is complete simply because llms.txt was placed. Gaining a false sense of security from the most lightweight, uncertain tactic leads to postponing crucial work on content and structured data. This is the most counterproductive outcome.
Next is paying exorbitant fees to external vendors purely for generating an llms.txt file. If a price tag of hundreds of thousands of yen is attached to a few dozen minutes of work, allocating that spend toward structured data or content enhancements will yield far greater returns for the same budget.
Frequently overlooked is the content of llms.txt contradicting the website body, resulting in adverse effects. Leaving outdated business lines or discontinued services in llms.txt hands AI an erroneous self-profile. Just as with structured data, information communicated to machines is meaningless unless it matches what is actually displayed to users.
How to Proceed
The llms.txt file is neither an enemy to dismiss nor a savior to cling to. We believe the correct stance right now is "deploy it lightly, measure the results, and avoid over-investing." Beyond that, focus limited budgets on the foundations AI and search engines truly evaluate: accurate primary information, structured data, content quality, and external citations. That is the sequence we recommend in good faith as custom developers.
If you feel anxious hearing competitors have added it, do not know what to spend on AI strategies, or want an evaluation of whether an estimate you received is fair, feel free to contact us via GleamHub's inquiry form. We will audit your current site, separate lightweight measures like llms.txt from high-impact initiatives like structured data, and propose an optimal budget allocation.
Sources
- Intro to robots.txt / Crawling and indexing － Google Search Central
- John Mueller on llms.txt being like the keywords meta tag － Search Engine Roundtable
- Should You Use llms.txt? Google’s View and What Actually Matters – baigie
- What Initiatives Really Work in the LLMO/GEO Era – CINC
- Restoring Search Appearance with Structured Data (JSON-LD) (GH Media)
- Structured Data and AI Search (GH Media)
- The Risk of AI Misrepresenting Your Company and Building Sites That Get Accurately Cited (GH Media)
- SEO Survival Strategy in the Age of AI Overviews (GH Media)
