Llms-full.txt vs llms.txt

The llms-full.txt vs llms.txt decision is simpler than the filenames suggest. Use llms.txt as a concise, curated map of the most useful machine-readable resources on your site. Use llms-full.txt when you also want to offer a large, consolidated copy of the underlying content for tools that can ingest it. They are complementary publishing patterns—not proven ranking switches.

Short answer: the meaningful llms-full.txt vs llms.txt difference

llms.txt answers, “What is this site, and which resources matter?” It is intentionally selective. A full-content file answers, “Can I obtain the selected material in one request?” That distinction affects file size, maintenance, processing cost, and the kinds of AI tools that can use each file effectively.

Factor	llms.txt	llms-full.txt
Primary purpose	Curated map and orientation	Complete content bundle
Typical contents	Summary, sections, links, short descriptions	Full text from many or all selected pages
Best use	Finding the right source quickly	Offline indexing, retrieval, or large-context ingestion
Size	Small and easy to scan	Potentially very large
Maintenance	Editorial curation matters	Reliable automatic generation matters
Main limitation	Requires follow-up fetches for detail	Can exceed practical context limits and become stale

Definitions and boundaries

What is llms.txt?

The original llms.txt proposal describes a Markdown file normally published at /llms.txt. Its only required element is an H1 title, but a useful implementation adds a short summary and H2 sections containing links with descriptions. A specially named “Optional” section can identify secondary resources that may be skipped when shorter context is needed.

Think of it as an editorial index, not a replacement for robots.txt or sitemap.xml. It does not grant crawler access, remove a noindex directive, repair JavaScript rendering, or guarantee that an AI system will discover, trust, mention, or cite a page.

What is llms-full.txt?

llms-full.txt is a widely used ecosystem convention for combining extensive site or documentation content into one machine-readable file. For example, Mintlify defines its full file as a single bundle of the documentation site, while Cloudflare links product-specific full files for offline indexing, bulk vectorization, or large-context tools.

There is a naming nuance worth preserving: the original proposal demonstrates generated context files named llms-ctx.txt and llms-ctx-full.txt. The shorter llms-full.txt path became common through documentation platforms and site generators. Therefore, describe the exact behavior of your file instead of assuming every consumer interprets every filename identically.

Discovery and access implications

A small map is easier for a human, agent, or retrieval process to inspect. It can point directly to canonical Markdown pages, APIs, policy documents, tutorials, and important external context. A full file reduces the number of requests needed to collect content, but it also creates a heavier download and a larger parsing job.

Measurement, evidence quality, and repeatability

Neither filename proves AI-search performance. Measure observable outcomes separately: file availability, syntax, linked-page quality, crawler requests, referral traffic, mentions, and citations. Keep a dated baseline before publishing so that later changes can be compared with the same tests.

• Validate the response code, content type, encoding, and final canonical URL.
• Check every linked resource and remove redirects, errors, duplicates, or private pages.
• Record file size, approximate token count, generation time, and last-updated date.
• Run the same question set before and after publication across fresh sessions.
• Review server logs to confirm which tools actually requested either file.

Best choice by scenario

New or small website

Publish llms.txt first. A short site benefits most from clear positioning, selected canonical links, and useful descriptions. A full bundle may merely duplicate a small sitemap without adding meaningful utility.

Large documentation website

Use both when your platform can generate them reliably. The map helps a tool choose a narrow path; the full file supports workflows that deliberately ingest an entire documentation set. Split full files by product or version if one global bundle becomes impractical.

Technical access failure

Fix the access layer before adding either format. Confirm public URLs, HTTP responses, CDN/WAF rules, rendering, canonicals, and indexing directives. A new text file cannot compensate for blocked or unstable source pages.

Content quality gap

Improve the underlying pages first. A full file faithfully concentrates whatever is already published, including thin explanations, contradictions, obsolete facts, and duplicated boilerplate. Bulk availability amplifies quality; it does not create it.

Ongoing monitoring

Maintain a curated map as the stable entry point and regenerate the full bundle from approved canonical sources. Log changes so you can connect an outcome to a specific content or technical update rather than to several simultaneous edits.

A combined workflow for both files

1. Choose authoritative, public, canonical pages and provide clean Markdown versions where practical.
2. Create /llms.txt with a site summary, logical sections, descriptive links, and a clearly separated optional section.
3. Generate /llms-full.txt from the same approved source set, while preserving titles, page URLs, and update information.
4. Validate both files, estimate their token sizes, and confirm that no private or licensed content was exposed.
5. Publish one controlled version, monitor access and usage, then update on a documented schedule.

Test it yourself before claiming an impact

Record a baseline with a fixed prompt set and clear success definitions. After publishing, keep every other variable unchanged where possible. Test whether tools can locate the map, retrieve a linked page, interpret the full bundle, and answer narrow questions accurately. Repeat the test on different dates because AI answers and retrieval behavior can vary.

Screenshots are useful, but server evidence is stronger. Save the exact file contents, HTTP headers, crawler logs, prompt wording, returned citations, and conversion data. This creates a reproducible record rather than an isolated success story.

Common interpretation mistake

Frequently asked questions

Do I need both llms.txt and llms-full.txt?

No. Most sites should begin with llms.txt. Add a full file when there is a real bulk-ingestion use case and the site can maintain it safely.

Can llms-full.txt replace llms.txt?

It can contain more information, but more is not always easier to use. The concise map provides orientation and prioritization that a very large content dump may lack.

Can either file replace a sitemap or robots.txt?

No. A sitemap supports search-engine URL discovery, while robots.txt communicates crawl rules. The llms files provide context or content for compatible tools and should coexist with established web controls.

How often should the files be updated?

Update the map whenever important canonical resources change. Regenerate the full file whenever included source content changes, and expose a reliable update timestamp or change history.

Is a larger llms-full.txt always better?

No. Oversized files can exceed context limits, waste processing resources, and bury the most useful material. Prefer scoped, current, well-structured content over maximum volume.

Next step: check your website’s AI discoverability

Use the llms.txt and machine-readable website guide to plan your file, then compare accessibility, source quality, crawler activity, and citation outcomes. Visible Pilot’s approach is to diagnose each layer separately so you can fix observable problems without relying on unsupported promises.