Llms.txt Validation Checklist

An llms.txt validation checklist helps you confirm that a proposed machine-readable site guide is accessible, correctly structured, useful, and maintainable. Use it before publishing a new file, after a website migration, or whenever linked resources change. A valid file can make important content easier for compatible tools to interpret, but it cannot force an AI platform to crawl, rank, recommend, or cite your website.

Before you start: gather a clean baseline

Open the proposed file at https://example.com/llms.txt in a private browser window. Save the current response headers, final URL, file contents, and test date. Also collect the canonical URLs of the pages you plan to list. This baseline lets you separate an old problem from a change introduced during validation.

• Access tools: a browser, an HTTP header checker, and server or CDN logs.
• Content tools: a plain-text editor with UTF-8 support and a Markdown previewer.
• Link evidence: status code, redirect destination, canonical URL, and page title for every listed resource.
• Testing record: the exact prompt, tool or model, date, answer, and cited URLs used before and after a change.

Check 1: confirm discovery and crawler access

Request the exact root URL and verify that it returns a stable HTTP 200 response without authentication, a cookie wall, geographic restriction, rate-limit page, or JavaScript challenge. The proposed format normally lives at /llms.txt, although the proposal also permits a subpath. Root placement remains the clearest default because it is predictable.

curl -I -L https://example.com/llms.txt
curl -L https://example.com/llms.txt

Review the full redirect chain. A single permanent redirect to a canonical HTTPS URL may be acceptable, but a loop, temporary chain, cross-domain jump, 403 response, or 5xx error is a failure. Check your firewall logs as well: a browser success does not prove that automated requests receive the same file.

Check 2: validate technical delivery and Markdown structure

The llms.txt proposal uses Markdown in a specific order. Its only required section is one H1 containing the site or project name. A short blockquote summary should follow, then optional explanatory text and H2 sections containing lists of resources. Each resource entry should include a Markdown link; a short description after the link is optional but useful.

• Filename and location: use llms.txt at a stable, public URL, preferably the domain root.
• Encoding: serve clean UTF-8 text without smart-quote substitutions or hidden editor markup.
• Opening heading: include one clear H1 that names the website, product, or documentation set.
• Summary: explain what the site provides and what a reader needs to understand it.
• Sections: group related resources under descriptive H2 headings.
• Links: use complete, valid URLs and concise labels; add a note when the destination is not obvious.
• Optional resources: place secondary material under an ## Optional section so shorter contexts can omit it.

Do not confuse “the Markdown renders” with “the file follows the proposal.” A file containing random headings or a pasted sitemap may look readable while providing poor hierarchy. Validate the order, the link-list pattern, and the purpose of every section.

Check 3: assess content clarity and citation readiness

A technically correct file can still be unhelpful. Every listed page should answer a clear question, identify the organization or product consistently, and contain enough evidence to stand on its own. Remove duplicate URLs, obsolete campaigns, tag archives, thin pages, and links that only make sense after a user session.

• Use precise link labels such as “Pricing and plan limits,” not “Learn more.”
• Prefer primary documentation, original research, policies, and detailed guides over promotional summaries.
• Confirm the page title, H1, canonical URL, and visible subject agree.
• Add dates, authorship, methodology, and limitations where freshness or evidence matters.
• Keep descriptions factual; do not promise that a platform will cite the page.

Check 4: run a repeatable platform test

First test the file independently of any AI answer. Parse it, open every link, and record pass, warning, or fail. Then use a small fixed set of factual questions whose answers exist on the linked pages. Repeat the same questions in fresh sessions and record whether the relevant page is found, linked, cited, or ignored.

1. Ask one brand-definition question.
2. Ask one product, service, or documentation question.
3. Ask one question that requires evidence from a listed guide.
4. Repeat the baseline after publishing, without changing the wording.
5. Compare results over time; do not report a single favorable answer as causation.

Prioritize findings by impact

Priority	Typical finding	Required action
Critical	404/403/5xx, private file, HTML challenge, broken core links	Fix before judging content quality
Important	Missing H1, unclear summary, weak grouping, stale or redirected links	Correct in the next publishing cycle
Improvement	Descriptions are vague, optional resources are mixed with essential pages	Refine after access and syntax pass

Fix critical delivery problems first. There is little value polishing descriptions while the file returns an error or important links are blocked. After access and syntax pass, improve information architecture and descriptions. Keep a change log so future teams can see what was added, removed, and retested.

Evidence and screenshots to save

Keep the final file, HTTP headers, redirect trace, parser output, and a link-check report. Capture representative screenshots of the public file and any error state you corrected. For ongoing measurement, store test prompts, platform responses, cited sources, crawler-log entries, referral traffic, and conversions. This evidence prevents assumptions from turning into unsupported SEO claims.

Common interpretation mistake

The biggest mistake is presenting llms.txt as a guaranteed ranking or citation mechanism. The project describes it as a proposal intended to help language models use website information at inference time, and its own guidance recommends testing with multiple models. A valid file proves that you implemented the convention coherently. It does not prove that every AI system discovers the file, processes it, or gives the listed pages more visibility.

Frequently asked questions

What is required in a valid llms.txt file?

Under the published proposal, the only required section is an H1 naming the project or site. A useful implementation normally adds a blockquote summary and H2 resource sections with Markdown links.

Should llms.txt replace robots.txt or sitemap.xml?

No. Robots.txt communicates crawler access preferences, while a sitemap helps search engines discover important URLs. Llms.txt is a curated Markdown guide with a different purpose. The three files can coexist.

Does the file need to be at the domain root?

Root placement at /llms.txt is the clearest default described by the proposal. Subpaths are allowed, but discovery expectations for them may be less obvious, so document and test the location carefully.

How often should I validate llms.txt?

Validate it whenever URLs, navigation, documentation, products, redirects, or access rules change. A monthly automated link check plus a quarterly editorial review is a sensible starting point for an active site.

Can a valid llms.txt guarantee AI citations?

No. Citation depends on platform behavior, query fit, available sources, freshness, authority, and other factors outside the file. Measure citations as an outcome, not as a validation rule.

Next step

Use the llms.txt and machine-readable website guide for broader implementation context. Then review common llms.txt mistakes and get the AI Search Readiness checklist to document access, content quality, and repeatable platform tests.

Sources

See the official llms.txt proposal for the current format and implementation guidance. For related standards, consult Google Search Central’s explanations of robots.txt and sitemaps.