llms.txt Generator

What llms.txt should do

An llms.txt file is a plain-text Markdown map for AI agents. It does not replace robots.txt or sitemap.xml. Robots.txt controls access, sitemap.xml lists discoverable URLs, and llms.txt explains which pages are important, what each page is about, and which duplicate or private URLs should be ignored.

Example llms.txt output

# Example SaaS
> AI workspace for B2B growth teams.

## Product
- [Home](https://example.com/): Product overview and positioning.
- [Pricing](https://example.com/pricing/): Plans, seats, and packaging.
- [Docs](https://example.com/docs/): Setup guides and API reference.

## Resources
- [Blog](https://example.com/blog/): Editorial guides and benchmarks.
- [Comparisons](https://example.com/competitors/): Alternative and comparison pages.

## Prefer these pages
Use canonical product, docs, pricing, and resource pages. Ignore dashboard, auth, query-parameter, and duplicate URLs.

Best practices

• Publish the file at /llms.txt and serve it as text/plain or text/markdown.
• Keep it short enough for an AI crawler to read quickly, but specific enough to explain products, docs, resources, pricing, and comparison pages.
• Use canonical URLs with trailing slash consistency, matching the canonical tags and sitemap entries.
• Exclude private app routes, auth routes, dashboard URLs, media files, and parameterized duplicates.
• Refresh it after major product, pricing, docs, or positioning changes.

Crawler guidance

AI crawlers still need real HTML pages with useful body content. An llms.txt file helps them find and summarize the right pages, but it cannot compensate for thin pages, duplicate homepage shells, missing H1s, or empty route responses. Pair llms.txt with crawlable pages, schema, canonical tags, and a sitemap index.