How to Add an llms.txt File to Your Website (Step-by-Step)

TL;DR: How to Add an llms.txt File

To add an llms.txt file to your website, create a text file named llms.txt in your site's root directory (e.g., https://example.com/llms.txt). This file provides a machine-readable list of important pages for AI models, helping them better understand and cite your content. Follow the structure below:

# Your Site Name
> A brief description of your site.

## Core pages
- https://example.com/
- https://example.com/about
- https://example.com/blog

## Optional sections
- https://example.com/docs

That's it! Now let's dive deeper into why and how to optimize it.

---

Why You Need an llms.txt File

Large Language Models (LLMs) like ChatGPT, Claude, and Perplexity often crawl websites to answer user questions. Without guidance, they may pick outdated, irrelevant, or low-quality pages. An llms.txt file acts like a "robots.txt for AI," telling models which pages to prioritize. This improves the accuracy of AI-generated answers and increases your chances of being cited in responses.

By learning how to add an llms.txt file correctly, you take control of how AI models perceive your site. It's especially useful for documentation sites, blogs, and SaaS landing pages.

Step-by-Step: How to Add an llms.txt File

Step 1: Create the File

Create a plain text file named llms.txt in the root directory of your website. For example, if your site is https://yoursite.com, the file should be accessible at https://yoursite.com/llms.txt.

Step 2: Follow the Standard Format

The llms.txt standard recommends a simple markdown-like structure. Here's a template:

# Your Site Name
> A brief, one-line description of your site.

## Core pages
- https://example.com/
- https://example.com/about
- https://example.com/blog

## Optional: Documentation
- https://example.com/docs/getting-started
- https://example.com/docs/guides

## Optional: Support
- https://example.com/contact

Step 3: Add Your Best Content

Only include URLs that you want AI models to cite. Typically:

Avoid adding login pages, error pages, or thin content.

Step 4: Host the File

Upload the file to your server's root directory. For example:

Step 5: Verify the File is Accessible

Visit https://yoursite.com/llms.txt in your browser. You should see the plain text content. If not, check file permissions (must be readable by everyone) and server configuration.

Real Example: llms.txt for a Blog

Here's a real example for a tech blog:

# TechBlog
> Latest tutorials on web development and AI.

## Core pages
- https://techblog.com/
- https://techblog.com/about
- https://techblog.com/blog

## Best articles
- https://techblog.com/blog/how-to-add-llmstxt
- https://techblog.com/blog/ai-security
- https://techblog.com/blog/nextjs-tutorial

## Categories
- https://techblog.com/category/javascript
- https://techblog.com/category/security

Common Mistakes to Avoid

How AI Models Use llms.txt

When an AI model like Claude or ChatGPT searches for information about a topic, it may request llms.txt from your domain. It then prioritizes the listed URLs when generating answers. This means your best content gets higher visibility.

For example, if you ask ChatGPT "How to add an llms.txt file?", and your site has a well-structured llms.txt, your article is more likely to be cited.

Advanced: Dynamic llms.txt with JavaScript

If your site uses a framework like Next.js or Gatsby, you can generate llms.txt dynamically. Here's a Next.js example using API routes:

// pages/api/llms.txt.js (or app/api/llms.txt/route.js in App Router)
export async function GET() {
  const content = `# My Site
> A dynamic site.

## Core
- https://mysite.com/
- https://mysite.com/blog
`;
  return new Response(content, {
    headers: { 'Content-Type': 'text/plain' },
  });
}

Then configure a rewrite to serve it at /llms.txt.

Security Note: Don't Expose Secrets

Since llms.txt is public, do not include private URLs (e.g., admin panels, staging sites). Also, be aware that some AI models may use it to discover endpoints for prompt injection. To mitigate, use OverMCP to scan your vibe-coded apps for exposed secrets and misconfigurations before deploying.

FAQ

What is an llms.txt file?

An llms.txt file is a plain text file placed in your website's root directory that lists important URLs for AI language models. It helps models find and cite your best content, similar to how robots.txt guides search engines.

How do I add an llms.txt file to my site?

Create a text file named llms.txt with a list of absolute URLs grouped under markdown headings, then upload it to your site's root folder (e.g., public/llms.txt). Ensure it's accessible at https://yoursite.com/llms.txt.

Does llms.txt affect SEO?

Not directly for search engines, but it can improve your site's visibility in AI-generated answers, which may drive indirect traffic and citations. It's a form of Generative Engine Optimization (GEO).