What Is the Difference Between llms.txt and llms-full.txt on llmstxt.new?

Updated

On llmstxt.new, both options produce a consolidated text file from a website, but they differ in scope: llms.txt is the standard generation, while llms-full.txt is the fuller generation option and requires a Firecrawl API key. Choose llms.txt when a standard consolidated file is enough; choose llms-full.txt when you need the fuller output and have a Firecrawl API key.

What the tool does

llmstxt.new (titled "LLMs.txt generator v2") generates consolidated text files from websites for LLM training and inference, and is powered by Firecrawl. The page presents two generation choices side by side: llms.txt and llms-full.txt.

The two options compared

Dimension llms.txt llms-full.txt
What it is Standard consolidated text file generated from a website Fuller generation option for the same kind of consolidated file
API key requirement Not stated as required on the page Firecrawl API key required for full generation
Best for Getting a consolidated site file without extra setup Cases where you need the fuller output and can supply a Firecrawl API key

The page does not publish a line-by-line description of what extra content llms-full.txt includes, so treat "fuller" as the documented distinction rather than assuming a specific format difference.

How to generate either file

  1. Open llmstxt.new.
  2. Enter the URL of the website you want to process. The page states a URL is required to generate an llms.txt file.
  3. Choose your output: select llms.txt for standard generation, or llms-full.txt for full generation.
  4. If you chose llms-full.txt, enter your Firecrawl API key. The page notes that an API key from Firecrawl gives a better experience.
  5. Run the generation and save the resulting consolidated text file.

Expected result: a single text file consolidating content from the site you entered, ready to feed to an LLM for training or inference.

Common sticking points

  • Missing URL: generation will not start without a website URL.
  • No API key for full generation: llms-full.txt specifically calls for a Firecrawl API key; the page frames the key as required for full generation.
  • API deprecation: the page carries a notice that this API is being deprecated in favor of the main endpoints. The endpoint remains active but will no longer be maintained after June 30, 2025. An example repository for generating LLMs.txt files is linked from the page (github.com/mendableai/create-llmstxt-py), and the page also notes the generator can be called via API.

Which should you pick?

  • Pick llms.txt if a standard consolidated file meets your needs and you want to avoid supplying an API key.
  • Pick llms-full.txt if you need the fuller generation and have a Firecrawl API key available.
  • If you are building this into a pipeline rather than using the page interactively, review the deprecation notice and the linked example repository before relying on the endpoint long term.