Is the llmstxt.new API Being Deprecated? What to Use Instead

Updated

Yes. The llmstxt.new generator page states that its API is being deprecated in favor of Firecrawl's main endpoints. The endpoint will stay active, but it will no longer be maintained after June 30, 2025. For new work, plan to switch to the recommended alternative: the example repository mendableai/create-llmstxt-py, which generates llms.txt files.

What the page actually says

The notice on llmstxt.new (the "LLMs.txt generator v2" page, powered by Firecrawl) contains three points:

  • The API is being deprecated in favor of the main endpoints.
  • An example repo is offered for generating llms.txt files: https://github.com/mendableai/create-llmstxt-py.
  • The API endpoint will remain active, but it will no longer be maintained after June 30, 2025.

So the tool is not switched off today. It is in a "still works, no longer maintained" state, with a stated maintenance cutoff.

What this means in practice

Situation What to do
You already call the llmstxt.new API It should keep responding, but treat it as frozen: no fixes or updates after June 30, 2025.
You are starting a new integration Use the recommended alternative (create-llmstxt-py) or Firecrawl's main endpoints instead of building on the deprecated endpoint.
You only use the web page occasionally The generator page still describes itself as working; for anything you depend on, prefer the maintained path.

The page does not state pricing or login requirements for the API, so don't assume it is free or keyless. The page does note that an API key from Firecrawl gives a better experience, and the full-generation option asks for a Firecrawl API key.

How to move off the deprecated endpoint

  1. Inventory your calls. Find every place your code or scripts send requests to the llmstxt.new API.
  2. Pick the replacement. For file generation, start from the create-llmstxt-py example repo; for broader crawling and extraction, use Firecrawl's main endpoints.
  3. Port one call and compare output. Run the same URL through the old endpoint and the new path, then diff the generated llms.txt to confirm the structure and content still match what your downstream system expects.
  4. Check your key handling. If the new path requires a Firecrawl API key, confirm where the key is stored and that it is not hard-coded.
  5. Set a deadline before June 30, 2025. After that date the old endpoint receives no maintenance, so any bug you hit stays unfixed.

Verification: after switching, regenerate a known site and confirm the file still lists the pages and descriptions you rely on. Common snag: the old endpoint continuing to respond can hide the fact that you never migrated — track the migration explicitly rather than waiting for a failure.

Related questions about the tool

  • What is llmstxt.new? A generator that produces consolidated text files from websites for LLM training and inference, powered by Firecrawl.
  • How do I generate a file? Enter a URL on the generator page; supplying a Firecrawl API key enables full generation.
  • llms.txt vs llms-full.txt? The page offers both outputs but does not spell out the difference in the captured content, so check the generated files themselves before relying on either format.