Where to Find LLMLingua Demos, Code, and Papers

Updated

The LLMLingua Series site (llmlingua.com) is the central hub for the Microsoft Research prompt-compression family: LLMLingua, LongLLMLingua, LLMLingua-2, and SeCom. From this page you can reach two live demos (LLMLingua and LLMLingua-2), the code repository, and the papers for the first three methods. Use the table below to jump straight to what you need.

Quick reference: what is linked where

Resource What you get Where it lives on the page
LLMLingua Demo Interactive demo of the perplexity-based token-removal compressor "LLMLingua Demo" link near the top
LLMLingua-2 Demo Interactive demo of the distilled, task-agnostic compressor "LLMLingua-2 Demo" link near the top
Code Repository with implementations and the ./examples directory "Code" link near the top
LLMLingua Paper Original method: identify and remove non-essential tokens using perplexity from a small language model (SLM) "LLMLingua Paper" link
LongLLMLingua Paper Query-aware compression and reorganization for long contexts "LongLLMLingua Paper" link
LLMLingua-2 Paper Data distillation to learn compression targets for efficient, faithful, task-agnostic compression "LLMLingua-2 Paper" link

Note that the page lists papers for LLMLingua, LongLLMLingua, and LLMLingua-2, but no separate SeCom paper link appears in the page evidence — SeCom is described only in the series overview.

Which method should you open first?

The three methods target different problems, so pick based on your task:

  • LLMLingua — general prompt compression by dropping low-information tokens scored with a small model's perplexity. Start here if you want the baseline approach and a simple demo.
  • LongLLMLingua — for long-context inputs where the question matters: it compresses with awareness of the query and reorganizes the retained content. Choose this when your prompt is long but only partly relevant to the question.
  • LLMLingua-2 — task-agnostic compression trained via data distillation. The site reports a 3x–6x speed improvement over LLMLingua, so prefer it when compression latency matters.

Practical entry points beyond the papers

  • Examples directory: the site points to a ./examples directory in the code repository covering LLMLingua-2, RAG, Online Meeting, CoT, Code, and RAG using LlamaIndex. These are the fastest way to see a working integration rather than reading the paper first.
  • Framework integrations: LLMLingua has been integrated into LangChain and LlamaIndex, so if you already use either RAG framework you can adopt compression without wiring the library yourself.
  • Related work: the same site announces SCBench (KV-cache-centric evaluation of long-context methods), RetrievalAttention (KV cache offloading via vector retrieval), and MInference (sparse attention for long-context pre-filling, reported up to 10X lower pre-fill latency on an A100 at 1M tokens). These are separate projects, not part of the compression pipeline, but useful if your bottleneck is long-context inference rather than prompt length.

Verifying you have the right resource

Each method has a "Read More" entry on the series page that restates its one-line mechanism (perplexity-based token removal, query-aware long-context compression, distillation-based task-agnostic compression). If the description under a link does not match the behavior you need, you are on the wrong method page. The BibTeX section at the bottom of the site is the place to grab citation entries for any paper you use.