简体中文
EnglishWebsite review
This review may include AI-generated content and is not individually verified. Domain, certificate and other facts come from the observed data on this page.
What is LMQL?
LMQL (Language Model Query Language) is a programming language for interacting with large language models, created by the SRI Lab at ETH Zurich and contributors. Instead of assembling prompts as loose strings, you write query programs that combine prompt text with typed variables, constraints, loops and Python control flow. The language is designed to make LLM prompting more robust and modular, with an optimizing runtime that enforces your constraints during generation.
What it looks like in practice
An LMQL query is written as a Python function decorated with lmql.query. Top-level strings act as prompts, bracketed variables like [ANSWER] mark where the model should generate, and a where clause adds hard constraints. For example, len(ANSWER) < 120 and STOPS_AT(ANSWER, ".") limits length and stops at a period, while a typed variable like [NUM: int] guarantees the output parses as an integer. Results are then available directly in Python, so the same function can return the generated value to surrounding code.
Who it suits
- Developers building structured LLM pipelines who want output formats enforced by the runtime rather than checked after the fact.
- Teams switching between models — the site says LMQL code is portable across several backends, including llama.cpp, OpenAI and Hugging Face Transformers, switchable with a single line.
- People writing multi-step prompts who need loops, string interpolation, nested queries and reusable prompt components instead of one long template.
Trade-offs to weigh
LMQL adds a learning layer: you are adopting a new language and runtime on top of Python, which is worthwhile when constraint enforcement and modular prompts matter, but likely overkill for a single simple prompt. It also ties your workflow to a specific toolchain, so check that your preferred model backend is supported before committing.
A useful next step is to open the LMQL playground with a small query of your own — for instance, a typed extraction task — and confirm the constraint behaves as expected on the backend you actually plan to use.
How does LMQL constrain LLM output using types, templates, and constraints?
LMQL constrains LLM output by treating the prompt as a program: output is captured in named variables, and the where clause attaches hard constraints that the runtime enforces during generation rather than checking after the fact.
Types
A variable annotation such as [NUM: int] declares the expected output format, so the model's generation is restricted to values that parse as that type. This is how LMQL guarantees structured output—the returned value is usable directly in code instead of needing a parse-and-retry step.
Constraints
The where clause holds conditions the generated text must satisfy, combining length limits, stop conditions and membership checks. The page's examples show len(ANSWER) < 120 and STOPS_AT(ANSWER, ".") and THING in ["Volleyball", "Sunscreen", ...]. Because these are enforced by the optimizing runtime, invalid tokens are masked during decoding.
Templates and modularity
Top-level strings act as prompt templates, with Python string interpolation and control flow (loops, functions) building multi-part prompts. Nested queries extend this by letting you reuse local instructions as components—the page describes this as bringing procedural programming to prompting.
| Mechanism | What it controls | Practical effect |
|---|---|---|
| Types | Output format | Values arrive already parsed (e.g., an int) |
Constraints (where) |
Valid content and length | Invalid generations are prevented, not filtered |
| Templates / nested queries | Prompt structure | Reusable, modular prompt components |
A useful decision criterion: if your pipeline currently validates model output and retries on failure, typed variables and where constraints move that validation into generation. For a concrete case, a date-extraction script can require DD/MM/YYYY formatting via a constraint instead of regex-repairing free text afterward. Portability across backends (llama.cpp, OpenAI, Transformers per the page) means the same constrained query can move between models with a one-line change, though tighter constraints may slow generation on smaller local models.
Next step: start from the docs and playground linked on LMQL and rewrite one fragile prompt as a query with a typed variable plus a single where constraint.
How do I switch between LLM backends like OpenAI, llama.cpp, and Transformers in LMQL?
LMQL is designed so that switching between supported backends, such as OpenAI, llama.cpp, and Hugging Face Transformers, is largely a matter of changing how the query is run rather than rewriting the prompt logic. The prompt, typed variables, and constraints stay in the LMQL query; the backend is chosen when you execute it, and the page describes this as portability "across several backends" with a single line of code change.
How the switch works in practice
LMQL separates the query program from the model that serves it. In the Python integration shown on the site, a query is a decorated function (@lmql.query) whose body is the LMQL string, and it is called like a normal function. The backend is a runtime/decoder setting passed at call time or in the surrounding configuration, not something embedded in the prompt text.
A practical pattern looks like this:
- Write the query once, with constraints such as
len(ANSWER) < 120,STOPS_AT(ANSWER, "."), or typed output like[NUM: int]. - Keep backend-specific choices (model name, decoding behaviour, endpoint or local model path) in a small configuration layer, for example an environment variable or a config dict.
- Call the same query function with the chosen backend, then compare outputs.
What changes and what doesn't
| Concern | Prompt/query logic | Backend selection |
|---|---|---|
| Prompt text, templates, loops | Stays the same | — |
| Types and regex constraints | Stay the same | Enforced by the runtime |
| Model name and local paths | — | Changes per backend |
| Hosted API vs local execution | — | OpenAI is hosted; llama.cpp and Transformers run locally |
| Output quality and speed | — | Varies by model and hardware |
The site's example shows a loop building a packing list with a constrained variable, and a separate example showing typed output ([NUM: int] returning 42). Those constructs are written once; the backend determines which model actually generates the tokens.
Choosing a backend
- OpenAI: use when you want a hosted model without managing hardware. Trade-off: network dependency and per-call cost, and constraints must be honoured by the runtime rather than by direct access to the model's token stream.
- llama.cpp: use for local, lightweight inference, especially on modest hardware or quantized models. Trade-off: you manage model files and local setup.
- Transformers: use when you want direct access to Hugging Face models and fine control over generation. Trade-off: heavier memory and setup than llama.cpp for comparable models.
For a concrete case: a developer prototyping a date-extraction prompt can test the same query against a local llama.cpp model for fast iteration, then run it against OpenAI for a quality check, changing only the backend configuration between runs. Because the constraints and typed variables live in the query, the comparison tests the models rather than two different prompt implementations.
Next step: start from the LMQL documentation and playground at LMQL to confirm the exact backend argument names for your installed version, then wrap backend choice in one config value so every experiment uses an identical query.
How can I integrate LMQL query programs into an existing Python application?
LMQL is designed to be called from Python directly: a query program is defined as a Python function decorated with @lmql.query, and calling that function runs the prompt and returns the result to your application code. The page's own example shows a decorated function whose trailing return NUM hands a typed value back to the caller, so you can treat an LMQL query as an ordinary function inside your existing code rather than a separate service.
Practical integration pattern
- Wrap one task per query. Keep each decorated function focused on a single job (extracting a field, classifying text, formatting a date) and call it from your application where you would otherwise call an LLM API.
- Use typed output variables for downstream code. Declaring a variable as
intor constraining it with a regex means your Python side receives a value in the expected format instead of free text you must parse and validate yourself. - Enforce limits in the
whereclause. Constraints such as a maximum length or a stop condition are checked by the runtime, which reduces post-hoc cleanup and retry loops in your application. - Compose with nested queries. The page describes nested queries as a way to modularize local instructions and reuse prompt components, which suits applications with several related prompts sharing common instructions.
- Switch backends with a single line. LMQL states that code is portable across backends including llama.cpp, OpenAI and Hugging Face Transformers, so you can change the model behind a query without rewriting the prompt logic.
Example: a field extractor
A function that answers a question and returns a number can be called wherever your app needs a numeric result, with the length and stop constraints keeping the output bounded. Your application code then handles only the returned value, not the raw completion.
Decision criteria
Choose this approach when you want prompts to live in version-controlled Python files, need structured or constrained outputs, or expect to change models later. If your application already routes LLM calls through a framework with its own prompt abstractions, compare how each handles typed outputs and backend switching before adding a second layer.
For setup details and the full syntax, start with the documentation linked from LMQL.
What are nested queries in LMQL and how do they enable modular prompt reuse?
Nested queries let you define a prompt-carrying query inside another LMQL query and call it like a function, so a reusable instruction block can be written once and invoked from several places. LMQL's own description presents them as "procedural programming for prompting," enabling modularized local instructions and reuse of prompt components. In practice, a nested query bundles a prompt fragment, its variables and its constraints into a named unit; the outer query calls it, receives the generated value, and continues with its own control flow. That gives you local scope for intermediate variables, so an inner step's output doesn't leak into the surrounding prompt unless you pass it back.
What it changes for prompt design
- Reuse without copy-paste: a date-format instruction, extraction step or classification block lives in one definition and is called wherever needed. Editing that definition updates every caller.
- Local instructions: an inner query can carry its own format rules and constraints that apply only inside that call, keeping the outer prompt focused on the overall task.
- Composability with Python: because LMQL queries are Python functions, nested results can be looped over, compared or fed into a later step — the pattern behind multi-part prompts and meta prompting.
The site's own illustration is a set of birth-date questions each answered under a shared "respond in DD/MM/YYYY" instruction, followed by an outer question asking which person was born last. Each inner call is a self-contained unit; the outer query reasons over the collected answers. That is the concrete benefit: the format rule is stated once, and the outer logic stays readable.
When it's worth the structure
Use nested queries when the same instruction pattern recurs across a workflow, when a step has its own constraints that shouldn't clutter the top-level prompt, or when you want to swap a component without rewriting the surrounding logic. For a single one-off prompt, a flat query is simpler and easier to debug. The trade-off is indirection: reasoning about behavior means tracing across query boundaries, so keep nesting shallow and name inner queries by what they return.
As a next step, take one instruction you currently paste into several prompts and extract it into a nested query with a typed output variable, then call it from two different outer queries to confirm the reuse works. For the language itself, see LMQL; if you are weighing alternative prompting frameworks, Prompt flow and LangChain cover adjacent ground with different abstractions.
How does LMQL's optimizing runtime improve prompt execution compared to plain prompting?
LMQL's optimizing runtime changes when and how constraints are applied during generation, rather than checking a finished response afterward. In plain prompting you write instructions in natural language and hope the model complies; with LMQL you declare constraints (types, regex patterns, stop conditions, allowed value sets) in a where clause, and the runtime enforces them while tokens are being produced. The page describes this as "robust and modular LLM prompting using types, templates, constraints and an optimizing runtime," and its example constrains an answer with len(ANSWER) < 120 and STOPS_AT(ANSWER, ".") and a typed [NUM: int] variable.
What the runtime actually buys you
- Fewer retries and repairs. Because invalid continuations are pruned during decoding instead of rejected after the fact, you avoid the generate-check-regenerate loop that plain prompting often needs for strict formats like dates, IDs or enumerations.
- Guaranteed output shape. Typed variables and regex mean the result is directly usable by surrounding code — the page's example returns
NUMto Python as42— so downstream parsing is simpler. - Portability across backends. The page states LMQL makes LLM code portable across several backends (llama.cpp, OpenAI, Transformers) switchable with a single line. Constraints live in your query, not in a vendor-specific API call, so behavior stays consistent when you change models.
- Modularity. Nested queries let you reuse prompt components and local instructions, so shared formatting rules are written once and applied wherever needed instead of being pasted into every prompt.
Where plain prompting still wins
For open-ended work — brainstorming, summarising, conversational replies — hard constraints add little and can distort output. The runtime also only helps where a constraint is expressible; "be accurate" or "sound friendly" cannot be enforced token by token. And constrained decoding can slow generation or bias the model toward awkward phrasing when the constraint is tight, so reserve it for fields where the format genuinely matters.
A concrete scenario
Suppose you extract birth dates from biographies and need DD/MM/YYYY. Plain prompting gives you a mix of formats and a validation step. In LMQL you would declare a date-shaped variable with a regex and a stop condition, letting the runtime steer decoding toward valid dates from the first token.
Next step: take one prompt where you currently validate and retry, rewrite only the output field as a constrained variable, and compare failure rates and latency against your existing version before converting the rest.
Namecheap
Fastly
User reviews (0)