Which LLM Backends Can LMQL Run On?

Updated

LMQL can run on several LLM backends, and the site names three: llama.cpp, OpenAI, and Hugging Face Transformers. According to the project, LMQL "automatically makes your LLM code portable across several backends," and you can switch between them "with a single line of code." That portability claim is the key thing to verify for your own setup: the query language and constraints stay the same, but the model you point at changes.

What the site actually states

The homepage lists the backends under a "Works Across Backends" heading and shows the three logos/names above. It does not publish a full compatibility matrix, per-backend feature differences, or setup instructions on that page. So treat the list as the supported set the project advertises, not as a guarantee that every LMQL feature behaves identically on each one.

Backend What it implies for you
llama.cpp Local inference with GGUF-style models; you manage the model files and runtime yourself
OpenAI Hosted API models; you need an API key and network access
Hugging Face Transformers Local or self-hosted models loaded through the Transformers library

Why "one line" matters

LMQL's pitch is that prompting logic — types, templates, constraints, and the optimizing runtime — lives in your query program, not in backend-specific code. The site's example shows a constrained variable:

"Q: What is the answer to life, the universe and everything?"
"A: [ANSWER]"
where len(ANSWER) < 120 and STOPS_AT(ANSWER, ".")

The constraint is enforced by the runtime, so the same query can be pointed at a different backend without rewriting the prompt logic. That is the practical meaning of portability here: your .lmql query and its where clauses are the stable part; the backend line is the swappable part.

How to decide which backend to use

  • Choose OpenAI if you want hosted models and don't want to manage weights or hardware. You'll need credentials, and usage is subject to OpenAI's terms — the LMQL site does not state pricing.
  • Choose llama.cpp if you want local inference and control over the model file. Expect to handle model download and runtime setup yourself.
  • Choose Transformers if your models are already in the Hugging Face ecosystem or you need to run them on your own infrastructure.

Because the site does not document per-backend feature parity, test your specific query — especially constraint-heavy ones like STOPS_AT or typed variables such as [NUM: int] — on the backend you plan to use before committing.

Getting started

The homepage offers Get Started, Docs, Playground, and GitHub entry points. The Playground is the fastest way to try a query before wiring up a backend locally; the Docs are where backend configuration details would live, since the homepage only names the backends. The project is created by the SRI Lab at ETH Zurich and contributors, and the site is currently running a developer survey for the next major version — worth knowing if you're evaluating LMQL for a longer-term project, since behavior may change.