Knowledge Base

Answers and Guides

Where to Find vLLM Troubleshooting and FAQ Resources

The vLLM documentation hosts its troubleshooting and FAQ material in the General section of the docs site (docs.vllm.ai), alongsi…

What Performance Features Does vLLM Offer?

vLLM's performance features fall into four groups: caching (automatic prefix caching, KV cache management), decoding and context …

Offline Inference vs. Online Serving in vLLM: Which Should You Use?

Choose offline inference when you have a fixed batch of prompts to process and want maximum throughput without maintaining a live…

How do I get started with vLLM?

Start by choosing the path that matches how you will call the model: offline inference if you are running generations inside a Py…

What is vLLM and what is it used for?

vLLM is an open-source engine for large language model (LLM) inference and serving. You use it when you need to run LLMs efficien…