docs.vllm.aiWebsite

Practical guides

Where to Find vLLM Troubleshooting and FAQ Resources

The vLLM documentation hosts its troubleshooting and FAQ material in the General section of the docs site (docs.vllm.ai), alongside pages on producti…

What Performance Features Does vLLM Offer?

vLLM's performance features fall into four groups: caching (automatic prefix caching, KV cache management), decoding and context handling (speculativ…

Offline Inference vs. Online Serving in vLLM: Which Should You Use?

Choose offline inference when you have a fixed batch of prompts to process and want maximum throughput without maintaining a live service. Choose onl…

How do I get started with vLLM?

Start by choosing the path that matches how you will call the model: offline inference if you are running generations inside a Python script or batch…

What is vLLM and what is it used for?

vLLM is an open-source engine for large language model (LLM) inference and serving. You use it when you need to run LLMs efficiently — either as offl…