简体中文
EnglishWhat does LMSYS Org's latest blog cover?
Updated
LMSYS Org's latest blog posts focus on three engineering topics: scaling decision models with SGLang, accelerating long-context and agentic inference with an NVFP4 KV cache, and adding day-0 support for DeepSeek-V4.1 in SGLang and Miles. The blog is where the organization documents serving-engine work, inference optimizations, and support for new model architectures.
Recent posts at a glance
| Post | Date | Core topic |
|---|---|---|
| Scaling JEV-like Decision Models with SGLang | September 25, 2026 | Serving decision-making agents with SGLang |
| Accelerating Long-Context and Agentic Inference with NVFP4 KV Cache | September 16, 2026 | KV cache quantization for long-context and agent workloads |
| SGLang and Miles Add Day-0 Support for DeepSeek-V4.1 | September 10, 2026 | Serving-stack support for a new model architecture |
What each post covers
Scaling JEV-like Decision Models with SGLang
This post frames a decision-model scenario: a customer asks whether an order has shipped, and an agent already holds the order ID plus three possible next actions — query the order-status service, search general delivery-policy documentation, or ask the user. The post is about scaling that kind of decision model with SGLang as the serving engine.
Accelerating Long-Context and Agentic Inference with NVFP4 KV Cache
The KV cache stores keys and values from multiple conversation rounds in GPU memory so the model can reuse context. This post addresses the cost of that cache as context and agent sessions grow, and covers using an NVFP4 KV cache to accelerate long-context and agentic inference.
SGLang and Miles Add Day-0 Support for DeepSeek-V4.1
DeepSeek-V4.1 introduces architecture choices that affect the serving stack, including low-ratio compression and sliding-window attention (SWA), where each layer maintains an fp8 sliding-window cache. The post covers day-0 support for the model in both SGLang and Miles.
What the blog tells you about LMSYS Org's direction
The three posts share a common thread: making large models practical to serve. LMSYS Org describes itself as developing large models and systems that are open, accessible, and scalable, and the blog reflects that through:
- Serving-engine work — SGLang appears in two of the three posts, consistent with its role as a high-performance serving engine for large language and multimodal models.
- Inference efficiency — KV cache quantization and long-context handling target the memory and latency costs of running models at scale.
- Fast support for new architectures — day-0 support for DeepSeek-V4.1 shows the project tracks new model releases and adapts the serving stack quickly.
How to read more
The blog section on lmsys.org lists the latest posts with a "See all posts" link, which is the entry point for the full archive. If you want the code behind the posts, the related projects are listed on the same site — SGLang (36.8k stars), FastChat (39.6k stars), RouteLLM (5.6k stars), Chatbot Arena, and Vicuna — and the organization reports 15+ projects, 87K+ GitHub stars, and 1000+ contributors.