vLLM vs SGLang

The SGLang vs vLLM choice is closer than most engine comparisons. Both are Apache-2.0 projects that serve open models behind an OpenAI-compatible API, publish a numbered release about every two weeks, document new models with a launch page per model and speak the Anthropic Messages API as well. They differ in what surrounds that core: how each caches prefixes, which flags and backends they offer, and which APIs, hardware and Docker images they document. I checked both projects' documentation and repositories on 12 November 2026 and quote no benchmark numbers, because I found no pair of current-release results for the two in either project's documentation.

What SGLang is, next to vLLM

SGLang's README calls it an open-source inference framework for large language, vision-language and diffusion models, optimized for agentic workloads, large-scale serving and RL rollouts, and its documentation says it is hosted under the non-profit organization LMSYS. The README also says its authors learned the design and reused code from several projects, vLLM among them. vLLM's README says it started in the Sky Computing Lab at UC Berkeley, and lists continuous batching, chunked prefill, prefix caching and tensor, pipeline, data, expert and context parallelism. SGLang also documents a frontend language for writing prompts in a structured way.

vLLM vs SGLang side by side

The sources, all checked on 12 November 2026, are vLLM's argument reference, structured outputs and tool calling pages and recipes, and SGLang's server arguments, structured outputs, Ollama API and installation pages and cookbook. Release dates come from each project's GitHub releases page, licenses from the repositories' GitHub pages and hardware from the READMEs.

Releases and model coverage

Both projects ship about every two weeks, and their model pages follow the same pattern. vLLM's recipes repository says it answers "How do I run model X on hardware Y for task Z?", and SGLang's cookbook says it answers "How do I use SGLang (and related models) on hardware Y for task Z?". On 12 November both sites had pages for the same recent releases, for example ERNIE-5.5-Flash and Kimi-K3.1-Pro. Both can also load a model through Hugging Face Transformers when they lack a native implementation, which vLLM's page says should perform the same as a dedicated implementation. For a new model I open both pages and pick the one that lists my GPU.

Multi-GPU flags

On a DeployGPU VM with 2, 4 or 8 GPUs, vLLM takes --tensor-parallel-size 4 or the short -tp 4. SGLang's reference table lists --tensor-parallel-size and --tp-size, and its own examples write --tp 4. Their notes about the interconnect differ. vLLM's parallelism page says to use pipeline parallelism instead of tensor parallelism on GPUs without NVLink, with the L40S as its example, for higher throughput and lower communication overhead. SGLang's server-arguments page says to add --enable-p2p-check if tensor parallelism reports that peer access is not supported between two devices. For data parallelism SGLang's page recommends its Model Gateway, and vLLM has --data-parallel-size. The multi-GPU page explains the interconnect inside one VM.

Caching

vLLM's design page describes hash-based prefix caching: each KV block is hashed from its tokens and the prefix before it, and only full blocks are cached. It adds that prefix caching is widely used by most open-source inference frameworks, SGLang among them. SGLang's radix cache page describes a tree of cached prefixes that evicts leaves under a policy you choose, and it documents HiCache for a hierarchical KV cache. vLLM has --kv-offloading-size for offloading KV cache to the CPU.

One place where the two pages give different advice is Claude Code. SGLang's Anthropic page says to set CLAUDE_CODE_ATTRIBUTION_HEADER=0 so the radix cache can reuse prefixes, while vLLM's page says versions above 0.17.1 handle the changing prefix on the server. The Claude Code guide covers both.

Structured output, tool calls and APIs

Both constrain output with grammar backends, and both need a parser to turn a model's tool calls into API objects. vLLM's structured outputs page names xgrammar and guidance, and the SGLang page names xgrammar, outlines and llguidance. For Anthropic clients, each documents a Claude Code setup, and the OpenCode guide covers the OpenAI-compatible route that any of these servers offers. SGLang documents an Ollama-compatible API, which lets the Ollama CLI and Python library talk to an SGLang server, and I found none in vLLM's documentation.

Docker images and CUDA

The current images of both projects are CUDA 13 only. vLLM's v0.33.0 release notes say its release wheels and images are CUDA 13 only and the -cu129 images are no longer published. SGLang's installation page says its images ship a CUDA 13 environment and lmsysorg/sglang:v0.5.19-cu129 is the last CUDA 12 tag. So run nvidia-smi before you pull either image, and see the vLLM guide and the SGLang guide for the commands. vLLM's Docker page says CUDA 13 needs an R580 or newer driver and also documents a compatibility mode for older drivers.

When the model pages here use each

On the model pages I print a command from whichever project's recipe or cookbook page lists the exact GPU and size, name the minimum version that page asks for, and link the other project's page when it covers the same setup. The DeepSeek page prints both for V4-Flash, because SGLang's cookbook serves it on a single H200 node at tensor parallel 4 and vLLM's recipe lists H200 among its verified hardware. The GLM page prints SGLang's command for GLM-5.3-Flash on 8x H100, because vLLM's static example does not carry over to Hopper, and vLLM's command for GLM-5.3 on 8x H200.

Decision table

When each one wins

I start from the engine the model's card or cookbook names, and move to the other only when its page matches my GPU count or format better. Neither project's README compares itself with the other, and both publish a serving benchmark tool: vLLM has vllm bench serve and SGLang has sglang.benchmark.serving, whose backend list includes vllm. If a number matters, run the same load against each server on your own model. The vLLM tuning guide covers the vLLM settings, the llama.cpp comparison with Ollama covers the lighter engines, and the llama.cpp server guide and Ollama guide cover running them.

On DeployGPU

DeployGPU has no vLLM or SGLang template. Both run in Docker on a plain GPU VM, bound to 127.0.0.1 and reached through an SSH tunnel.