vLLM vs llama.cpp

The vLLM vs llama.cpp choice starts with the file format. vLLM serves Hugging Face checkpoints, safetensors first, with its own quantization formats such as FP8 and AWQ, and treats GGUF as an experimental extra. llama.cpp serves GGUF files, with 1.5-bit to 8-bit integer quantization, and can run on the CPU what does not fit in GPU memory. The projects' own descriptions show the split: vLLM calls itself a high-throughput and memory-efficient inference and serving engine, and llama.cpp states its main goal as inference with minimal setup and performance on a wide range of hardware. I checked both projects' documentation on 12 November 2026 and quote no speed numbers, because I found no pair of current-release results for the two in either project's documentation.

vLLM vs llama.cpp in one table

The sources are vLLM's README, argument reference, optimization page, Docker page and releases, and the llama.cpp README, server README and Docker page, all checked on 12 November 2026.

Formats, and what vLLM does with GGUF

vLLM's README lists GGUF with the quantization formats it supports, and the GGUF page is more careful. It calls the support highly experimental and under-optimized, says it can be incompatible with other features and that for now you can use GGUF as a way to reduce memory footprint, and says it moved to an out-of-tree plugin you install with uv pip install vllm-gguf-plugin. The command takes the repository and the quant, vllm serve unsloth/Qwen3-0.6B-GGUF:Q4_K_M --tokenizer Qwen/Qwen3-0.6B, and the page recommends the base model's tokenizer because converting one from GGUF is slow and unstable. Tensor parallelism works with GGUF. The plugin's README says a model in vLLM's supported list is not guaranteed to load as GGUF, and its latest PyPI release, 0.0.5, is from 10 August 2026. The hardware table on vLLM's quantization page lists GGUF for Volta through Hopper and for AMD GPUs and has no Blackwell column, so check the plugin before planning GGUF on a B200, B300 or RTX PRO 6000.

llama.cpp reads GGUF natively. Its quantize README describes two phases, converting the original model to GGUF and quantizing that file. The GGUF guide explains the quant names, and the quantization guide says which formats run on which GPU generation.

Memory: fitting the model on the card

vLLM bounds a server with --gpu-memory-utilization, which its v0.33.0 argument reference defaults to 0.92. The main-branch reference now says 1.0 with the extensible KV cache and 0.92 otherwise, and that cache is sized from the memory actually free after CUDA graph capture. --max-model-len caps the context length and --kv-cache-dtype fp8 shrinks the cache. When a model is too big, --cpu-offload-gb offloads part of the weights, and the reference says it needs a fast CPU-GPU interconnect because those weights are loaded to the GPU on the fly in every forward pass.

llama.cpp's answer is hybrid inference. -ngl sets how many layers live in VRAM and the rest run on the CPU, which its multi-GPU page says makes inference much slower. --fit, on by default, adjusts unset arguments to fit device memory, and -ctk and -ctv set the KV cache type. On either engine the weights plus the KV cache decide whether a model fits, and the KV cache explainer has the arithmetic.

Concurrency: many callers or a few

vLLM's README lists continuous batching, chunked prefill and prefix caching, and its V1 guide says chunked prefill is on by default whenever possible. The limits you meet first are --max-model-len and --max-num-seqs, and the startup log prints a GPU KV cache size line and a Maximum concurrency estimate. llama-server also batches continuously, which its README lists as on by default. It divides work over -np slots, defaults to a unified KV buffer shared across sequences when the slot count is auto, and reports slot state on /slots. So llama-server exposes the slot count as a setting, while vLLM exposes limits and reports the capacity it found. The vLLM tuning guide covers the settings for many callers.

Multi-GPU: tensor parallel or layer split

vLLM's parallelism page says to use tensor parallelism when the model fits on one node but not on one GPU, with --tensor-parallel-size set to the GPU count. It says to use pipeline parallelism instead when the GPUs do not split the model evenly or lack NVLink, with the L40S as its example. It also supports expert and data parallelism.

llama.cpp multi-GPU support is documented on its multi-GPU page. --split-mode layer is the default, a pipeline where each GPU holds a slice of the layers and their KV cache, and the page calls it the most compatible choice and one that tolerates slow interconnects. row is deprecated and tensor is experimental, and the page says pipeline parallel maximizes batch throughput while tensor parallel minimizes latency. --tensor-split 3,1 sets uneven shares. The multi-GPU page on this site explains how the interconnect inside one DeployGPU VM changes the choice.

Start and operations

vLLM does work before the first request. Its optimization page lists three ways to shorten a repeat start: reuse the compile cache under VLLM_CACHE_ROOT, pass --kv-cache-memory-bytes to skip memory profiling (the page spells it --kv-cache-memory, while the argument reference and the source use the longer name), or use --enforce-eager, which skips compilation and graph capture at the cost of decode speed. llama-server loads the file and answers /health with 503 meanwhile.

The Docker images differ on CUDA. vLLM's v0.33.0 release notes say its images are CUDA 13 only and the -cu129 images are no longer published, and its Docker page says CUDA 13 needs an R580 or newer driver and documents a compatibility mode for older drivers. llama.cpp publishes server-cuda built with CUDA 12 and server-cuda13, so it has an image for a CUDA 12 header and one for CUDA 13. The vLLM guide and the llama.cpp server guide have the commands.

Decision table

When each one wins

In the llama.cpp vs vLLM choice I start from the format of the model I want. vLLM wins when the model exists as a Hugging Face checkpoint, an official quantized build exists for your GPU and several callers share the server. llama.cpp wins when the model you want is a GGUF file, when one or a few users share it, when it has to squeeze onto less memory or when the hardware is not NVIDIA. The GPU VM families DeployGPU offers are all NVIDIA, so the hardware row does not decide it there. My default splits the way the table does: vLLM the moment code, not a person, is calling the model, and llama.cpp or Ollama when a person is. The Ollama comparison and vLLM vs Ollama cover that side, and vLLM vs SGLang covers the other server-grade engine.

On DeployGPU

Both are do-it-yourself on a plain GPU VM, bound to 127.0.0.1 and reached through an SSH tunnel, because DeployGPU has no vLLM or llama.cpp template. The Ollama guide covers the browser chat route.