Run Mistral models on a cloud GPU

The Mistral models you can host yourself run from Ministral 3 at 3 billion parameters, which fits one 48 GB card with room to spare, to Mistral Large 4 at 1.05 trillion, whose 1,060.55 GB FP8 checkpoint needs 8x B200 or 8x B300. Mistral Small 4 sits at 120.93 GB and Mistral Medium 3.5 at 133.61 GB, and both fit 2x H200. Most of these weights are Apache 2.0. Mistral Medium 3.5 and Devstral 2 carry a Modified MIT License with a revenue limit, and Mistral Large 4 has a license of its own for Model as a Service businesses.

Open-weight Mistral models and the GPU each needs

Parameters come from Mistral's model pages, which its models overview links, and the licenses from the Hugging Face license field and the LICENSE files. Most repositories hold the weights twice, in Mistral's consolidated format and in Transformers format, so the sizes below are one copy of the summed safetensors files. Every page linked here was checked on 12 November 2026.

The left-over figures are 92% of the VM's memory minus the checkpoint, so they leave out activations and CUDA graphs. Mistral's overview lists Mistral Large 3, Medium 3.5, Devstral 2, Devstral Small 2 and Magistral Small 1.2 as deprecated in its API, with Large 4, Medium 3.5 and Small 4 as replacements, but the weights stay public. The 80 GB tier holds the Magistral build, and two RTX PRO 6000 cards from the 96 GB-plus tier hold Small 4 with about 56 GB left by the memory rule, although neither the card nor the recipe lists that shape.

Which Mistral Large is which

Mistral Large 3 is the December 2025 Large model: 675 billion parameters with 41 billion active, a 256k context and the Apache 2.0 license, released on 2 December 2025 according to its model page. The overview lists it as deprecated on 31 October 2026, retiring on 30 April 2027, with Mistral Large 4 as the replacement. vLLM's recipe for Large 3 serves FP8 on 8x H200 and NVFP4 on 4x B200, and says NVFP4 suits contexts under 64K tokens. It adds that on older GPUs such as the A100 and H100 vLLM falls back to Marlin FP4, so by the memory rule an 8x H100 VM holds the NVFP4 build with about 186 GB left.

Mistral Large 4 has 1.05 trillion parameters with 52 billion active, a 1M context and a release date of 5 November 2026 on its model page. Its FP8 checkpoint is too large for 8x H200, because 1,128 GB times 92% is 1,037.76 GB. vLLM's recipe validates 8x B300 for both builds and 8x B200 for NVFP4, and it lists H200 only as 2 nodes of 8, which is a cluster and not one VM. Ollama's mistral-large tag is neither of these: it is still Mistral Large 2, the 123B model.

The recipe says no vLLM release includes Large 4 yet and names the vllm/vllm-openai:mistral-large-4 image. This is the NVFP4 card's command for 8 GPUs, wrapped in Docker with the port published on the VM's loopback address:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:mistral-large-4 \
  --model mistralai/Mistral-Large-4-1T-A52B-NVFP4 \
  --tensor-parallel-size 8 \
  --kv-cache-dtype fp8_ds_mla \
  --tool-call-parser mistral \
  --reasoning-parser mistral \
  --enable-auto-tool-choice \
  --gpu-memory-utilization 0.85

Reach the server through an SSH tunnel, as in the vLLM guide. The FP8 card's command names mistralai/Mistral-Large-4-1T-A52B and leaves out the KV cache flag. The NVFP4 card says the format targets Blackwell GPUs, so the quantization guide is the place to check a format against a GPU generation. SGLang has an lmsysorg/sglang:mistral-large-4 image. An 8x B300 VM holds either build, and Auto Provision can deploy one when stock appears.

Serve Small 4 and Medium 3.5 with vLLM

Mistral Small 4's card says it unifies three earlier families, Instruct, Reasoning (previously called Magistral) and Devstral, in one model. vLLM's recipe lists 2x H200 or 2x B200 for FP8 and asks for vLLM 0.20.0 or newer. Run nvidia-smi first, because the latest image is vLLM v0.33.0, whose release notes say its images are CUDA 13.0 only, and vLLM's Docker page says that needs an R580 or newer driver. On an older driver, use v0.32.0-cu129, the last release with a CUDA 12.9 image (checked 12 November 2026). This is the recipe's command in Docker:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest mistralai/Mistral-Small-4-119B-2603 \
  --max-model-len 262144 \
  --tensor-parallel-size 2 \
  --tool-call-parser mistral --enable-auto-tool-choice \
  --reasoning-parser mistral \
  --max-num-batched-tokens 16384 --max-num-seqs 128 \
  --gpu-memory-utilization 0.8

The model card's command also adds --attention-backend FLASH_ATTN_MLA. The recipe says the full 256K context can run out of memory on 2x H200, and to lower --max-model-len to 131072 or 65536 or use the NVFP4 build, which it runs on one B200 with a reduced context. The multi-GPU page explains how the two GPUs share the model.

Mistral Medium 3.5 is dense, and its recipe recommends 8x H200 with 4x B200 as the alternative. It says one B200 also holds the weights but leaves little room for the 256K cache. The cache costs 352 KiB per token in BF16 (88 layers, 8 KV heads, head dimension 128, from its config.json), so the 126 GB left on 2x H200 holds about 349,000 tokens, calculated, and the KV cache explainer shows the method. The recipe says the architecture had not shipped in a stable release and asks for a nightly vLLM build, and the card says the same. This is the recipe's command for 8 GPUs with the nightly image that the card links:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:nightly mistralai/Mistral-Medium-3.5-128B \
  --tensor-parallel-size 8 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral \
  --enable-auto-tool-choice --tool-call-parser mistral \
  --reasoning-parser mistral

The tokenizer, config format and load format flags

The three flags in that command are Mistral-specific. vLLM's tool calling page says "The official format that is used by default with auto or mistral arguments" is --tokenizer_mode mistral --config_format mistral --load_format mistral, and that it uses mistral-common, Mistral AI's tokenizer backend. The Transformers format uses hf for all three. vLLM's engine arguments say mistral for --tokenizer-mode "will always use the tokenizer from mistral_common", for --config-format "will load the config in mistral format" and for --load-format "will load weights from consolidated safetensors files used by Mistral models". The Ministral 3, Magistral and Medium 3.5 sources spell all three out, the Small 4 and Large 4 commands rely on the default, and the Voxtral Realtime recipe makes --tokenizer-mode mistral mandatory.

The two formats also double the download. Medium 3.5's repository is 267.21 GB for a 133.61 GB checkpoint, so a full download of such a repository fetches both copies, and the disk in an offer's vCPU / RAM / Disk column has to hold the larger figure if you do not filter the files.

The Mistral licenses, quoted

By the Hugging Face license field and Mistral's overview, the Apache 2.0 weights include Mistral Small 4, Mistral Large 3, Ministral 3, Magistral Small 1.2, Devstral Small 2, Voxtral Small and Mistral 7B. The others have their own terms.

Mistral Medium 3.5 and Devstral 2 use the Modified MIT License. Its file says: "You are not authorized to exercise any rights under this license if the global consolidated monthly revenue of your company (or that of your employer) exceeds $20 million (or its equivalent in another currency) for the preceding month." The file applies the restriction to the model and "any derivatives, modifications, or combined works based on it", and says Mistral AI may grant a commercial license "at its sole discretion".

Mistral Large 4 uses the Mistral Ecosystem Open-Weight License, which its file calls "a modified version of the MIT license". Its second condition reads: "If the Licensee or any of its affiliates operates a Model as a Service business, the Licensee must enter into a separate commercial license with Mistral before using the Model or its derivative works for any commercial purpose." The file defines Model as a Service as "giving a third party access to the Model or its derivative works for inference or post-training by any means (for instance via API or hosted endpoints), in a manner that allows such third party to exercise meaningful control over the inputs, parameters, or training data". It exempts "internal use of the Model or its derivative works, defined as any use that does not make the Model or its derivative works available to third parties". The condition has no revenue threshold. Voxtral 4B TTS is CC BY-NC 4.0, which is non-commercial. Read each LICENSE file before you ship.

Devstral, Magistral and what replaced them

Mistral's overview lists Mistral Medium 3.5 as the replacement for Devstral 2 and Devstral Small 2, and Mistral Small 4 as the replacement for Magistral Small 1.2. The Medium 3.5 card says it "supersedes all our previous coding models, namely Devstral, across all benchmarks" and reports 77.6% on SWE-Bench Verified, while the Devstral 2 card reports 72.2% and the Devstral Small 2 card 68.0%. Those are Mistral's own numbers on their own setups, so they are not a ranking. The Devstral weights stay public, and Mistral's offline models page for its Vibe coding CLI still recommends Devstral Small 2 and Mistral Small 4 for local use. Devstral Small 2 fits one 48 GB card and its card's command needs --tool-call-parser mistral and --enable-auto-tool-choice. Magistral Small 1.2 is 48.02 GB in BF16 and needs one 80 GB card. The open coding models page sets the coding models beside the other families by GPU.

Ollama and GGUF for Mistral models

On 12 November 2026 Ollama's mistral-medium-3.5 page lists 128b tags at 80 GB (q4_K_M), 138 GB (q8_0) and 255 GB (bf16), devstral-small-2 lists 15, 26 and 48 GB, devstral-2 lists 75, 133 and 250 GB, and ministral-3 lists 3b at 3.0 GB, 8b at 6.0 GB and 14b at 9.1 GB. Its mistral-large-4 page lists only a cloud tag, mistral-large-3 has no models, and I found no Mistral Small 4 page: mistral-small is still Mistral Small 3 at 14 GB. magistral is the 2506 release with a 39K context. A 48 GB card runs the Ministral 3 tags and the 15 GB and 26 GB Devstral Small 2 tags through Open WebUI. The Ollama server guide covers the service for the larger tags.

Voxtral for speech

Voxtral is Mistral's speech family. The open weights include Voxtral Small 24B and Voxtral Mini from July 2025 and Voxtral Mini 4B Realtime, all Apache 2.0, plus Voxtral 4B TTS under CC BY-NC 4.0. The Voxtral 3 models announced on 20 October 2026 are API models in Public Preview, and Mistral's model pages list no open weights for them. Voxtral Small is 48.52 GB in BF16, so it needs one 80 GB card or 2x 48 GB, and its card says about 55 GB of GPU memory with --tensor-parallel-size 2. The Realtime model is 8.86 GB, and its recipe asks for a single GPU with 16 GB or more and vLLM 0.20.0 or newer. For Whisper instead, the Whisper guide has the setup.

Mistral 7B in 2026

Mistral 7B is the 2023 model that many searches still mean. Its weights are Apache 2.0 and the v0.3 checkpoint is 14.50 GB in BF16, so one 48 GB card holds it with about 30 GB left. Mistral's overview lists it as retired from the API since 30 March 2025, with Ministral 3 8B as the replacement. vLLM's tool calling page lists mistralai/Mistral-7B-Instruct-v0.3 as the confirmed model for its mistral parser and says Mistral 7B struggles to generate parallel tool calls correctly. Ollama's mistral page is v0.3 at 4.4 GB with a 32K context.

Disk space and download time

Stopping a DeployGPU instance deletes its disk, so every session downloads the weights again. At an assumed 10 Gbit/s the 1,060.55 GB Large 4 checkpoint takes about 14 minutes and at 1 Gbit/s about 2.4 hours, calculated because provider network speeds are not published. The download guide covers faster ways to pull large checkpoints and the stop guide shows what to copy off first.