What it takes to self-host Kimi K3 and other frontier models

The Kimi K3 checkpoint is 1,560.94 GB, which is more than an 8x B200 VM holds and fits an 8x B300 VM with about 371 GB left for the cache and runtime (calculated). DeepSeek V4-Pro, GLM-5.3, MiniMax-M3 and Kimi K2.7 Code are smaller and fit an 8x H200 VM. All of them need a full 8-GPU VM, the largest DeployGPU launches, and anything bigger goes through the cluster quote form.

Kimi K3 needs 1,561 GB for the weights alone

Moonshot AI's model card lists 2.8 trillion parameters, 104 billion active per token, 93 layers and a 1,048,576-token context. The weights ship in 4-bit MXFP4, trained that way with quantization-aware training. The safetensors files sum to 1,560.94 GB. Only 24 of the 93 layers grow a KV cache, about 27 KiB per token (calculated), and the 69 KDA layers keep a fixed state per sequence that this figure leaves out. The weights still decide the hardware.

vLLM's recipe asks for at least 8x GB300 and, on H100, a minimum of 32 GPUs. The SGLang cookbook sizes it as 1 node of 8 on B300, 2 nodes of 8 on B200 and 2 nodes of 8 on H200. NVIDIA's HGX page lists 2.1 TB of total memory for an 8-GPU HGX B300. That makes 8x B300 the only DeployGPU VM shape that holds the checkpoint, and B300 stock comes and goes:

Both engines need CUDA 13 builds for Kimi K3: vLLM's recipe uses the vllm/vllm-openai:kimi-k3 image and an NVIDIA driver from the r580 branch or newer, so run nvidia-smi before pulling anything. The SGLang cookbook builds its commands in an interactive configurator on that page. For 8x B300 it runs in SGLang's Kimi K3 image, and these are the essential flags, with the port published on the VM's loopback address only:

sudo docker run -it --gpus all --shm-size 32g --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang:kimi-k3 bash

sglang serve \
  --trust-remote-code \
  --model-path moonshotai/Kimi-K3 \
  --tp-size 8 \
  --reasoning-parser kimi_k3 \
  --tool-call-parser kimi_k3 \
  --host 0.0.0.0 \
  --port 30000

The --host 0.0.0.0 stays inside the container, where only the loopback-published port reaches it. For the other profiles, such as speculative decoding, copy the command the configurator generates. Ollama lists Kimi K3 as a cloud model with a single tag, kimi-k3:cloud, so there is no local build to pull.

Which frontier models fit one 8-GPU VM

Each cell is 92% of the VM's memory minus the checkpoint, calculated with the same rule as the GPU pages. It leaves out activations, CUDA graphs and any vision encoder, so read it as an upper bound on the KV cache. The engine's startup log has the real figure.

Checkpoint sizes are the summed safetensors files on each model page, and every card and recipe linked here was checked on 1 October 2026. The recipes ask for more than the weights. vLLM sizes DeepSeek V4-Pro at 960 GB, sizes GLM-5.3 at 893 GB and wants 8x B200 for the full 1M-token window, and calls MiniMax-M3 in BF16 a tight fit on 8x H200. MiniMax-M3 also needs vLLM's dedicated vllm/vllm-openai:minimax-m3 image and --block-size 128. The DeepSeek V4 page and the GLM-5.3 page have the commands for those two.

Serving Kimi K2.7 Code on 8 H200s

Kimi K2.7 Code is the K2-generation model in this group. Its model card lists 1 trillion parameters, 32 billion active and a 256K context, and the INT4 checkpoint sums to 595.18 GB. vLLM's recipe lists 8x H200 as verified and asks for vLLM 0.19.1 or newer. This is the recipe's command wrapped in the official Docker image, with the port bound to loopback:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  --model moonshotai/Kimi-K2.7-Code \
  --tensor-parallel-size 8 \
  --mm-encoder-tp-mode data \
  --trust-remote-code \
  --tool-call-parser kimi_k2 \
  --enable-auto-tool-choice \
  --reasoning-parser kimi_k2

Reach it through the SSH tunnel in the vLLM guide. Pick an 8x H200 offer whose Interconnect column says NVLink (SXM). The H200 page explains the difference: SXM joins up to 8 GPUs through NVSwitch, while an NVL bridge joins only 2 or 4.

Disk space and a 1.5 TB download

The Disk column in those tables matters as much as the GPU. Disks are fixed per offer on QuantaCloud, Massed Compute and Hyperstack, and adjustable on Verda up to the offer's limit, so compare it with the checkpoint: 1,561 GB for Kimi K3, 893 GB for DeepSeek V4-Pro. An offer with a smaller disk cannot hold the weights.

A volume on Verda or Hyperstack is the exception: it survives stopping and bills hourly while it exists. The download guide covers faster ways to pull large checkpoints, and the stop guide covers what survives a stop.

Past 8 GPUs: Qwen3.8-2.4T and the cluster form

DeployGPU VMs top out at 8 GPUs and there is no multi-node. Qwen3.8-2.4T-A95B shows where that ends. vLLM's recipe sizes the BF16 checkpoint (4,892.37 GB) at 24 B300 GPUs and the official FP8 build (2,496.10 GB) at 16 GPUs. It also lists an NVFP4 build from Inferact, 1,447.33 GB, at 8 B300 GPUs, which needs a nightly vLLM. Kimi K3 on B200 or H200 sits in the same group, since SGLang's cookbook wants 2 nodes of 8 for either. Anything that needs 16 GPUs or more goes through the cluster quote form, and a person answers it.

When a hosted API is the better choice

I would not rent an 8x node just to try Kimi K3. The model card points to Moonshot's own API at platform.kimi.ai, and Ollama serves kimi-k3:cloud, so a few prompts do not need 1.5 TB of weights on a rented disk. A node pays off when you keep its GPUs busy for hours, when prompts must not go to a hosted model API, or when you need serving settings an API does not expose. The break-even is the hourly rate in the offers table divided by the tokens your workload pushes through in an hour.

Licenses for the five models in the table

Kimi K3 uses the Kimi K3 License. A Model as a Service business above 20 million US dollars in revenue over any 12 months needs a separate agreement with Moonshot AI, and products above 100 million monthly users or 20 million US dollars in monthly revenue must display the model name. Kimi K2.7 Code has a Modified MIT License with the same display rule, and DeepSeek V4-Pro is MIT. GLM-5.3 has its own license that asks Model as a Service businesses above 10 billion US dollars in revenue to pass a Z.AI security review. The MiniMax Community License asks commercial users to show "Built with MiniMax M3", and companies above 20 million US dollars in yearly revenue need MiniMax's authorization. These are summaries, so read each text first.