Run Kimi K3 on cloud GPUs

The Kimi K3 weights are open, and at 1,560.94 GB the checkpoint decides the hardware: an 8x B300 VM is the only DeployGPU VM that holds it, with about 371 GB left for the cache and runtime (calculated). Kimi K3.1-Pro, whose Hugging Face repository was created on 2 November 2026, is 1,561.29 GB and needs the same VM. Ollama lists both only as cloud models, so there is no local build to pull, and the license lets you run them yourself with a few conditions.

Kimi K3 is open weight and 1,561 GB

Moonshot AI published Kimi K3 on Hugging Face as moonshotai/Kimi-K3. Its model card lists 2.8 trillion parameters, 104 billion active per token, 93 layers and a 1,048,576-token context, with text and image input. The weights ship in 4-bit MXFP4, trained that way with quantization-aware training, and the 96 safetensors files sum to 1,560.94 GB. Every card, recipe and cookbook linked on this page was checked on 12 November 2026.

The weights decide the hardware, not the cache. Only 24 of the 93 layers keep a KV cache that grows with the prompt, each holding 576 values per token, so a BF16 cache costs about 27 KiB per token, or about 27 GiB for one sequence at the full context (calculated from the config). The other 69 layers use Kimi Delta Attention and keep a fixed state per sequence that this figure leaves out. The KV cache explainer shows how such figures are worked out.

Only an 8x B300 VM holds it

NVIDIA's HGX page lists 2.1 TB of total memory for an 8-GPU HGX B300. Each cell below is 92% of the VM's memory minus the checkpoint, calculated with the same rule as the GPU pages. It leaves out activations, CUDA graphs and the vision encoder, so read it as an upper bound on the cache. The engine's startup log has the real figure.

vLLM's recipe renders Kimi K3 on one 8x B300 node at tensor parallel 8, on two nodes of 8 for B200 or H200 and on four nodes of 8 for H100. SGLang's cookbook lists B300 as 1 node of 8, B200 as 2 nodes of 8, H200 as 2 nodes of 8 for most profiles and H100 as 4 nodes of 8. DeployGPU has no multi-node VMs, so those shapes are a cluster. Kimi K2.7 Code, the K2-generation coding model, is far smaller: 1 trillion parameters, 32 billion active, a 256K context and a 595.18 GB checkpoint. The frontier models page has its command, and the DeepSeek page covers V4-Pro.

When the B300 is out of stock

B300 stock comes and goes, and Auto Provision is the dashboard page for waiting it out. You pick the GPU, the providers you accept from Verda, Hyperstack and QuantaCloud, and a maximum price per hour for the whole machine with fees included. DeployGPU watches their stock and deploys the moment a machine shows up. One order deploys one machine, and if someone else gets it first the order keeps waiting for the next one.

Placing an order and waiting both cost nothing. When a machine appears your balance must cover one hour at its price, and it deploys only at or below your maximum. It is a plain GPU VM with no app template and no volume, and it works from the dashboard only, not the API. One order for an 8x B300 VM was filled on 27 October 2026: the create request went out 1.7 seconds after the stock appeared and the VM was running about 3 minutes later. That is one event, not a rate. The Auto Provision page has the rules, the B300 page covers the GPU, and you place the order on the Auto Provision page in the dashboard.

The other way around a missing B300 is a cluster. Running the full model on B200 or H200 takes 16 GPUs, and anything over 8 goes through the cluster quote form, which a person answers. A smaller GGUF quantization is the third option, covered below.

Serve Kimi K3 with vLLM or SGLang

vLLM's recipe names a vllm/vllm-openai:kimi-k3 image in its prerequisites, a CUDA 13 build that needs an NVIDIA driver from the R580 branch or newer, so run nvidia-smi before pulling anything. Its Docker command for one 8x B300 node uses vllm/vllm-openai:latest and asks for vLLM 0.29.0 or newer. The latest image is vLLM 0.33.0, whose release notes say its images are CUDA 13.0 only, so the same R580 driver rule applies. This is that command with sudo, a container name and the port published on the VM's loopback address. It keeps --privileged as written and the recipe's engine timeout of 3,600 seconds for the first start, which loads 1,561 GB:

sudo docker run -d --name vllm --gpus all --privileged --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_ALLREDUCE_USE_FLASHINFER=1 \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \
  -e VLLM_USE_V2_MODEL_RUNNER=1 \
  -e VLLM_USE_RUST_FRONTEND=1 \
  vllm/vllm-openai:latest moonshotai/Kimi-K3 \
  --trust-remote-code \
  --gpu-memory-utilization 0.95 \
  --tensor-parallel-size 8 \
  --load-format fastsafetensors \
  --disable-flashinfer-autotune \
  --max-model-len 1048576 \
  --kv-cache-dtype fp8 \
  --attention-config '{"use_prefill_query_quantization":true,"mla_prefill_backend":"TOKENSPEED_MLA"}' \
  --enable-prefix-caching \
  --attention-backend TOKENSPEED_MLA \
  --prefix-match-unit 128 \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

SGLang's cookbook builds its commands in an interactive configurator and pins --mamba-full-memory-ratio from a calculator that sizes the KDA state pool for your average request length, so use the generated command for real work. The only single-node command printed in full on that page is its low-memory vision profile. This is that command without its 65,536-token context cap and its memory flags (--context-length, --enable-symm-mem, --mem-fraction-static and --mm-feature-transport), in the lmsysorg/sglang:latest image the cookbook names for NVIDIA:

sudo docker run -d --name sglang --gpus all --shm-size 32g --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang:latest \
  sglang serve \
  --trust-remote-code \
  --model-path moonshotai/Kimi-K3 \
  --tp-size 8 \
  --reasoning-parser kimi_k3 \
  --tool-call-parser kimi_k3 \
  --host 0.0.0.0 \
  --port 30000

SGLang's installation page asks for a CUDA 13-compatible driver for its current images, so run the same nvidia-smi check before you pull lmsysorg/sglang:latest.

The --host 0.0.0.0 stays inside the container, where only the loopback-published port reaches it. Reach either server through an SSH tunnel, as in the vLLM guide and the SGLang guide. The model card says Kimi K3 always thinks, takes a reasoning_effort of low, high or max with max as the default, and needs the whole assistant message, including reasoning_content and tool_calls, passed back on later turns.

What Kimi K3.1-Pro changes

The Kimi K3.1-Pro repository on Hugging Face was created on 2 November 2026. Its card calls it post-trained from Kimi K3 with large-scale joint reinforcement learning and the first model in the K3.1 series. The size class is the same: 2.8 trillion parameters, 104 billion active, 93 layers, MXFP4 weights and a 1,048,576-token context. The 96 safetensors files sum to 1,561.29 GB, about 0.4 GB more than Kimi K3, so the VM is the same 8x B300 with about 371 GB left. The one architecture change the card lists is that its 24 full-attention layers use Gated DeepSeek Sparse Attention instead of Gated MLA, for more efficient long-context modeling.

Engine support is new and moving. vLLM's recipe renders one B300 node at tensor parallel 8 and asks for vLLM 0.34.0, which it calls not released yet. The newest release on PyPI today is 0.33.0, so use the vllm/vllm-openai:kimi-k31 image the recipe names. SGLang's cookbook installs it from the kimi-k3.1-day-0 branch, names an lmsysorg/sglang:dev-kimi-k31-pro image and marks most of its recipes as still in verification. This is the vLLM recipe's command for one B300 node, whose throughput the recipe says was not tuned:

sudo docker run -d --name vllm --gpus all --privileged --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \
  vllm/vllm-openai:kimi-k31 moonshotai/Kimi-K3.1-Pro \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --load-format fastsafetensors \
  --enable-auto-tool-choice \
  --tool-call-parser kimi_k3 \
  --reasoning-parser kimi_k3

GGUF files and Ollama

Ollama's Kimi K3 page lists one tag, kimi-k3:cloud, and its Kimi K3.1-Pro page lists kimi-k3.1-pro:cloud. Both are hosted models, so ollama run downloads no weights. GGUF files do exist from third parties. Unsloth's unsloth/Kimi-K3-GGUF holds nine quantizations, from 466.37 GB for UD-Q1_0 to 1,561.16 GB for UD-Q8_K_XL, and the 4-bit UD-Q4_K_XL is 1,508.67 GB (summed per folder). llama.cpp merged the Kimi K3 text architecture on 15 August 2026 in pull request 26185 and the K3.1-Pro one on 6 November in pull request 31779, while Unsloth's card points to its own llama.cpp fork and to Unsloth Studio.

Running Kimi K3 locally still means hundreds of gigabytes of memory, because even the smallest file is 466.37 GB. By the memory rule the 4-bit file leaves an 8x B300 VM about 423 GB, the 2-bit UD-Q2_K_XL at 861.28 GB leaves an 8x H200 VM about 176 GB, and the 1-bit UD-IQ1_S at 594.04 GB leaves it about 444 GB (calculated). Low bit widths lose accuracy, and Unsloth's card links its quantization benchmarks. The GGUF guide explains the format and the llama.cpp server guide covers the engine.

When Moonshot's API is the better choice

I would not rent an 8x node just to try Kimi K3. The model cards say Moonshot serves them at platform.kimi.ai as kimi-k3 and kimi-k3.1-pro, with an OpenAI and Anthropic compatible API. A few prompts do not need 1.5 TB of weights on a rented disk. A node pays off when you keep its GPUs busy for hours, when prompts must not go to a hosted API, or when you need serving settings an API does not expose. The break-even is the hourly rate in the offers table divided by the tokens your workload pushes through in an hour, and Moonshot's pricing page has the other side of that sum.

The Kimi K3 license in plain terms

The Kimi K3 License lets you use, copy, modify, deploy, fine-tune and sell copies of the weights, with conditions. A business that runs a Model as a Service, meaning it gives third parties inference or fine-tuning access they can control, for example through an API, needs a separate agreement with Moonshot AI before commercial use once its revenue passes 20 million US dollars over any 12 months. A product with more than 100 million monthly active users, or more than 20 million US dollars in monthly revenue, must display "Kimi K3" on its interface. Neither rule applies to internal use that does not make the model, its outputs or its capabilities available to third parties, or to use through Moonshot's official products and certified inference partners.

The Kimi K3.1-Pro License has the same text with "Kimi K3.1-Pro" as the name to display. Kimi K2.7 Code uses a Modified MIT License whose only change is the same display rule. These are summaries, so read each text before you ship.

Disk space and download time

The disk size in the offers table, the last figure in the vCPU / RAM / Disk column, matters as much as the GPU. Disks are fixed per offer on QuantaCloud, Massed Compute and Hyperstack, and adjustable on Verda up to the offer's limit, so compare it with the checkpoint: 1,561 GB for Kimi K3 and for K3.1-Pro. An offer with a smaller disk cannot hold the weights. A volume on Verda or Hyperstack is the exception, because it survives stopping and bills hourly while it exists.

The download guide covers hf download --dry-run and faster ways to pull large checkpoints, and the stop guide covers what survives a stop.