Run gpt-oss-120b and 20b on a cloud GPU

The short answer for gpt-oss-120b is one GPU with 80 GB or more: an A100 or H100 80GB, an RTX PRO 6000 or an H200 all have room. gpt-oss-20b is smaller still and fits a 48 GB card with space left for a long conversation. Both numbers come from the checkpoint OpenAI actually publishes, not from treating the model as if it shipped in 16-bit precision.

Why 16-bit math gets gpt-oss wrong

Sizing gpt-oss as if it shipped in 16-bit precision gives a number four times too large: 116.83 billion parameters (Hugging Face's count from the checkpoint, against 117 billion on the model card) times 2 bytes is about 234 GB. gpt-oss does not ship that way. OpenAI quantizes the mixture-of-experts weights to MXFP4, 4.25 bits per parameter, and the published checkpoint sums to 65.2 GB across its safetensors files. The 20b model follows the same recipe: 20.91 billion parameters and a 13.8 GB checkpoint, not the roughly 42 GB that plain 16-bit precision would need. The model card states the MXFP4 format, and the exact bit width is documented in OpenAI's technical report.

Which GPU to launch

My rule of thumb is to treat 80 GB as the floor for gpt-oss-120b and not try to squeeze it onto anything smaller. vLLM reserves a fraction of the card for the model and its running state by default, so on an 80 GB GPU the checkpoint leaves roughly 8 GB for everything else, calculated below. That headroom is tight for several long-context users at once, so an H200 with 141 GB, or two GPUs with tensor parallelism, give more room to grow. vLLM's own recipe for this model lists a single 80 GB A100, H100, H200 or B200 as enough to serve it, with Blackwell cards using a dedicated FlashInfer MXFP4 kernel and Hopper cards using a different code path that does not support an FP8 KV cache for this model, according to the recipe.

Serve it with vLLM

Launch a GPU VM rather than a template for this. DeployGPU has no vLLM template, and app templates only open a private browser link, so an OpenAI-compatible server needs a plain VM with SSH and Docker. The official vLLM instructions publish the API port to every interface on the machine, which is fine on hardware you firewall yourself but is not safe here, since DeployGPU does not firewall VMs and Hyperstack VMs in particular are created with inbound traffic open. Bind to localhost instead, and reach the server over an SSH tunnel:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN=$HF_TOKEN \
  vllm/vllm-openai:latest \
  --model openai/gpt-oss-120b \
  --max-model-len 32768

Swap the model argument for openai/gpt-oss-20b on a 48 GB card, and raise or drop --max-model-len once the KV cache math below tells you what the card can hold. vLLM 0.10.1 or newer supports both sizes. Watch the container's own log for the line that reports the KV cache size and the maximum number of sequences it found room for, since that reflects the actual GPU rather than this calculation:

sudo docker logs -f vllm

Then open a tunnel from the machine that will call the API, using the user, host and port shown on the deployment, and confirm the server answers:

ssh -N -L 8000:127.0.0.1:8000 ubuntu@YOUR_VM_IP -p 22
curl http://localhost:8000/v1/models

The full walk-through, including tensor parallelism across more than one GPU and the --api-key flag for anyone who decides to expose the port beyond localhost, is in the vLLM guide.

Or chat with it instead

The Open WebUI + Ollama template is the faster path when the goal is a chat interface rather than an API. Launch it, pull gpt-oss:20b or gpt-oss:120b, and use the private browser link. Ollama's library lists gpt-oss:20b at 14 GB and gpt-oss:120b at 65 GB, both quantized to MXFP4 like the original checkpoint, checked 28 September 2026.

How much context and how many users fit

gpt-oss-120b has 36 attention layers split evenly: 18 use full attention and grow with every token, and 18 use a sliding window of 128 tokens that never grows past a small fixed size, whatever the conversation length. Only the 18 full layers matter for long-context planning. Each token costs 2 (for the key and the value) times 8 KV heads times 64 dimensions times 2 bytes in BF16, times 18 growing layers: 36 KiB per token. A 32,000-token conversation costs about 1.1 GiB of cache, and a 128,000-token one costs about 4.5 GiB, calculated from the published config.json. gpt-oss-20b has half as many full layers and costs 24 KiB per token, about 0.75 GiB at 32,000 tokens.

Put that next to what is left after loading weights. vLLM's gpu_memory_utilization setting defaults to 0.92 on the current main branch, 0.9 in older releases, and that fraction has to cover activations and CUDA graphs as well as the KV cache. On one 80 GB GPU running gpt-oss-120b, 0.92 times 80 GB is about 73.6 GB, and the 65.2 GB checkpoint leaves roughly 8 GB for everything past the weights, room for a handful of long-context sessions rather than dozens. The engine prints the exact KV cache size and maximum concurrency it settled on when it starts, and that log line is the number to trust over this estimate. The KV cache explainer goes through the same formula for models with sliding-window and linear-attention layers.

Fine-tuning and the license

Both sizes carry an Apache-2.0 license, so commercial use, fine-tuning and redistribution do not need a separate agreement. OpenAI's model card says gpt-oss-120b can be fine-tuned on a single H100 node and that gpt-oss-20b fine-tunes on consumer hardware. Axolotl ships example configs for both, and Unsloth supports gpt-oss-20b directly. Treat those as vendor statements rather than something calculated here, since fine-tuning memory depends on the training method as much as the model.

Stopping and the next run

Stopping a DeployGPU instance deletes its disk along with whatever you downloaded, so the 65 GB or 14 GB model comes down again on the next launch. Budget that against the provider's network speed rather than assuming it happens instantly. The guide to stopping without losing work covers what to copy off first, and pricing covers what the hour itself costs.