Run Qwen3-Coder on a cloud GPU

The Qwen3-Coder model for one GPU is Qwen3-Coder-30B-A3B. It is 61.07 GB in BF16, which an 80 GB card holds with about 12 GB left, and 31.18 GB in Qwen's official FP8 build, which fits a 48 GB Ada card. Qwen3-Coder-Next is the larger 80B model at 159.36 GB and takes 2x H200 or 4x 80 GB GPUs. Both are Apache-2.0, and an SSH tunnel is how a coding tool on your own machine reaches either one.

Two coder models: 61 and 159 GB

The 30B-A3B card lists 30.5B parameters with 3.3B active, 48 layers, 4 KV heads and a native context of 262,144 tokens. The Next card lists 80B parameters with 3B active, 48 layers in a hybrid of Gated DeltaNet and attention, and the same context. Both models run in non-thinking mode only. Sizes are the summed safetensors files and tag sizes come from Ollama's library, checked 1 October 2026.

The left-over figures are 92% of the VM's memory minus the checkpoint, so they leave out activations. FP8 runs natively on the Ada cards in the 48 GB tier but not on the RTX A6000. The 30B costs 96 KiB of KV cache per token, which is 3 GiB for a 32,000-token session and 12 GiB for a 128,000-token one (calculated from its config), so the 12 GB left on an 80 GB card is a few short sessions and the 27 GB on an RTX PRO 6000 is one session near the full context. Next has 12 attention layers and costs 24 KiB per token, plus a fixed state for its DeltaNet layers. The KV cache explainer has the method, and an FP8 KV cache halves them where the model supports it.

Serve the 30B-A3B with vLLM

Qwen's README says Qwen3-Coder function calling relies on a dedicated tool parser in vLLM and SGLang. The flags for it come from the Next card and vLLM's recipe for the 480B coder. This is the 30B in Docker, with the port bound to loopback and a 64K context to start with:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  --model Qwen/Qwen3-Coder-30B-A3B-Instruct \
  --max-model-len 65536 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

The card recommends temperature=0.7, top_p=0.8, top_k=20 and repetition_penalty=1.05, and an output length of 65,536 tokens for most queries, so set those in the coding tool. Raise --max-model-len only once sudo docker logs vllm shows the KV cache has room.

Serve Qwen3-Coder-Next on 2x H200

The Next card's vLLM command uses a tensor parallel size of 2 and needs vLLM 0.15.0 or newer. It says to reduce the context to 32,768 if you hit out-of-memory errors, with --max-model-len 32768. The card also gives an SGLang command with the same tool parser and a tensor parallel size of 2, and it needs SGLang 0.5.8 or newer. In Docker:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  --model Qwen/Qwen3-Coder-Next \
  --tensor-parallel-size 2 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

On four 80 GB GPUs, set the tensor parallel size to 4. Pick an H200 offer whose Interconnect column says NVLink, as the H200 page explains.

Point a coding tool at your own endpoint

Open the tunnel from your own machine with the user, host and port from the SSH command on your deployment, and leave it running. Qwen Code's authentication docs list OPENAI_API_KEY, OPENAI_BASE_URL and OPENAI_MODEL for any OpenAI-compatible endpoint, and any tool with a base URL setting works the same way:

ssh -N -L 8000:127.0.0.1:8000 ubuntu@YOUR_VM_IP -p 22

export OPENAI_BASE_URL="http://localhost:8000/v1"
export OPENAI_API_KEY="EMPTY"
export OPENAI_MODEL="Qwen/Qwen3-Coder-30B-A3B-Instruct"

The server has no key, so the placeholder is enough. Check the model name with curl http://localhost:8000/v1/models. The port forwarding guide covers keeping a tunnel alive, and the vLLM guide has the rest of the server setup. Qwen's README lists Qwen Code, Cline and Claude Code among the tools the models are built for, and each tool has its own way to set a custom endpoint, so check its documentation.

Ollama, GGUF and the license

For chat, qwen3-coder:30b is a 19 GB download with a 256K context window and runs on a 48 GB card:

Qwen publishes official GGUF files for Next in Qwen/Qwen3-Coder-Next-GGUF, where the Q4_K_M files total about 48 GB, and llama.cpp registers both the Qwen3 MoE and Qwen3-Next architectures. Both models are Apache-2.0, so commercial use and fine-tuning need no separate agreement. Stopping a DeployGPU instance deletes its disk, so each launch downloads the weights again, and the stop guide covers what to copy off first. The Qwen page covers the general-purpose models in the same family.