Run Qwen models on a cloud GPU

The Qwen family spans a 9 billion parameter model that fits any DeployGPU card to a 397 billion parameter mixture-of-experts model that needs an eight-GPU node, and the size that matters is the checkpoint on disk, not the headline parameter count. Qwen3.8-27B is the one most people mean when they say Qwen right now, and it fits one 80 GB GPU in its native BF16 format, or one 48 GB card at FP8.

The 27B dense model, and Qwen3.6 being the same size

Qwen3.8-27B is a 27 billion parameter dense model, 64 layers built from 16 repeated blocks of three Gated DeltaNet layers followed by one full-attention layer, according to the model card. Its checkpoint sums to 55.6 GB in BF16, calculated from the safetensors files. vLLM's own recipe for it asks for 67 GB on a single GPU in BF16 to leave room to run, 38 GB for the official FP8 build, and as little as 24 GB for a community INT4 build, all on one GPU. Native context is 262,144 tokens, extensible toward 1,000,000 with YaRN scaling, and the license is Apache-2.0.

Qwen3.6-27B, released in April 2026, a few months before Qwen3.8-27B, uses the identical architecture and an identical 55.6 GB checkpoint. My default for a new deployment is Qwen3.8-27B, since it replaced Qwen3.6-27B with no size or hardware trade-off attached to picking the newer one.

The 35B mixture-of-experts model

Qwen3.6-35B-A3B activates only 3 billion of its 35 billion parameters per token, which is why it runs almost as fast as a much smaller model, but memory follows the total, not the active count, because every expert has to be resident on the GPU whether a given token uses it or not. Its BF16 checkpoint is 71.9 GB, and vLLM's recipe asks for 84 GB in practice, which is where an 80 GB card runs out of room and a 96 GB card or a two-GPU BF16 setup takes over. The official FP8 build drops that to 42 GB, which fits a 48 GB card with little room to spare, and an NVFP4 build exists for Blackwell GPUs at 21 GB.

The small one: Qwen3.5-9B

Qwen3.5-9B is a 9 billion parameter dense model with the same DeltaNet-plus-attention design, a 19.3 GB BF16 checkpoint, and a vLLM recipe that asks for 22 GB, well inside a 48 GB card with plenty left over for context. It is the practical choice for a chat or coding assistant on the smallest card DeployGPU launches as a VM, rather than reaching for a 27B model that does not need to be that large.

Serve it with vLLM or Ollama

For an API, launch a GPU VM and run vLLM in Docker, binding the port to localhost so the machine's open inbound access does not turn it into a public, unauthenticated endpoint:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN=$HF_TOKEN \
  vllm/vllm-openai:latest \
  --model Qwen/Qwen3.8-27B \
  --max-model-len 65536

Swap the model for Qwen/Qwen3.8-27B-FP8 on a 48 GB card, or Qwen/Qwen3.6-35B-A3B on a larger one. vLLM 0.17.0 or newer carries the architecture for the whole Qwen3.5 through 3.8 line. The vLLM guide covers the tunnel, the --api-key flag and tensor parallelism in full.

For chat, Ollama's library lists qwen3.8:27b at 18 GB in its default 4-bit quantization, qwen3.8:27b-q8_0 at 30 GB, and qwen3.6:35b at 23 GB, checked 28 September 2026:

How the hybrid architecture changes the KV cache

Only the full-attention layers grow a KV cache with context length. The 48 Gated DeltaNet layers in Qwen3.8-27B keep a fixed-size recurrent state per sequence instead, the kind of layer vLLM's hybrid KV-cache manager treats separately from ordinary attention. That is why Qwen3.8-27B costs just 64 KiB per token, 2 GiB for a 32,000-token conversation, calculated from its 4 KV heads and 256-dimension head size across 16 growing layers, far less than a plain transformer of the same size would need. Qwen3.5-9B costs 32 KiB per token, and Qwen3.6-35B-A3B costs 20 KiB per token, both for the same reason. The KV cache explainer has the formula and more worked examples.

When the model needs more than one GPU

Two members of the family go past what one DeployGPU offer holds. Qwen3.5-122B-A10B, a 10-billion-active mixture-of-experts model, has a 250.2 GB BF16 checkpoint that needs four H200s by vLLM's own recipe, though an official NVFP4 or GPTQ-Int4 build at 74 GB fits one 80 GB GPU. Qwen3.5-397B-A17B goes further still: 806.8 GB in BF16, which the recipe states plainly needs eight H200s, or 477 GB at FP8 on a smaller multi-GPU node. Both need a multi-GPU VM, and the largest VM DeployGPU launches has 8 GPUs, so check the H200 page for 4x and 8x stock. The multi-GPU page covers how the GPUs inside one node talk to each other. Anything past 8 GPUs goes through the cluster quote form.

Stopping and the next run

Stopping a DeployGPU instance deletes its disk, so any checkpoint sitting in the Hugging Face cache or pulled through Ollama disappears with it, and the next launch downloads it again. A 55.6 GB BF16 checkpoint takes longer to come back than an 18 GB Ollama tag, so weigh that against the provider's network speed before picking a format for a session you expect to repeat. The guide to stopping without losing work covers what to copy off first, and pricing covers what the hour itself costs.

License

Every model on this page, from Qwen3.5-9B through the 397B mixture-of-experts model, ships under Apache-2.0, so commercial use and fine-tuning need no separate agreement. Some newer Qwen releases, such as Qwen3.8-Flash-Next, use a different license, so check the model card before building on one that is not covered here.