Run GLM-5.3 on cloud GPUs

The two big GLM-5.3 models do not fit one GPU in their official formats. GLM-5.3-Flash is 328.34 GB in FP8, which an 8x H100 VM or a 4x H200 VM holds, and GLM-5.3 is 755.63 GB, which needs 8x H200. The small GLM-4.7-Flash is 62.44 GB and fits one 80 GB GPU, which is why it is the one to try first.

Three GLM sizes: 62, 328 and 756 GB

Z.ai's GLM-5 repository lists GLM-5.3 at 744 billion parameters with 40 billion active, stored in FP8, and GLM-5.3-Flash at 320 billion with 18 billion active, also FP8, and vLLM's recipes list a 1M-token context for both. The GLM-4.7-Flash card calls the small one a 30B-A3B mixture-of-experts model. Checkpoint sizes below are the summed safetensors files, checked 1 October 2026.

The left-over figures are 92% of the VM's memory minus the checkpoint. They leave out activations and CUDA graphs, so the engine's startup log has the real KV cache size. The cache is small in both big models. GLM-5.3 costs 87.8 KiB per token before its indexer's keys and GLM-5.3-Flash costs 11 KiB, both calculated from their configs, and the KV cache explainer has the method. Flash is Hopper or newer only, so an 8x A100 VM is out even though its 640 GB would hold the weights.

GLM-5.3-Flash on 8 H100s

SGLang's cookbook has recipes for H100, H200, B200, B300, GB200 and GB300, and it needs SGLang 0.5.20 or later. On H100 and H200 its default is a BF16 KV cache with the TileLang attention backend, because FP8 KV with the TRT-LLM backend is disabled on Hopper, so the plain command already picks the right settings. Wrapped in Docker, with the port published on loopback only and the --host 0.0.0.0 staying inside the container:

sudo docker run -d --name sglang --gpus all --shm-size 32g --ipc=host \
  -p 127.0.0.1:30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  lmsysorg/sglang:latest \
  sglang serve \
  --model-path zai-org/GLM-5.3-Flash \
  --tp-size 8 \
  --host 0.0.0.0 \
  --port 30000

The cookbook's configurator generates tuned versions, such as its high-throughput profile, for each GPU. Run nvidia-smi first to check the driver. vLLM's recipe supports Hopper and newer, points NVIDIA users to the vllm/vllm-openai:glm53-flash image and requires a BF16 KV cache on Hopper. Its static example is TP 4 with FP8 KV on a GB200 tray, which does not carry over to Hopper.

GLM-5.3 on 8 H200s

This one is the vLLM recipe's own command for 8x H200, wrapped the same way. The recipe asks for vLLM 0.28.0 or newer and transformers 5.15.0 or newer, so check the versions inside the container:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  --model zai-org/GLM-5.3 \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 5 \
  --tool-call-parser glm47 \
  --reasoning-parser glm47 \
  --enable-auto-tool-choice \
  --served-model-name glm-5.3

SGLang's GLM-5.3 cookbook also runs it on 8x H200 at TP 8. For the full 1M-token window, the vLLM recipe wants 8x B200 with --max-num-seqs 32 as the starting point. Pick an 8x H200 offer whose Interconnect column says NVLink (SXM), as the H200 page explains, and reach the server through the SSH tunnel in the vLLM guide. Both big models think at full depth by default. The GLM-5 repository says reasoning_effort accepts low, high or max and defaults to max, so pass a lower level when a short answer is enough.

Quantized builds that shrink the node

Blackwell has 4-bit builds. RedHatAI's NVFP4 build of Flash is 197.84 GB, which 2x B200 holds with about 133 GB left, and Inferact's NVFP4 build of GLM-5.3 is 464.82 GB, which 4x B200 holds with about 198 GB left, both calculated. Hopper cannot run those kernels. The quantization guide says which formats run on which GPU generation.

llama.cpp merged GLM-5.3-Flash support on 30 September 2026 in pull request 27773, so recent builds carry it. Unsloth's GLM-5.3-Flash guide lists GGUF files from 93.09 GB at 1-bit to 199.71 GB at 4-bit, which puts a 2-bit build at 108.72 GB inside a single H200. That guide was written before the merge and builds its own llama.cpp branch, so follow its page on the day you read it and the llama.cpp server guide for the engine itself. Ollama lists GLM-5.3 and GLM-5.3-Flash as cloud models only.

GLM-4.7-Flash for one 80 GB GPU

GLM-4.7-Flash is the size most people can actually try. The card's vLLM example uses 4 GPUs, but the model needs one, so this drops the tensor parallel flag:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  --model zai-org/GLM-4.7-Flash \
  --speculative-config.method mtp \
  --speculative-config.num_speculative_tokens 1 \
  --tool-call-parser glm47 \
  --reasoning-parser glm45 \
  --enable-auto-tool-choice \
  --served-model-name glm-4.7-flash

vLLM's supported models page lists the architecture, and SGLang's cookbook uses --tp 1. For chat, Ollama lists glm-4.7-flash at 19 GB in its default 4-bit tag, glm-4.7-flash:q8_0 at 32 GB and glm-4.7-flash:bf16 at 60 GB, checked 1 October 2026. A 48 GB card holds the first two:

Licenses and what a stop deletes

GLM-5.3-Flash and GLM-4.7-Flash are MIT licensed. GLM-5.3 has its own license, which allows use and fine-tuning and asks Model as a Service businesses above 10 billion US dollars in revenue to pass a Z.AI security review. Stopping a DeployGPU instance deletes its disk, so each session downloads 756 GB for GLM-5.3 or 328 GB for Flash again, and the stop guide shows what to copy off first. The frontier models page compares GLM-5.3 with the other models that need a full 8-GPU VM.