Run Llama 4 and Llama 3 models on cloud GPUs

The Llama that fits one 48 GB card is Llama 3.1 8B, and the one most people ask about, Llama 3.3 70B, needs two H200s in BF16. Llama 4 Scout is 217.28 GB in BF16, which takes four 80 GB GPUs, and Maverick needs a full 8x H100 VM even in FP8. There is no Llama 5 to size: Meta's newer open model is the separate Muse Glimmer 30B.

Four Llama sizes: 16, 141, 217 and 417 GB

Meta's Llama 4 Scout card, checked 1 October 2026, lists Scout as 17B activated and 109B total with 16 experts and a 10M context, and Maverick as 17B activated and 400B total with 128 experts and a 1M context. Memory follows the total, because every expert has to be resident. Scout ships in BF16 and, in the card's words, "can fit within a single H100 GPU with on-the-fly int4 quantization". Maverick ships in BF16 and FP8, and "The FP8 quantized weights fit on a single H100 DGX host while still maintaining quality."

Sizes are the summed safetensors files on Hugging Face and the tag sizes come from Ollama's library, both checked 1 October 2026. The left-over figures are 92% of the VM's memory minus the checkpoint, so they leave out activations and the engine's own overhead. Maverick in BF16 is 803.17 GB and needs 8x H200. The KV cache explainer shows how to turn what is left into context length.

Llama 3.3 70B needs two H200s, never one

At 141.11 GB, the BF16 weights are more than the roughly 130 GB a single H200 offers to vLLM at its default 92% limit. Use 2x H200 or 4x 80 GB GPUs. The 70B costs 320 KiB of KV cache per token, which is 10 GiB for a 32,000-token conversation (both calculated), so the 118 GB left on 2x H200 holds about eleven such conversations (calculated). NVIDIA's FP8 build is 72.66 GB and fits one H200, and its NVFP4 build is 42.71 GB for Blackwell only. vLLM's recipe lists H100, H200 and B200 as verified and asks for an NVIDIA driver at version 575 or newer, so run nvidia-smi first. The Llama repositories are gated, so pass a Hugging Face token whose account has been approved:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN=$HF_TOKEN \
  vllm/vllm-openai:latest \
  --model meta-llama/Llama-3.3-70B-Instruct \
  --tensor-parallel-size 2 \
  --max-model-len 32768

For the FP8 build on one H200, swap the model for nvidia/Llama-3.3-70B-Instruct-FP8 and add --kv-cache-dtype fp8, which is what the recipe's FP8 variant does, and set the tensor parallel size to 1. Reach the server through the tunnel in the vLLM guide.

Llama 4 on 8 H100s or 4 80 GB GPUs

vLLM's launch post says 8x H100 serves Scout with a 1M-token context and Maverick in FP8 with about 430K. On 8x H200 it reaches 3.6M for Scout and 1M for Maverick, and it says the BF16 models work well on A100. This is its 8x H100 Scout command in Docker. The post dates from April 2025, so if the server rejects a flag, compare it with the current Scout recipe:

sudo docker run -d --name vllm --gpus all --ipc=host \
  -p 127.0.0.1:8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e HF_TOKEN=$HF_TOKEN \
  -e VLLM_DISABLE_COMPILE_CACHE=1 \
  vllm/vllm-openai:latest \
  --model meta-llama/Llama-4-Scout-17B-16E-Instruct \
  --tensor-parallel-size 8 \
  --max-model-len 1000000 \
  --override-generation-config='{"attn_temperature_tuning": true}'

On four 80 GB GPUs, set the tensor parallel size to 4 and lower the context length until the engine's log shows a KV cache that fits. For Maverick in FP8, the post's model is meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 with a tensor parallel size of 8 and a context of 430,000.

Ollama and the 8B model

Ollama lists llama4:scout at 67 GB with a 10M context window and llama4:maverick at 245 GB, both 4-bit. An 80 GB card holds Scout with little room left, and a 96 GB RTX PRO 6000 leaves about 21 GB (calculated). For the 8B, llama3.1:8b is 4.9 GB and the BF16 weights are 16.06 GB, so any 48 GB card runs it either way. vLLM's recipe sizes the BF16 model at 20 GB:

Access, licenses and Llama 5

Every Meta repository above is gated. Accept the license on the model page, wait for approval, then create a read token for HF_TOKEN. The Llama 4 Community License, which the 3.1 and 3.3 agreements resemble, requires a "Built with Llama" display when you distribute or make available the models or a product containing them. A model trained on their outputs and then distributed must start its name with Llama. A service with more than 700 million monthly active users in the month before the release date must ask Meta for a license. Read the text before you build on it.

There is no Llama 5. The newest repository in the meta-llama organization on Hugging Face was created on 28 April 2025, and Llama 4 dates from April 2025 (checked 1 October 2026). Meta's newer open weights sit in a separate meta-models organization, and the one covered here is Muse Glimmer 30B. Stopping a DeployGPU instance deletes its disk, so a 141 GB or 217 GB download repeats on the next launch, and the stop guide covers what to copy off first.