GPU memory for fine-tuning: full, LoRA and QLoRA

The memory a fine-tune needs is set by the method before the model. Full fine-tuning costs about 18 bytes per parameter in Hugging Face's accounting, LoRA costs about the 2 bytes per parameter of its frozen BF16 base plus a margin, and QLoRA costs about half a byte. A 70B model is therefore a 1,260 GB job in full (calculated), a 164 GB job with LoRA by Unsloth's table, and a 41 GB job with QLoRA, which fits one 48 GB card.

Full fine-tuning costs 16 to 18 bytes per parameter

Hugging Face's memory guide, checked 1 October 2026, counts 6 bytes per parameter for the weights in mixed precision, an fp16 copy and an fp32 main copy. It adds 8 bytes for Adam's momentum and variance in fp32, and 4 bytes for gradients kept in fp32. That is 18 bytes before activations, and the guide's own example is a 4B model at batch size 16 that needs roughly 85 GB, against 72 GB for those 18-byte states (calculated). The ZeRO paper counts 16 bytes for mixed-precision Adam: an fp16 copy of the parameters and of the gradients at 2 bytes each, plus fp32 parameters, momentum and variance at 4 bytes each. The 2-byte gap is the gradients. ZeRO stores them in fp16 and Hugging Face keeps them in fp32, so both are right for the setup each describes. I plan with 18 and treat 16 as the best case.

The table is the total for a perfectly sharded run with FSDP or ZeRO-3, calculated and before activations. The multi-GPU training guide covers the sharding itself.

The last row is the useful one. A full fine-tune of a 70B model does not fit the largest H200 VM at 18 bytes, and at 16 it leaves no room for activations. That job goes to 8x B200 or larger, or to the cluster quote form. Activation checkpointing helps with the rest: the ZeRO paper puts its cost at about 33% extra computation, and Hugging Face has a gradient checkpointing guide.

LoRA and QLoRA: what the papers claim

The LoRA paper says: "Compared to GPT-3 175B fine-tuned with Adam, LoRA can reduce the number of trainable parameters by 10,000 times and the GPU memory requirement by 3 times." Memory falls by 3 times and not by 10,000 because the frozen base model still has to sit on the GPU. In BF16 that base costs 2 bytes per parameter, 140 GB for a 70B model (calculated), before the adapters, their optimizer state and the activations.

The QLoRA paper begins: "We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance." It lists three ideas behind that: a 4-bit data type called NormalFloat (NF4), double quantization of the quantization constants, and paged optimizers for memory spikes. The base at 4 bits costs about half a byte per parameter, which is 35 GB for a 70B model (calculated). The quantization guide covers the formats.

Unsloth's table, mapped to DeployGPU cards

Unsloth's requirements page, checked 1 October 2026, lists VRAM by model size and says the figures are the absolute minimum, with a batch size of 1 to 3 as the first fix for running out of memory. The card column compares its figure with each card's memory directly, calculated, with the spare memory beside it.

LLaMA-Factory's README gives an estimated table of its own: 16 GB for LoRA on a 7B model, 160 GB for LoRA on a 70B model and 48 GB for 4-bit QLoRA on a 70B model, plus a rule of 18 times the parameter count in GB for full fine-tuning in bf16 or fp16. Against Unsloth's 164 GB and 41 GB for the 70B, the two sources agree to within about 17%.

Why Unsloth, Hugging Face and LLaMA-Factory differ

Bytes per parameter differ with the gradient dtype, as above. Activations scale with batch size, sequence length, depth and hidden size, which is why Hugging Face says a model that fits can still run out of memory on a long sequence. Unsloth calls its figures the absolute minimum and LLaMA-Factory marks its table as estimated, so my rule is to read the spare memory in the table above as a budget for activations and nothing else. If the spare memory is under 10% of the card, go up one tier, as in the 80 GB tier, or fix the batch size first.

The fine-tuning guide picks a framework and the Unsloth guide runs one. For jobs that need 2 or more GPUs in one VM, see the multi-GPU page. If a run still fails, the CUDA out of memory page lists fixes. Stopping a DeployGPU instance deletes its disk, so save adapters and checkpoints off the VM first, as the stop guide explains.