Fine-tune an LLM with Unsloth on a cloud GPU

The shortest path to a first LLM fine-tune on a rented GPU is QLoRA with Unsloth on a 48 GB card: a 4-bit base model, a small set of trained adapter weights and a short Python script. Everything happens on the instance disk, which is deleted when the instance stops, so the last step of this guide matters as much as the first: get the adapter off the machine before you stop it.

Pick the GPU: Unsloth's VRAM minimums against 48, 80 and 96 GB cards

Unsloth's requirements page, checked 1 October 2026, lists minimum VRAM by model size and says to treat the numbers as the absolute minimum. The last column compares them with the cards DeployGPU launches. It is calculated, and it ignores the extra memory that long context and bigger batches need.

Unsloth's page for Qwen3.8-27B says QLoRA works with 24 GB and LoRA needs more than 36 GB, which agrees with the QLoRA row. My rule for a first fine-tune is QLoRA on the smallest card that fits. Unsloth's own fine-tuning guide recommends it and says its dynamic 4-bit quants recover most of the accuracy loss against LoRA. The smallest VM class has 48 GB, and how much VRAM to fine-tune covers the memory math behind the table.

Launch a GPU VM and check the driver

Launch a plain GPU VM, which the dashboard labels Bare Metal, and connect with the command shown on the deployment. The login user depends on the provider:

Run nvidia-smi and df -h, as the first five minutes guide explains. For a 27B model, one disk holds the 4-bit base model, which is 22.4 GB on the Hub according to hf download unsloth/Qwen3.8-27B-unsloth-bnb-4bit --dry-run on 1 October 2026, plus the Python environment, your dataset and the outputs. On Blackwell GPUs, PyTorch needs a CUDA 12.8 or newer build, as the PyTorch install guide explains.

Install Unsloth with uv

These are the commands from Unsloth's install page, which uses Python 3.13 in its uv example:

curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install unsloth --torch-backend=auto

The --torch-backend=auto option picks the PyTorch build that fits the driver. If a later step complains about a missing compiler, sudo apt-get install -y build-essential installs one. Unsloth also ships a web UI called Studio. Its docs launch it with unsloth studio -H 0.0.0.0 -p 8888, which listens on every interface of a VM that DeployGPU does not firewall. Pass -H 127.0.0.1 instead and reach it through an SSH tunnel.

Train: the SFT recipe from Unsloth's Qwen3.8 page

This is Unsloth's minimal SFT recipe for Qwen3.8-27B with two additions: a line that renders chat messages into the text column the recipe expects, and two lines that save the adapter. Save it as train.py:

from unsloth import FastModel
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig

max_seq_length = 2048

model, tokenizer = FastModel.from_pretrained(
    model_name = "unsloth/Qwen3.8-27B-unsloth-bnb-4bit",
    max_seq_length = max_seq_length,
    load_in_4bit = True,
    full_finetuning = False,
    offload_embedding = True,
)

model = FastModel.get_peft_model(
    model,
    finetune_vision_layers     = False,
    finetune_language_layers   = True,
    finetune_attention_modules = True,
    finetune_mlp_modules       = True,
    r = 16,
    lora_alpha = 16,
    lora_dropout = 0,
    bias = "none",
    use_gradient_checkpointing = "unsloth",
    random_state = 3407,
    use_rslora = False,
    loftq_config = None,
)

# Each line of train.jsonl is {"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
dataset = load_dataset("json", data_files = "train.jsonl", split = "train")
dataset = dataset.map(lambda ex: {"text": tokenizer.apply_chat_template(ex["messages"], tokenize = False)})
print(dataset[0]["text"])

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    args = SFTConfig(
        dataset_text_field = "text",
        max_seq_length = max_seq_length,
        per_device_train_batch_size = 1,
        gradient_accumulation_steps = 4,
        warmup_steps = 10,
        max_steps = 100,
        learning_rate = 2e-4,
        logging_steps = 1,
        optim = "adamw_8bit",
        output_dir = "outputs",
        seed = 3407,
        dataset_num_proc = 1,
        report_to = "none",
    ),
)
trainer.train()

model.save_pretrained("finetuned_lora")
tokenizer.save_pretrained("finetuned_lora")

Run it inside tmux, as keeping jobs running after you disconnect shows: python -u train.py 2>&1 | tee train.log. Read the printed example before the run starts. A chat template can add text of its own, such as a default system message or an empty think block, and whatever it adds becomes part of your training data.

max_steps = 100 is a smoke test. Unsloth's guide says that for a full run you replace it with num_train_epochs = 1, that 1 to 3 epochs avoid overfitting, and that 2e-4 is the default learning rate to lower for more precise tuning. The page also warns that the first run takes longer while the Gated DeltaNet kernels compile, and that offload_embedding keeps a large embedding in system RAM. If you run out of memory, cut max_seq_length and keep the batch size at 1.

Save the adapter before you stop

The adapter is the small part. Unsloth's guide describes it as a file of about 100 MB for its example, and the size depends on the model and the rank. Upload it to a private Hub repository with a token that can write to it, or pull it to your own machine:

hf upload your-name/qwen38-lora ./finetuned_lora . --private
rsync -avP -e "ssh -p 22" ubuntu@YOUR_VM_IP:~/finetuned_lora/ ./finetuned_lora/

For a run longer than one sitting, set save_steps and save_total_limit in SFTConfig and copy the checkpoints as they appear, as checkpoint and resume shows. A merged 16-bit save is a full copy of the model, so it is about the size of the 55.6 GB BF16 checkpoint the Hub lists for Qwen3.8-27B, calculated from the same parameter count. Check df -h before you ask for one.

Serve the result with vLLM

Unsloth's vLLM page merges the adapter into a 16-bit model and serves the folder:

model.save_pretrained_merged("finetuned_model", tokenizer, save_method = "merged_16bit")

vllm serve finetuned_model

On DeployGPU, run vLLM the way the vLLM guide does, in Docker bound to 127.0.0.1 and reached through a tunnel. A 27B model in BF16 wants an 80 GB class card, per the Qwen page. Unsloth also documents loading adapters into vLLM without merging.

Six things that go wrong

Unsloth's multi-GPU page says it works through Accelerate and DeepSpeed, for example torchrun --nproc_per_node N_GPUS train.py, so the multi-GPU guide applies to a VM with up to 8 GPUs. DeployGPU has no multi-node.