Install PyTorch with CUDA on a cloud GPU VM

The PyTorch build that works on a rented GPU is decided by one number: the CUDA Version in the header of nvidia-smi. It is the newest CUDA the driver supports, and the wheel you install must not be newer, which is why a plain pip install torch can succeed on a VM and still leave torch.cuda.is_available() returning False. Provider images ship different driver branches, so I read the number off the machine and pick the wheel from it.

Read the driver's CUDA ceiling first

nvidia-smi
python3 --version

Write down the GPU name and the CUDA Version. NVIDIA describes that version as the latest CUDA supported by the driver, so it says nothing about an installed toolkit, and you do not need one. A PyTorch forum answer from December 2023 says the binaries ship with all CUDA runtime dependencies and only a properly installed NVIDIA driver is needed. You can see this when you install: on the CUDA 13 builds pip fetches nvidia-cudnn-cu13, nvidia-nccl-cu13 and other CUDA packages next to torch. The first five minutes guide shows how to read the rest of the output.

PyTorch's release table, checked 1 October 2026, lists Python 3.10 or newer for the current 2.14 release. If python3 --version prints something older, the uv route below downloads a newer Python for you.

Pick the wheel index that fits the driver

PyTorch publishes one wheel index per CUDA version at https://download.pytorch.org/whl/INDEX. Choose the newest index that is not newer than your CUDA Version. These are the indexes that matter today, with the newest torch release on each, read from the PyTorch index on 1 October 2026:

A newer wheel can still run on an older driver through NVIDIA's minor version compatibility, which allows CUDA 12.x builds on driver 525 or newer and CUDA 13.x builds on driver 580 or newer. NVIDIA says that comes with a limited feature set, so I do not rely on it.

Blackwell is the row that trips people. The B200 has compute capability 10.0 and the RTX PRO 6000 Blackwell has 12.0, according to NVIDIA's CUDA GPU list. PyTorch added Blackwell in 2.7 with wheels for CUDA 12.8, per the release notes, and the 2.14 table lists the CUDA 12.6 build as covering Maxwell through Hopper (9.0) and the 13.0 and 13.2 builds as adding Blackwell (10.0 and 12.0). So a cu126 wheel cannot run on a B200 or an RTX PRO 6000. The RTX A6000, L40, L40S, RTX 6000 Ada, A100, H100 and H200 are covered by every build in the table.

One default catches people out. PyPI's metadata for torch 2.14.1 lists CUDA 13.0 libraries for Linux, so a plain pip install torch installs a CUDA 13.0 build today. On a VM whose driver tops out at CUDA 12.x it installs without complaint and then cannot use the GPU.

Create an environment on the disk with room

uv picks the index for you. Its PyTorch guide says --torch-backend=auto queries the installed CUDA driver and uses the most compatible PyTorch index, and that the flag exists only in the uv pip interface:

curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv venv ~/venv --python 3.12
source ~/venv/bin/activate
uv pip install torch torchvision --torch-backend=auto

To choose the backend yourself, replace auto with the index name from the table, for example --torch-backend=cu130. With plain pip, PyTorch's own instructions note that pip is not installed by default on Linux, so on Ubuntu you may need the first line:

sudo apt-get install -y python3-venv python3-pip
python3 -m venv ~/venv
source ~/venv/bin/activate
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130

The download is large. For torch 2.14.1 on the cu130 index, the torch wheel is 0.55 GB and the NVIDIA libraries and Triton it depends on add 2.45 GB, about 3 GB in total, calculated from the wheel sizes that the PyTorch index and PyPI list for Python 3.12 on Linux x86_64. That leaves out the unpacked size and the pip cache. On QuantaCloud, Massed Compute and Hyperstack the disk is fixed by the offer, so run df -h first, and use pip cache purge or uv cache clean if the install fills it. The environment is deleted when the instance stops, so keep the commands above in a script that you copy off the machine, and the next launch is a rerun.

Verify that CUDA works and that the build covers your GPU

import torch

print("torch", torch.__version__, "built for CUDA", torch.version.cuda)
print("available:", torch.cuda.is_available())
print(torch.cuda.get_device_name(0), torch.cuda.get_device_capability(0))
print("build covers:", torch.cuda.get_arch_list())

x = torch.randn(4096, 4096, device="cuda")
y = x @ x
torch.cuda.synchronize()
print("matmul ran on", y.device)

The get_arch_list() function returns the CUDA architectures the build was compiled for, and it should include your GPU's generation, such as sm_100 for a B200 or sm_120 for an RTX PRO 6000. The multiplication matters more than the True: a build that does not cover the GPU can report available and still fail at the first kernel. If torch.version.cuda prints None you have a CPU build, and Torch not compiled with CUDA enabled is the fix.

Four failures after the install

PyTorch prints the first message from its C++ CUDA code and the second from torch/cuda/__init__.py when no compiled architecture matches the GPU. The kernel image text is CUDA's own error string. If nvidia-smi itself fails with a driver and library version mismatch, that is not a PyTorch problem, and the NVML mismatch fix covers it.

Skip the install with the Jupyter template or a container

The Jupyter template starts from a PyTorch build for CUDA 12 that is already installed, and its page explains the Blackwell check. For a container, pick an image tag at or below the CUDA Version you wrote down, because a tag newer than the driver supports will not start, and the NVIDIA Container Toolkit guide covers the GPU flags. The Blackwell page for the RTX PRO 6000 lists the live offers.