How to check your CUDA version

The way to check CUDA version on a Linux GPU server is to run three commands, because three different numbers share that name. nvidia-smi prints the newest CUDA the installed driver supports, nvcc --version prints the toolkit installed on the machine if there is one, and torch.version.cuda prints the CUDA runtime that your PyTorch build was compiled with. On a rented VM they often disagree, and only one of them decides the job in front of you. The NVIDIA and PyTorch facts below were checked on 12 November 2026.

The three numbers and what each one decides

NVIDIA's own documentation shows how they come apart. In its minor version compatibility guide, a driver numbered 580.65.06 prints CUDA Version: 13.0 in the nvidia-smi header, while deviceQuery on the same machine reports CUDA Driver Version / Runtime Version 13.0 / 13.4. Both numbers are right. One is the ceiling of the driver and the other is the runtime of the program that was run. I write all three numbers down before I install anything.

Check CUDA version with nvidia-smi

nvidia-smi
nvidia-smi --query-gpu=name,driver_version --format=csv

NVIDIA's nvidia-smi manual calls the number in the header the CUDA UMD Version and defines it as the latest CUDA version supported by the driver. It comes from the driver, so it is there even when no toolkit is installed. If nvidia-smi itself fails, NVIDIA-SMI has failed and the NVML mismatch fix come first, and NVIDIA drivers on Ubuntu covers installing or updating the driver.

The driver depends on the image. The deploy page pre-selects an OS image with CUDA and Docker, newest CUDA first, and an API call without os_image_id gets the provider's default image, which can be a different one. So I read the number off the machine. If you only know the driver number, NVIDIA's CUDA release notes map it to a CUDA release, where R580 means a driver numbered 580.x. A driver between two rows supports the lower one.

Check the installed toolkit with nvcc --version

nvcc --version
ls -d /usr/local/cuda*

NVIDIA's nvcc manual defines --version (-V) as printing version information on the tool, and the compiler ships with the toolkit, so the release it reports is the toolkit's. The installation guide puts a toolkit in a directory such as /usr/local/cuda-13.5 and points the /usr/local/cuda symbolic link at the one that was installed, so the ls line lists the toolkits in the default location.

Why nvcc: command not found appears when nvidia-smi works

bash: nvcc: command not found

There are three reasons. The first is that the driver and the toolkit are separate installs: NVIDIA's guide says that from CUDA 13.4 they are versioned and installed independently, and that the cuda-toolkit packages install no driver. DeployGPU documents the plain GPU VM as the provider's Ubuntu image with NVIDIA drivers and Docker, and a toolkit is not part of that description. The second is a toolkit that is installed with its bin directory missing from your PATH, which NVIDIA's FAQ gives as the cause of an nvcc: No such file or directory error:

export PATH=/usr/local/cuda-13.5/bin${PATH:+:${PATH}}

Use the directory that ls printed, not 13.5 if yours differs. The third reason is that you may not need a toolkit at all. PyPI's metadata for torch 2.15.0 lists its CUDA libraries, from cuda-toolkit 13.2.2 and nvidia-cudnn-cu13, as pip dependencies, and the official vLLM and llama.cpp images carry their own runtime. A toolkit matters when you compile: a custom extension, flash-attn from source, or llama.cpp, whose build guide says to have the CUDA toolkit installed for -DGGML_CUDA=ON. For that case NVIDIA's guide says to add NVIDIA's apt repository with the cuda-keyring package and install cuda-toolkit-X-Y, which locks the release. I pick an X-Y at or below the number nvidia-smi printed.

Check the PyTorch CUDA version

python3 -c "import torch; print(torch.__version__, torch.version.cuda)"

PyTorch writes torch.version.cuda into the build. Its version generator records the CUDA version the build was made with and stores None when there is none, so the value says nothing about the driver or about any toolkit on the disk. A CPU build prints None, and Torch not compiled with CUDA enabled is the fix.

PyTorch CUDA compatibility comes down to one question: can the driver run the CUDA family of the build? This is what exists today, read from the PyTorch wheel index for Python 3.12 on Linux x86_64 on 12 November 2026:

The driver column is NVIDIA's minor version compatibility table: CUDA 13.x needs 580 or newer, 12.x needs 525 or newer and 11.x needs 450 or newer, with a limited feature set. PyTorch's get started page lists CUDA 13.2 and 13.4 for 2.15.0 on Linux, and a plain pip install torch gets the 13.2 build. On a VM whose header shows CUDA 12.x, that wheel installs and then cannot use the GPU, so install an older release from the index that matches:

pip install "torch==2.14.1" --index-url https://download.pytorch.org/whl/cu126

NVIDIA's CUDA Compatibility guide is the reference for all of this. The PyTorch install guide covers the whole flow, and no kernel image is available covers a build that predates the GPU.

Check the cuDNN version

python3 -c "import torch; print(torch.backends.cudnn.version())"
dpkg -l 'libcudnn*'
python3 -m pip show nvidia-cudnn-cu13

PyTorch documents torch.backends.cudnn.version() as returning the version of cuDNN, and its source returns None when cuDNN is not there. The number is an integer. NVIDIA's cuDNN documentation defines the version macro as CUDNN_MAJOR * 10000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL, so 92600 means 9.26.0. PyPI's metadata for torch 2.15.0 pins nvidia-cudnn-cu13==9.26.0.51, so pip show should print that version and the integer should be 92600 for that wheel (calculated). A system-wide cuDNN comes as Debian packages named libcudnn9-cuda-13 or libcudnn9-cuda-12, which dpkg -l lists. PyTorch's own error text for a cuDNN mismatch says it already comes bundled with cuDNN and warns about a conflicting copy on LD_LIBRARY_PATH, so the integer torch prints describes the copy it loaded.

Check the CUDA version inside a Docker container

sudo docker run --rm --gpus all ubuntu nvidia-smi
sudo docker exec CONTAINER nvidia-smi
sudo docker exec CONTAINER nvcc --version
sudo docker exec CONTAINER python3 -c "import torch; print(torch.version.cuda)"

NVIDIA's Container Toolkit documentation says its NVIDIA_DRIVER_CAPABILITIES variable controls which driver libraries and binaries are mounted inside the container, so the first two commands print the host driver's ceiling. The last two report what is inside the image, and an image without a toolkit or without PyTorch will say so. The official nvidia/cuda images on Docker Hub come in three flavors: base has the CUDA runtime, runtime adds the math libraries and NCCL, and devel adds headers and development tools. The NVIDIA Container Toolkit guide shows the tag rule and the error a too-new tag gives.

The ceiling matters for images that move fast. The vLLM 0.33.0 release notes say its release wheels and images are CUDA 13.0 only, so a VM whose header shows CUDA 12.x needs an older vLLM release.

Which number matters for which job

NVIDIA's forward compatibility package can run a newer toolkit on an older driver across major versions, but NVIDIA says it applies only to Data Center GPUs, select NGC Server Ready SKUs of RTX cards and Jetson boards. I do not plan around it for a rented card.