Run WhisperX with speaker diarization on a cloud GPU

The way to get a transcript with speaker labels is one WhisperX command, whisperx audio.mp3 --model large-v2 --diarize --hf_token "$HF_TOKEN", which transcribes with faster-whisper, times every word with a wav2vec2 alignment model and labels the speakers with a pyannote pipeline. Three things have to be right first: a Python between 3.10 and 3.13, the CUDA libraries that PyTorch brings, and a Hugging Face token for a gated pyannote model. Memory is not the limit, because the README says the faster-whisper backend needs under 8 GB for large-v2 with beam size 5 and the smallest GPU DeployGPU rents has 48 GB. The WhisperX, pyannote, PyPI and Hugging Face facts below were checked on 12 November 2026.

Pick a GPU by audio hours per rented hour

The README claims 70x realtime transcription with large-v2 and does not name the GPU, so read it as an upper bound and time your own files. The Whisper guide explains how to time a sample and divide the hourly rate by the audio hours per hour, and it recommends starting on the cheapest 48 GB card. On a VM with 2, 4 or 8 GPUs, run one WhisperX process per GPU and pin each one as the CUDA_VISIBLE_DEVICES page shows.

Install WhisperX on a GPU VM

The CLI reads audio by running the ffmpeg command, which its source says must be installed, and the OpenAI Whisper README gives the Ubuntu command. PyPI lists WhisperX 3.8.6 from 25 May 2026, which needs Python 3.10 up to 3.13 and pins torch 2.8, faster-whisper 1.2.0 or newer and pyannote-audio 4.0.0 or newer. If python3 --version is outside that range, uv can create an environment with Python 3.12, as the PyTorch install guide shows:

sudo apt update && sudo apt install ffmpeg
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv venv ~/whisperx-env --python 3.12
source ~/whisperx-env/bin/activate
uv pip install whisperx

The README asks for the CUDA toolkit 12.8 before WhisperX, and NVIDIA's installation guide has those steps. PyPI's metadata for torch 2.8.0 also lists CUDA 12.8 runtime libraries and cuDNN 9.10.2.21 as dependencies on Linux x86_64, so the pip install brings the libraries, and the driver has to support them. Read the CUDA Version in nvidia-smi first, as the CUDA version guide explains. The repository has no Dockerfile at its root and the README documents no image, so this guide installs WhisperX with uv and not from a container.

Get access to the pyannote speaker diarization model

The CLI's --diarize_model defaults to pyannote/speaker-diarization-community-1 from pyannote.audio, and the README asks you to accept its user agreement and pass a Hugging Face token with the read role. The model page is gated under the CC-BY-4.0 license. It says the repository is publicly accessible but you must accept the conditions to see its files, the form asks for your company or university and your use case, and the authors may email you about pyannote models. Hugging Face's gated models documentation says an access request shares your username and email address with the authors, unless they opted out of collecting it. The pipeline is small, because its model files sum to 32.8 MB.

Create the token at the Hugging Face settings page, and type it on the VM without leaving it in your shell history. Hugging Face's documentation says HF_TOKEN configures the token for its libraries and warns against pasting raw tokens into command lines:

read -rsp "Hugging Face token: " HF_TOKEN; export HF_TOKEN

Revoke the token when the job is done, so that a copy left on the VM or in a log stops working.

Run WhisperX with speaker diarization on one file

whisperx audio.mp3 --model large-v2 --diarize --hf_token "$HF_TOKEN" --output_dir transcripts

This is the README's diarization example with the token and an output folder added and --highlight_words True left out. --model defaults to small, so name the model. If you know the number of speakers, the README adds --min_speakers 2 --max_speakers 2. The first run downloads the Whisper model, which faster-whisper fetches from Hugging Face. The large-v2 model.bin in Systran's repository is 3.09 GB, and the alignment model for the detected language and the pyannote pipeline come on top. The CLI's --model_dir option sets where the model files go and defaults to ~/.cache/whisper, so check df -h before a long run.

Run a folder of audio files

The CLI takes several files in one call and loads each model once, but, as the callout says, it writes nothing until every file has finished. For a long queue, a loop that skips files that already have a JSON result restarts where a stopped run ended:

mkdir -p transcripts
for f in audio/*.mp3; do
  [ -e "transcripts/$(basename "${f%.*}").json" ] && continue
  whisperx "$f" --model large-v2 --diarize --hf_token "$HF_TOKEN" --output_dir transcripts
done

Each call reloads the models, which is the price of getting a result per file. Run the loop in tmux, as keeping jobs running shows, and pull the results with rsync, as the data transfer guide shows. Stopping the instance deletes the disk and the transcripts with it, so copy them off first, and DeployGPU has no idle timer, so the meter runs until you stop it.

Read the output: speaker labels and word timestamps

The CLI writes <audio name>.<extension> into the output folder, and --output_format defaults to all, which means txt, vtt, srt, tsv and json:

The pyannote README shows labels such as SPEAKER_00 and SPEAKER_01. --output_format aud writes an Audacity label file with [[SPEAKER_00]] prefixes, and --highlight_words True underlines each word in the srt and vtt files. A segment that overlaps no speaker turn gets no label. The alignment model depends on the language. The README lists defaults for English, French, German, Spanish and Italian from torchaudio and many other languages from Hugging Face, and for a language with none you must find a phoneme-based model on the Hub. Words such as 2014. or £13.60 have no characters in the alignment dictionary, so they get no timing, and --suppress_numerals is the CLI's option for that. The README also lists the limits that matter for meetings: overlapping speech is not handled well by Whisper or WhisperX, and diarization is far from perfect.

Batch size, compute type and memory

The README's list for saving GPU memory is a smaller --batch_size, such as 4, a smaller --model, such as base, and a lighter --compute_type, such as int8, and it says the last two can affect quality. The CLI default batch size is 8, and the README's Python example uses 16 with a comment to reduce it when GPU memory is low, and sets float16 with a note that int8 may reduce accuracy. On a 48 GB card you will rarely need any of this, because the README's figure is under 8 GB for large-v2.

Fix the cuDNN library errors

The repository's CUDNN_TROUBLESHOOTING.md describes two cuDNN errors on GPU. The first is Unable to load any of {libcudnn_cnn.so.9.1.0, libcudnn_cnn.so.9.1, libcudnn_cnn.so.9, libcudnn_cnn.so}. It means the cuDNN libraries arrived with the dependencies but are not where the dynamic linker looks. The faster-whisper README says LD_LIBRARY_PATH must be set before Python starts and gives this command, which you run in the activated environment before whisperx:

export LD_LIBRARY_PATH=$(python3 -c 'import os, nvidia.cublas, nvidia.cudnn; print(os.path.dirname(nvidia.cublas.__path__[0]) + "/cublas/lib:" + os.path.dirname(nvidia.cudnn.__path__[0]) + "/cudnn/lib")')

The WhisperX file offers a symlink into a directory that is already on the path as a second fix. The second error, RuntimeError: cuDNN version incompatibility: PyTorch was compiled against (9, 10, 2) but found runtime version (9, 2, 1), means another cuDNN comes first on LD_LIBRARY_PATH, while PyTorch brings its own. The file says to print the path with echo $LD_LIBRARY_PATH and remove the conflicting entry. The faster-whisper README states the requirement behind both: cuBLAS for CUDA 12 and cuDNN 9 for CUDA 12. If nvidia-smi itself fails, NVIDIA-SMI has failed comes first.