Transcribe audio with Whisper on a cloud GPU

The GPU question for Whisper is not how much memory you need, because the smallest GPU DeployGPU rents has 48 GB and Whisper large-v3 needs a small fraction of that. The question is how many hours of audio each rented hour gets through, and the only way to know is to time your own files. This guide uses faster-whisper on a plain GPU VM, gives a short script that does the timing, and covers moving audio in and transcripts out before you stop the instance.

Pick large-v3 or large-v3-turbo

The large-v3 model card says the model has the same architecture as large-v2 apart from 128 Mel bins and a new Cantonese language token, and shows 10% to 20% fewer errors than large-v2. The large-v3-turbo card describes it as the same model with the decoding layers cut from 32 to 4, so it is much faster at the cost of a minor quality drop. OpenAI's README adds that turbo was not trained for translation, so translating speech into English needs one of the multilingual models, such as medium or large, instead. All figures below were checked on 1 October 2026:

In faster-whisper you ask for them by the names large-v3 and turbo, and the library downloads the converted model from Hugging Face the first time, as its README describes. The download sizes are the model.bin files in the Systran and mobiuslabsgmbh repositories that those names point to.

Choose the GPU by audio hours per rented hour

Memory is not the limit. faster-whisper's README measured large-v2 in fp16 at 4,525 MB with no batching and 6,090 MB at batch_size=8, on an RTX 3070 Ti with 8 GB. That card is not one DeployGPU rents, and large-v3 is the same size class, so every card DeployGPU offers has far more room than the model needs. The README's timing shows what batching can do, not what to expect from your GPU: 13 minutes of audio took 1 minute 3 seconds without batching and 17 seconds with batch_size=8 on that card, with CUDA 12.4 and the README at faster-whisper 1.1.0.

So compare cost per audio hour. If the script below prints R audio seconds per second, one rented hour covers R hours of audio, and the cost per audio hour is the hourly rate divided by R. A faster card only pays when its R grows by more than its price grows. Start on the cheapest 48 GB card, such as the RTX A6000, time a sample of your own files, and move up only if the numbers say so. The README does not say which GPU generations CTranslate2 supports, so I leave the Blackwell cards out of this guide and stay with the 48 GB class. Prices and stock move by the minute, so read them in the live table:

On a VM with 2, 4 or 8 GPUs, run one worker per GPU and split the files between them. The CUDA_VISIBLE_DEVICES page shows how to pin each worker to a card.

Install faster-whisper on a plain GPU VM

The README lists the requirements: Python 3.9 or newer, no separate FFmpeg because audio decoding uses the PyAV package, and for the GPU the NVIDIA libraries cuBLAS for CUDA 12 and cuDNN 9 for CUDA 12. Its pip route installs those libraries and puts them on the library path. I use a virtual environment, because a distribution's Python can be marked as externally managed, in which case pip refuses a system-wide install, as PEP 668 describes. If python3 -m venv is missing, Ubuntu's python3-venv package provides it.

python3 -m venv ~/whisper-env
source ~/whisper-env/bin/activate
pip install faster-whisper nvidia-cublas-cu12 "nvidia-cudnn-cu12==9.*"
export LD_LIBRARY_PATH=`python3 -c 'import os; import nvidia.cublas.lib; import nvidia.cudnn.lib; print(os.path.dirname(nvidia.cublas.lib.__file__) + ":" + os.path.dirname(nvidia.cudnn.lib.__file__))'`

The README says LD_LIBRARY_PATH must be set before Python starts, so repeat the export in every new shell. It also names an NVIDIA image that already contains both libraries, nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04. If you go that way, the toolkit guide covers running GPU containers and picking a tag your driver supports.

Transcribe a folder and time it

This script is built from the README's batched example. BatchedInferencePipeline.transcribe is a drop-in replacement for WhisperModel.transcribe, and batched mode has the voice activity filter on by default. The segments come back as a generator, which the README warns does no work until you iterate it, so the script joins them. Loading the model, and downloading it on the first run, happens before the timer starts:

import sys
import time
from pathlib import Path

from faster_whisper import BatchedInferencePipeline, WhisperModel

source = Path(sys.argv[1])
target = Path(sys.argv[2])
target.mkdir(parents=True, exist_ok=True)

model = WhisperModel("large-v3", device="cuda", compute_type="float16")
pipeline = BatchedInferencePipeline(model=model)

audio_seconds = 0.0
start = time.perf_counter()
for path in sorted(source.glob("*.mp3")):
    segments, info = pipeline.transcribe(str(path), batch_size=16)
    text = "\n".join(segment.text.strip() for segment in segments)
    (target / f"{path.stem}.txt").write_text(text, encoding="utf-8")
    audio_seconds += info.duration
elapsed = time.perf_counter() - start

print(f"{audio_seconds / 3600:.2f} audio hours in {elapsed / 60:.1f} minutes")
print(f"{audio_seconds / elapsed:.1f} audio seconds per second")

Run it with python3 transcribe.py ~/audio ~/out, then change the file pattern for other formats. The second printed line is R from the formula above. Swap "large-v3" for "turbo" and run it again to see what the faster model buys on your audio. Try batch_size 8, the default in the source, and 16, and keep the one with the higher R. The README's own batched example uses 16.

Move audio in and transcripts out

From your own computer, rsync resumes if the connection drops. Use the user, host and port from the SSH command on your deployment, which is ubuntu on QuantaCloud and Hyperstack, Ubuntu with a capital U on Massed Compute and root on Verda:

rsync -avz --progress -e "ssh -p 22" ./audio/ ubuntu@YOUR_VM_IP:~/audio/
rsync -avz --progress -e "ssh -p 22" ubuntu@YOUR_VM_IP:~/out/ ./transcripts/

The data transfer guide compares scp, rsync and the other routes. Pull the transcripts down before you stop the instance, because Stop deletes the disk and everything on it, including the model cache and your results. DeployGPU has no idle timer either, so a queue that finishes at night keeps billing until you stop it. The stop guide has the checklist, and the batch jobs guide shows how to launch, run, copy off and stop from a script.

When something goes wrong