The pip install flash-attn command fails or runs for a long time because the package on PyPI is a source distribution: pip downloads the source, its setup script guesses the name of a prebuilt wheel on GitHub, and when no wheel fits your Python, PyTorch, CUDA and C++ ABI it compiles the CUDA kernels on your VM. PyPI lists flash-attn 2.8.3.post2, uploaded on 14 October 2026, and the GitHub release for that version has wheels only for PyTorch 2.4 to 2.8, so on any newer PyTorch, including today's 2.15.0, the compile always runs. The facts below were checked on 12 November 2026.
Getting requirements to build wheel ... error
error: subprocess-exited-with-error
...
ModuleNotFoundError: No module named 'torch'
flash_attn was requested, but nvcc was not found. Are you sure your environment has nvcc available? If you're installing within a container from https://hub.docker.com/r/pytorch/pytorch, only images whose names contain 'devel' will provide nvcc.
The first is from the project's tracker, issue 1920. The second is the warning that setup.py in the 2.8.3.post2 source package prints when it finds no CUDA toolkit.
What flash-attn is and which version fits your GPU
FlashAttention is an exact attention algorithm that uses tiling to cut the reads and writes between GPU high bandwidth memory and on-chip SRAM, according to the paper's abstract. The README on the main branch covers three generations. The flash-attn package is FlashAttention-2, FlashAttention-3 is a beta for Hopper built from the hopper folder, and FlashAttention-4 is a CuTeDSL implementation. This is what the README names for the GPUs DeployGPU rents:
| GPUs | Architecture | Versions the README names |
|---|---|---|
| RTX A6000, A100 | Ampere | FlashAttention-2 |
| RTX 6000 Ada, L40, L40S | Ada | FlashAttention-2 |
| H100, H200 | Hopper | FlashAttention-2, FlashAttention-3 (its requirements name H100 and H800), FlashAttention-4 |
| RTX PRO 6000 Blackwell, B200, B300 | Blackwell | FlashAttention-4, with B200 as its example. The FlashAttention-2 list names no Blackwell GPU |
Why pip install flash-attn fails or compiles for so long
The first error comes from build isolation. setup.py imports torch on its first lines, and pip's build system reference says it builds in an isolated environment, with build requirements handled independently of the environment you installed torch in, so the import fails. The README's fix is --no-build-isolation, and the pip docs say you then install the build requirements yourself. The README lists packaging, psutil and ninja.
A source build needs three more things. The first is an nvcc whose major version matches torch.version.cuda, because PyTorch's extension builder raises an error when the major versions differ. PyPI pins torch 2.15.0 to CUDA 13.2 libraries, so it needs a CUDA 13 toolkit, and the CUDA version guide shows how to read both numbers and why nvcc is often missing on a GPU VM. The second is memory. The README says ninja may run too many parallel jobs on a machine with less than 96 GB of RAM and suggests MAX_JOBS, and a comment in setup.py puts each job at 8 to 9 GB with 4 nvcc threads. The third is a working ninja, because the README says compiling takes 3 to 5 minutes on a 64-core machine with it and can take 2 hours without it.
python3 -c "import torch; print(torch.__version__, torch.version.cuda)"
nvcc --version
python3 -m pip install packaging psutil ninja
ninja --version; echo $?
MAX_JOBS=4 python3 -m pip install flash-attn --no-build-isolation
setup.py also reads FLASH_ATTN_CUDA_ARCHS, whose default is 80;90;100;120, and it adds 100 and 120 only when nvcc is 12.8 or newer, so the package can compile for Blackwell even though the README does not list it. Setting the variable to your GPU's architecture builds one target instead of four. NVIDIA's rule that code for one capability runs on the same major and the same or a higher minor lets 80 cover the A100, RTX A6000 and the Ada cards, with 90 for the H100 and H200, 100 for the B200 and B300 and 120 for the RTX PRO 6000:
FLASH_ATTN_CUDA_ARCHS=90 MAX_JOBS=4 python3 -m pip install flash-attn --no-build-isolation
If the VM has no toolkit, a container is the other way. The setup warning says only pytorch/pytorch images with devel in the name provide nvcc, and the README recommends NVIDIA's PyTorch container, which has the tools to install FlashAttention. The container toolkit guide covers the GPU flag and the tag rule:
sudo docker run --rm -it --gpus all pytorch/pytorch:2.15.0-cuda13.2-cudnn9-devel bash
Match a prebuilt wheel to your Python, PyTorch and CUDA
setup.py forms the wheel name from your environment, and a real asset of the newest release looks like this:
flash_attn-2.8.3.post2+cu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
The fields are the flash-attn version, cu12 for the major version of the CUDA that torch was built with, torch2.8 for torch's major and minor version, cxx11abiTRUE or FALSE for its C++ ABI, cp312 for your Python and linux_x86_64 for the platform. One command prints your values:
python3 -c "import sys, torch; print('cp%d%d' % sys.version_info[:2], torch.__version__, torch.version.cuda, torch._C._GLIBCXX_USE_CXX11_ABI)"
uname -m
The release page lists 48 wheels of about 256 MB each, for torch 2.4 to 2.8, CPython 3.9 to 3.13 and both ABI settings. Across the 4,586 assets of the project's v-tagged releases, none is named for a PyTorch newer than 2.10. The project's release workflow builds these wheels with CUDA 12.9.1 and leaves FLASH_ATTN_CUDA_ARCHS at its setup.py default of 80;90;100;120, and NVIDIA's cuobjdump with -lelf lists the cubins in a library if you want to check a wheel. If your values match, install the wheel by its URL, which is the release download address plus the file name:
python3 -m pip install https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3.post2/flash_attn-2.8.3.post2+cu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl
On a Blackwell GPU that means a PyTorch 2.8 build from the cu128 or cu129 index, and the install guide explains how to pick one. The other route is the compile above.
FlashAttention-3 on an H100 or H200
The README says FlashAttention-3 is optimized for Hopper GPUs, lists H100 and H800 with CUDA 12.3 or newer under requirements, and recommends CUDA 12.8. It builds from the hopper folder of a clone:
git clone https://github.com/Dao-AILab/flash-attention
cd flash-attention/hopper
python3 setup.py install
from flash_attn_3 import flash_attn_interface
flash_attn_interface.flash_attn_func()
That import is from the main-branch README, while the README of the 2.8.3.post2 tag shows import flash_attn_interface, so match the README to the commit you build. The H200 is a Hopper GPU, with compute capability 9.0 on NVIDIA's CUDA GPU page, but the README names only the H100 and H800. PyPI's flash-attn-3 holds only a yanked 0.0.0 stub of about 1.5 KB, so pip install flash-attn-3 is not the route.
FlashAttention-4 on Blackwell
The README says FlashAttention-4 is optimized for Hopper and Blackwell GPUs, with H100 and B200 as examples. It installs without a compile, because PyPI's flash-attn-4 is a pure Python wheel, and every release there is a beta, with 4.0.0b39 on 11 November 2026:
python3 -m pip install flash-attn-4
python3 -m pip install "flash-attn-4[cu13]"
from flash_attn.cute import flash_attn_func
out = flash_attn_func(q, k, v, causal=True)
The README recommends the second line on CUDA 13.
When to skip flash-attn
PyTorch's scaled dot product attention docs for 2.15 list FlashAttention-2 as one of three implementations, all enabled by default, and say the function picks the best one for the inputs. The sdpa_kernel context manager restricts the choice:
from torch.nn.attention import SDPBackend, sdpa_kernel
with sdpa_kernel(SDPBackend.FLASH_ATTENTION):
out = torch.nn.functional.scaled_dot_product_attention(q, k, v)
Hugging Face's Transformers docs, version 5.22.2, say that attn_implementation="flash_attention_2", "flash_attention_3" or "flash_attention_4" falls back to the matching Hub kernel when the compiled flash-attn package is not installed, and the transformers[kernels] extra adds the kernels library. vLLM bundles its own FlashAttention fork: its requirements/cuda.txt at 0.33.0 lists no flash-attn package, and its attention docs pick FlashAttention 4 on Blackwell, 3 on Hopper and 2 elsewhere unless you set --attention-config.flash_attn_version. If a recipe such as the Unsloth fine-tuning guide or a model README asks for flash-attn, I try these settings before I compile.
When to contact support
Email support@deploygpu.ai if nvidia-smi fails or lists fewer GPUs than the offer, or if the VM stops answering during a build. Send the first failing lines, the nvcc --version output, the torch line from above, the provider and the deployment ID. Compiler errors inside flash-attention's own code belong in the project's issue tracker. The contact page is the other way in, and the first five minutes guide lists the checks for a new VM.
Frequently asked questions
Why does pip install flash-attn say No module named torch?
pip builds the package in an isolated environment where torch is not installed. Install torch first, then add packaging, psutil and ninja, and run pip install flash-attn with --no-build-isolation.
How long does pip install flash-attn take?
The README says compiling takes 3 to 5 minutes on a 64-core machine with ninja, and can take 2 hours without it. A prebuilt wheel that matches your PyTorch avoids the compile.
Does flash-attn work on an H100, an H200, a B200 or an RTX PRO 6000?
The README lists FlashAttention-2 for Ampere, Ada and Hopper, FlashAttention-3 for Hopper with H100 and H800 named, and FlashAttention-4 for Hopper and Blackwell. It names no Blackwell GPU for FlashAttention-2.
Do I need flash-attn to run vLLM?
No. vLLM's CUDA requirements list no flash-attn package, and its build compiles its own FlashAttention fork, which its docs let you select by version.
What is the difference between FlashAttention 2 and FlashAttention 3?
FlashAttention-3 targets Hopper GPUs such as the H100, and its paper's abstract credits Tensor Core and TMA asynchrony and FP8 support. FlashAttention-2 supports Ampere, Ada and Hopper, per the README.
What is flash attention?
It is an exact attention algorithm that uses tiling to reduce reads and writes between GPU memory and on-chip SRAM. The original paper introduced it, and flash-attn is the library that implements the versions.