Multi-GPU instances: 2x, 4x and 8x GPUs in one node

The difference between a 2-GPU VM and an 8-GPU VM is not only more compute, it is a different question about how those GPUs talk to each other. This page covers every multi-GPU offer on DeployGPU and what the interconnect actually means for a job.

How GPUs inside one VM talk to each other

SXM cards, such as the H100 SXM, H200 SXM and B200, sit on a shared board with NVSwitch, so every GPU reaches every other GPU over NVLink at the same speed, whether the VM has 2, 4 or 8 of them. PCIe cards have no such switch built in. NVIDIA makes NVLink bridges for some of them: pairs of RTX A6000, A100 PCIe, H100 PCIe or H100 NVL cards, and groups of 2 or 4 H200 NVL cards. Beyond what a bridge joins, cards in one VM talk over the server's PCIe fabric, which is slower and shared with the rest of the system. The RTX PRO 6000, L40, L40S and RTX 6000 Ada have no NVLink option at all, so those VMs are PCIe end to end regardless of GPU count. Whether a bridge is fitted on a given PCIe offer is up to the provider, which is what the Interconnect column reports.

Tensor parallelism or data parallelism

Data parallelism keeps a full copy of the model on every GPU and splits the batch of requests or training examples across them, so each GPU mostly works on its own and only needs to sync occasionally. It tolerates a slower link well, which is why it is a reasonable fit even on PCIe-only VMs. Tensor parallelism splits the layers of a single model across GPUs so a model too large for one card can run at all, or so one request gets answered faster by spreading the work. Every layer needs the other GPUs' output before it can continue, so tensor parallelism depends on a fast link between GPUs, which is why it is the usual choice on NVLink-connected SXM nodes and a poor fit across a slow PCIe path. The vLLM guide covers the flag that sets tensor parallelism across GPUs on a rented VM.

Checking the topology yourself

Once connected over SSH, nvidia-smi topo -m prints a grid of how each GPU reaches every other GPU:

nvidia-smi topo -m

Entries like NV1 or NV12 mean an NVLink connection between that pair. Entries like PIX, PXB or PHB mean the GPUs only share a PCIe path, at varying distance through the server's PCIe switches. On an SXM node, expect an NV entry between every pair. On a VM built from PCIe cards with no bridge, expect PIX, PXB, PHB or SYS instead.

How the Interconnect column works

The Interconnect column on DeployGPU's live tables follows the card, not a live measurement of each machine: SXM cards show NVLink (SXM), cards with no NVLink hardware show PCIe no matter how many are in the VM, and PCIe cards show NVLink bridge when the provider lists one. It is a quick way to tell, before launching anything, whether tensor parallelism across the whole VM makes sense or whether the job should stick to data parallelism instead. The H100 and L40S pages show what that looks like for an NVLink card and a PCIe-only card side by side.

One VM, up to 8 GPUs

DeployGPU launches VMs in fixed sizes of 1, 2, 4 or 8 GPUs. One VM is the largest unit available: there is no multi-node cluster, and no self-serve InfiniBand between machines. For more than 8 GPUs working together, or a fixed term at a fixed price, request a cluster quote and a person answers by email rather than a checkout page. The individual GPU pages, including H100, H200 and A100, cover each card's own interconnect in more detail.