SSH into a cloud GPU

The two things that decide whether SSH works on the first try are the key you launched with and the username, and on DeployGPU the username changes with the provider. Get those two right and everything else in this guide is comfort: a config entry so you can type ssh gpu, VS Code on the remote machine, and tunnels to the apps you run there.

Before you launch: have a key

SSH keys are installed when the machine is created. A key you add afterwards does not reach a VM that is already running, so sort this out first.

If you do not have a key yet, create one. Ed25519 is the modern default and works on every provider DeployGPU launches on:

ssh-keygen -t ed25519 -C "you@laptop" -f ~/.ssh/deploygpu

That writes the private key to ~/.ssh/deploygpu and the public key to ~/.ssh/deploygpu.pub. On Windows 10 and 11 the same command works in PowerShell, and the files land in C:\Users\YOU\.ssh. Paste the contents of the .pub file on the SSH keys page. The private key never leaves your machine.

When you signed up, DeployGPU also generated a key called Default Key and offered its private half for download once, for about 10 minutes. If you did not save it, it cannot be downloaded again: add your own key and use that instead.

Which keys work where

The dashboard attaches one key per launch, the one you pick on the deploy page. If you work from two laptops, either use the same key on both or launch through the API, which installs every key on your account when you leave out ssh_key_ids.

Find the command and the username

Every deployment in the dashboard shows its SSH command with a copy button, in the form ssh user@ip -p port. Copy it rather than typing it, because the username is the part people get wrong:

Usernames are case-sensitive, so ubuntu on a Massed Compute machine fails with Permission denied (publickey) even though the key is right. On providers that log you in as a normal user, commands that change the system need sudo.

Running is not always ready

Nothing in DeployGPU checks that SSH answers before a VM is marked Running. QuantaCloud only reports running after its startup script has finished, so SSH is normally ready by then. Hyperstack marks a VM ready as soon as it is active with a public IP, and SSH can refuse connections for a short while after that. A Connection refused in the first minute is usually just that. This loop waits for it:

until ssh -o ConnectTimeout=5 -o BatchMode=yes -o StrictHostKeyChecking=accept-new \
      -i ~/.ssh/deploygpu ubuntu@YOUR_VM_IP true; do
  echo "SSH not ready yet, retrying in 10 seconds"
  sleep 10
done

App templates are different: they only show Running once the app itself answers, which is why they take about 6 to 8 minutes rather than 1 to 3.

Make it one word next time

An entry in ~/.ssh/config saves typing the IP, user, port and key every time. Use the values from your deployment:

Host gpu
    HostName YOUR_VM_IP
    User ubuntu
    Port 22
    IdentityFile ~/.ssh/deploygpu
    IdentitiesOnly yes
    ServerAliveInterval 30

Now ssh gpu connects, and scp and rsync accept gpu: as the host too. IdentitiesOnly yes stops SSH from offering every key in your agent first, which some servers count as failed attempts. ServerAliveInterval 30 keeps a quiet session from being dropped by a router along the way.

Each new launch gets a new IP address, and a provider can hand you an IP that another machine used before. When that happens SSH refuses to connect with REMOTE HOST IDENTIFICATION HAS CHANGED. For a fresh instance that is expected. Remove the old entry and connect again:

ssh-keygen -R YOUR_VM_IP

VS Code on the remote machine

The Remote - SSH extension (ms-vscode-remote.remote-ssh in the VS Code marketplace) opens folders on the GPU machine as if they were local, with the terminal and debugger running remotely. With the config entry above, run Remote-SSH: Connect to Host from the command palette and pick gpu. The first connection installs the VS Code server in your home directory on the VM, so it takes a little longer than the ones after it. Microsoft's Remote - SSH documentation covers the details.

Reach apps on the VM through a tunnel

A service on the VM should listen on 127.0.0.1, not on a public interface, and you reach it with local port forwarding. This forwards port 8888 on your laptop to port 8888 on the VM, for a Jupyter server you started yourself:

ssh -N -L 8888:127.0.0.1:8888 gpu

Open http://localhost:8888 while it runs. The same pattern with port 8000 is how the vLLM guide reaches an OpenAI-compatible API without exposing it to the internet.

Keep jobs running when you disconnect

A job started in a plain SSH session dies when the connection drops. Run long jobs inside tmux, which keeps them alive on the server:

tmux new -s train        # start a named session and run your job inside it
# press Ctrl-b, then d, to detach and leave it running
tmux attach -t train     # come back to it later

If tmux is missing, sudo apt-get install -y tmux installs it on the Ubuntu images these providers use.

Security basics

DeployGPU does not put a firewall in front of your VM, and Hyperstack VMs are created with inbound traffic open. Anything you start on 0.0.0.0 is reachable from the internet, so bind services to 127.0.0.1 and use tunnels as above. If you want a host firewall as well, allow SSH before you turn it on, or you lock yourself out:

sudo ufw allow OpenSSH
sudo ufw enable

In the interest of being straight with you: DeployGPU keeps an encrypted platform-managed key for each account. It is installed on Verda VMs and used as a fallback on some other providers when no key of yours is attached. You control your own keys, but treat a rented VM as shared infrastructure and keep irreplaceable secrets off it.

When it still does not connect

When you are done, copy your results off and stop the instance. Stopping deletes the disk, and the stop guide shows how to save what matters first.