Run Open WebUI and Ollama on your own cloud GPU

The quickest way I know to get a private ChatGPT-style chat on open models is to rent the GPU instead of buying one: launch the Open WebUI + Ollama template on DeployGPU, pull a model, and you are chatting on your own hardware in about ten minutes. Nothing you type goes to a hosted model API. The prompts, the model and the chat history all live on a VM that only you can open.

What the template sets up

The template runs the official ghcr.io/open-webui/open-webui:ollama image, which bundles Open WebUI with Ollama in one container, on a GPU VM from QuantaCloud, Massed Compute, Verda or Hyperstack. When it is ready, the dashboard shows an Open Application button. It opens a private HTTPS link on app.deploygpu.ai that only your DeployGPU account can reach, signed in, in a browser.

Three things surprise people, so here they are up front. No model is preloaded: you pull the ones you want after the first sign-in, which keeps the launch fast and lets you pick the size. Open WebUI has its own accounts on top of the DeployGPU sign-in, and the first account you create on a new deployment becomes its admin. And the private link is for you, in a browser. It cannot be shared with a teammate, it has no custom domain, and code cannot call it with an API key, although there is a supported way to reach the API from code, covered below.

The template usually takes about 6 minutes from launch to ready, longer than a plain VM, because DeployGPU waits until Open WebUI itself answers before it marks the deployment Running.

Pick a GPU for the model you want

The GPU decides which models you can run. Ollama keeps the whole model in GPU memory when it fits and splits it with the CPU when it does not, which works but is many times slower. Pick a card with room for the model plus its context.

Sizes come from the Ollama library and the model cards, checked on 28 September 2026. The 70B row is calculated from the parameter count. My rule for chat is a 48 GB card unless you already know you need gpt-oss-120b: it runs every model in the first four rows entirely on the GPU, without paying for an 80 GB card. For the reasoning behind "room for the context", see how the KV cache uses GPU memory.

Launch it

1. Click the button above, or open the marketplace, pick any VM offer and click Launch. You can also start from Templates in the dashboard sidebar: choose Open WebUI + Ollama, then one of the suggested GPUs.
2. On the deploy page, choose Open WebUI + Ollama under Template if it is not already selected.
3. Check the cost summary and the SSH key, then deploy. The first hour is reserved from your credit balance.
4. Wait on the Deployments page until the status reads Running, about 6 minutes.
5. Click Open Application.

If a launch fails, the reserved hour comes back to your balance automatically, and you can try another offer straight away.

The first visit

The link first asks you to sign in to DeployGPU if this browser is not signed in already. Then Open WebUI shows its own sign-up screen. Create an account there: it becomes the admin of this deployment, and it exists only on this VM.

Next, pull a model. Type a tag such as gpt-oss:20b into the model selector at the top of a new chat and choose the option to pull it from Ollama, or use the model settings in the admin panel. You can also pull over SSH, which shows progress in the terminal:

# The container has no fixed name, so find it by image
CID=$(sudo docker ps -q --filter ancestor=ghcr.io/open-webui/open-webui:ollama)
sudo docker exec -it "$CID" ollama pull gpt-oss:20b

Pull time depends on the provider's network and the model size. At 1 Gbit/s, a 14 GB model takes about two minutes to download. Once the pull finishes, pick the model and start a chat.

The sign-in step has a history. Open WebUI sends its own Authorization: Bearer header with its requests, and until 24 September 2026 the DeployGPU link gate rejected them, so users bounced back to the sign-in page in a loop. The gate logged 34 of those rejected requests on 22 September alone. We fixed the gate on 24 September and there have been none since. If you ever see a loop, clear the cookies for app.deploygpu.ai and open the app again from the dashboard.

Using it from code

The private link needs your browser session, so a script or an OpenAI client pointed at it gets a sign-in redirect, not an answer. The way around it is an SSH tunnel to the port Open WebUI listens on inside the VM, which is 8080 bound to 127.0.0.1:

# Use the user, host and port from the SSH command on your deployment
ssh -N -L 8080:127.0.0.1:8080 ubuntu@YOUR_VM_IP -p 22

While the tunnel is open, Open WebUI answers on http://localhost:8080 on your machine. Create an API key in Open WebUI under Settings, Account (an admin may need to switch API keys on in the admin settings first), then call its OpenAI-compatible endpoint:

curl http://localhost:8080/api/chat/completions \
  -H "Authorization: Bearer $OPENWEBUI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-oss:20b", "messages": [{"role": "user", "content": "Say hello in five words"}]}'

The endpoint and key flow are described in the Open WebUI API documentation. The Ollama API itself, on port 11434, stays inside the container. If you want a server built for API traffic rather than chat, run vLLM on a GPU VM instead.

Before you stop

Export what you want to keep first. Open WebUI can export chats as JSON from its settings. To keep the whole Open WebUI state, including accounts and settings, copy its data folder out of the container and download it:

CID=$(sudo docker ps -q --filter ancestor=ghcr.io/open-webui/open-webui:ollama)
sudo docker cp "$CID":/app/backend/data ./openwebui-data
sudo tar czf openwebui-data.tar.gz openwebui-data
# then, from your own machine:
scp ubuntu@YOUR_VM_IP:openwebui-data.tar.gz .

Models are not worth copying. Write down the tags and pull them again next time. The stop guide covers the rest, including what a volume can and cannot keep for you.

What it costs

You pay the hourly rate of the GPU you launch on, shown on the button caption above and in the live 48 GB GPU table, and nothing extra for the template. Say you chat for 1 hour and 20 minutes and then stop: you pay for the first hour, the second hour is charged when it starts, and the unused 40 minutes of it go back to your balance when you stop. Credits are bought by card with a 5% + $0.30 processing fee per top-up. The pricing page has the details.

The cost that catches people is the forgotten VM. There is no idle timer, so a chat left open overnight bills all night. Stop it when you are done.

When something goes wrong

For anything else, the troubleshooting section and contact page are the next stops.