tmux for GPUs. Split an H100 four ways and each job gets 158 TFLOP/s, within 0.3% of the others. Share cards across many jobs in space and in time, resize them while they run, and see exactly who used what.
Use them from anywhere. Point a laptop, a CI runner or an agent at a gmux host,
and gmux run runs there, on a share of a card, with the output streaming
back. The machine you type on needs no GPU.
No root and no host access needed. If you can run Docker, you can run gmux, so it works on almost any GPU provider: RunPod, Vast, Lambda, DigitalOcean, a cloud VM, a university node, or your own box.
Works with NVIDIA, AMD and Intel GPUs. What each one can enforce differs; platform support is documented in Isolation.
On a machine with a GPU:
go build -o /usr/local/bin/gmux ./cmd/gmux
gmux serve --addr :7070
# gmux serving machines without a GPU on :7070 over TLS
# on each of them, run:
# gmux remote add NAME 5c1e...@THIS-HOST:7070#9f2a...On a laptop, a CPU box, a CI runner, anything without a GPU:
gmux remote add gpu1 5c1e...@gpu1.example.com:7070#9f2a...
gmux run --share 0.25 --mem 20G -- python train.py # runs on gpu1, output streams back
gmux run --share 0.5 --pull 'out/*' -- python eval.py # then fetches its results
gmux top # what is on gpu1's cardsOn the GPU machine itself the same commands run locally. No GPU handy at all?
Try it against a pretend card: gmux serve --fake 1xH100:80G.
machines without a GPU GPU host: gmux serve --addr :7070
(laptops, CI runners, agents)
+------------------------------------+
+----------------------------+ TLS | scheduler |
| gmux run --share 0.25 | --------> | seats, admission, queue, preempt |
| -- python train.py | files up | |
+----------------------------+ output | card 0 |
<-------- | +--------+--------+-------------+ |
+----------------------------+ | | job A | job B | job C | |
| gmux run --share 0.5 ... | --------> | | 1/4 | 1/4 | 1/2 | |
+----------------------------+ | +--------+--------+-------------+ |
| each job: compute cap, memory cap,|
+----------------------------+ | network fence |
| gmux top / ps / usage | --------> | |
+----------------------------+ | per-second usage ledger |
+------------------------------------+
Every job, local or remote, is admitted onto seats of a card. Its compute share
is enforced by MPS on NVIDIA and by dedicated shader engines on AMD, its memory
by MPS or a preload library, and its network by a seccomp fence. With several
GPU hosts configured, each gmux run goes to the one with the most room.
Most GPU jobs do not use the whole card. A notebook idles, an eval runs in bursts, a small model serves a few requests a minute. Renting a card per job wastes most of it, and so does renting a GPU machine for every developer, CI runner or agent that needs a GPU now and then. Sharing by hand means jobs running out of memory, fighting over compute, copying files around, and no record of who used what.
gmux turns a few GPU machines into a pool: every job gets a capped share of a card, from wherever it is started, and the books are kept.
Spatial sharing. Several jobs run on one card at the same time, each with its own slice of compute and memory.
Temporal sharing. Idle guaranteed capacity is lent to jobs that can use it, and low priority jobs can wait for a free slot instead of competing.
gmux run --share 0.25 --burst -- python eval.py # use idle compute above the guarantee
gmux run --share 0.5 --priority 1 --preemptible -- python sweep.py
gmux run --share 0.5 --priority 9 -- python urgent.py # can push the low job offLive resize. Grow or shrink a running job's share.
gmux resize j1234 --share 0.75Admission. gmux refuses a job the card cannot fit, instead of letting it start and crash its neighbours.
Accounting. Every second of every share is recorded, by job, owner, card or vendor.
gmux usage --since 24h --by ownerNetwork fence. Limit what a job can reach, per job, enforced in the kernel with seccomp. No root needed.
gmux run --share 0.25 --allow api.openai.com --allow pypi.org -- python agent.py
gmux run --share 0.25 --deny-net -- python offline.pyAnywhere you get a container with a GPU. gmux never installs drivers, loads
kernel modules, or reconfigures the host. It detects whatever vendor tools are
present (nvidia-smi, rocm-smi, xpu-smi) and uses them. Tested providers
and their quirks are in docs/providers.md.
A machine with no GPU runs jobs on GPU hosts it has been told about. There is nothing to mount and no driver to install on it.
- Where a command goes. --local, or--remote NAME, or a local daemon if one is running, or else the configured hosts. So on a machine without a GPU,run,ps,top,cards,stop,resizeandusageall reach the GPU hosts by default.runasks every host what it has free and goes to the one with the most room for the job, skipping hosts that do not answer.
- What happens on a run. The working directory is synced to the host
(content addressed: unchanged files are neither re-read nor re-sent), the
command runs there as an ordinary gmux job with its share, its compute and
memory caps and its network fence, the output streams back, and the exit
code is yours. --pull GLOBfetches result files. Ctrl-C stops the job on the host.
- Security. Hosts serve over TLS with a certificate made once and pinned by
its fingerprint in the target, plus a bearer token, both printed by
gmux serve --addr. A host with a different certificate or a wrong token is refused.
gmux remote ls shows each host and what is free on it; gmux remote default NAME picks the one to prefer on a tie.
Measured from a machine with no GPU to an AMD MI350X host over the internet (4 ms round trip):
Memory caps, the network fence, exit codes, Ctrl-C, host placement and certificate pinning were all checked on that run: docs/evidence/remote-mi350x.txt.
This is command-level remoting: the command and its files go to the GPU, the GPU does not come to the machine. A program on the laptop cannot open a remote card directly. That is deliberate. The caps and the fence run next to the card, where they can be enforced, and a round trip per command, not per CUDA call, is what makes it work over an ordinary network.
A few things people do with a GPU multiplexer:
- Batch many small evals or inferences onto a few cards instead of renting one per job.
- Pack fine-tuning and serving together, giving serving the guaranteed share and letting training burst into the rest.
- Give a team fair shares of a shared box, with usage they can see.
- Run untrusted or semi-trusted jobs with a network allowlist and a memory cap, without handing out root.
- Squeeze a fixed rented pod by fitting more work onto the card you are already paying for by the hour.
- Give laptops, CI runners and agents GPU access without a GPU machine each: point them at a couple of shared GPU hosts.
gmux came out of batching GPU evaluations of AI agents: many short agent trials across a handful of cards, each trial holding a slice, with per-second accounting and a network allowlist per trial, dispatched from machines with no GPU of their own.
gmux is for jobs you trust, or mostly trust. Without host access, sharing is enforced in user space. On NVIDIA, MPS caps compute and the driver caps memory; on AMD, compute units are masked and a preload library caps memory; on Intel, sharing is by admission only. In every case a job you do not trust at all can still affect its neighbours in ways only the driver or the hardware could prevent. If you need hard isolation between strangers, use MIG or separate cards. The full picture is in docs/isolation.md.
Each job runs a sustained fp16 8192-square matmul for 20 seconds, long enough
that concurrent jobs genuinely overlap. Numbers are TFLOP/s per job, measured
through gmux on real cards. Reproduce with bench/run.sh.
NVIDIA shares are capped by MPS; AMD shares get their own shader engines on every chiplet (the AMD rows show each of the four quarters alone) and a memory cap through the preload shim. On every card the memory cap is hard, the split between concurrent jobs is even, and a quarter share alone cannot take the whole card. Without gmux, every over-limit allocation above succeeds.
Sharing pays when jobs do not fill the card on their own: notebooks, bursty evals, small models serving a few requests. A job that already saturates the card, like this matmul, runs fastest alone; four saturating quarter shares add up to between 64% (MI325X) and 95% (H100) of one dedicated job. Transcripts are in docs/evidence.
gmux runs as a daemon in a container next to your jobs. A scheduler decides who goes where, in space and time. For NVIDIA it runs one MPS daemon per card and binds each job to its share; for AMD and Intel it sets the vendor's device and compute-unit variables and, where possible, caps memory with a small preload library. The network fence installs a seccomp filter on the job and answers its connections from outside the filter, so the job cannot route around the allowlist. Usage is a plain append-only ledger. Machines without a GPU reach the daemon over TLS: they sync their working directory, the job runs as an ordinary job on the host, and the output streams back. Details in docs/how-it-works.md.
go build -o gmux ./cmd/gmux # the daemon and CLI
make -C shim # libgmux.so, the memory-cap shim
go test ./... # unit and integration tests
pip install -e sdk/python # the Python clientSpatial and temporal sharing, admission, live resize, per-second accounting, the network fence, remote GPUs from machines without one, and multi-vendor detection are built and covered by tests on every build.
On real NVIDIA H100, B300 and L40S cards, gmux detects the card, starts MPS, applies each job's
compute and memory caps, splits the card fairly, and enforces the network
fence: --deny-net blocks everything, and --allow resolves and connects to
allowlisted hosts while refusing the rest.
On real AMD MI325X, MI350X and MI355X cards, gmux selects the card by UUID, gives each job its own
shader engines on every chiplet (checked with a compute-unit probe, see
bench/cuprobe.hip), and caps HIP allocations through the preload shim, with
PyTorch.
The Intel backend drives xpu-smi; its parser is tested against recorded
output from Max 1550, Flex 170 and Arc A770. gmux cards reports what each
card can enforce.
Preempt-and-resume through cuda-checkpoint is the next backend feature.
Apache-2.0.