Analysis read

Nvidia shows 753B-parameter GLM-5.3 running on two DGX Stations

An Nvidia technical guide splits Z.ai's 753-billion-parameter GLM-5.3 across two GB300 DGX Stations with vLLM, peaking at 607.8 tokens per second. It shows frontier-scale open-weight models can now be served entirely from on-premises workstations.

AE
Aidose Editorial
Nov 15, 2026

Nvidia has published a step-by-step guide to serving Z.ai's GLM-5.3, a 753-billion-parameter mixture-of-experts model, across two DGX Station workstations using vLLM. The post, by Fatima Fall on the Nvidia Technical Blog, is dated November 13 and reports peak aggregate throughput of 607.8 tokens per second.

The setup uses the NVFP4-quantized checkpoint of GLM-5.3, which activates 40 billion parameters per token. At about 464 GB, the checkpoint is larger than the 252 GB of HBM3e on a single GB300 GPU, so the guide uses tensor parallelism across two stations, each with a GB300, linked by ConnectX-8 RDMA over a 400 Gb/s RoCE connection. Quantization is mixed precision, with NVFP4 applied to the MoE-block linear weights and activations. vLLM v0.30.0 serves the model behind an OpenAI-compatible endpoint, with multi-token prediction (four draft tokens per step), an FP8 KV cache, prefix caching and a 98,304-token context limit.

On Nvidia's SPEED-Bench coding prompts, measured with AIPerf, aggregate throughput rose from 107.4 tokens per second at a concurrency of 1 to 386.0 at 8 and peaked at 607.8 at 16, before falling to 385.0 at 32 as mean latency climbed to 453.5 seconds. Per-user throughput fell from 115.3 tokens per second at concurrency 1 to 13.0 at 32.

The guide warns that the resulting API is unauthenticated and should be firewalled, with authentication and TLS added before wider access. It also notes that each station needs roughly 464 GB of disk space for the weights, and that running the benchmark requires access to a gated Hugging Face dataset.

For teams that need to keep data on premises, the post is a concrete reference: a frontier-scale open-weight model can be held entirely in GPU memory across two workstation-class machines, with throughput that tops out around 16 concurrent requests. Beyond that point, the numbers show latency, not compute, becoming the limiting factor.

AE
Aidose Editorial
Nov 15, 2026 · read
Back to News