How to Serve GLM-5.3 Across Two NVIDIA DGX Stations
NVIDIA Technical Blog shows how to serve the quantized 753-billion-parameter GLM-5.3 locally with vLLM across two connected NVIDIA DGX Stations. Tensor parallelism splits weights and computation between the GB300 GPUs, keeping the roughly 464 GB checkpoint in GPU memory behind one OpenAI-compatible API. In NVIDIA’s SPEED-Bench test, throughput peaked at 607.8 tokens per second at concurrency 16 before falling to 385.0 tokens per second at concurrency 32.
The tutorial provides a working distributed-inference configuration for a model too large for one GB300 GPU and supplies measured concurrency data, although the results come from a single vendor-published source.
Running a frontier-scale model locally means keeping all its weights in GPU memory; the largest models no longer fit on one machine. Two NVIDIA DGX Stations connected over ConnectX-8 RDMA serve one model through a single endpoint, with tensor parallelism splitting the model weights and computation across both GPUs.
The NVIDIA GLM-5.3 NVFP4 model, a quantized version of Z.ai’s mixture-of-experts (MoE) model, has 753 billion parameters, with 40 billion activated per token. Sparse activation reduces computation, but all model weights still need to remain available because each token can be routed to different experts. At about 464 GB, the checkpoint exceeds the 252 GB of HBM3e on a single GB300 GPU. This tutorial uses two DGX Stations to keep the weights in GPU memory, with CPU weight offload disabled.
Before starting, connect the two Stations with the Connect Two DGX Stations playbook.
This post shows how to serve GLM-5.3 locally across two NVIDIA DGX Stations with vLLM, send a request, and measure behavior under concurrent load. Tensor parallelism (TP=2) splits the model’s tensors across the two GB300 GPUs. Both Stations compute every request, and Station A returns the answer through one OpenAI-compatible API. By the end, you’ll have a working endpoint and a baseline for how it behaves under load.
#GLM-5.3#NVIDIA#Z.ai#vLLM#NVIDIA DGX Station#NVIDIA GB300#SPEED-Bench