Baseten cuts GPU-resident delta weight sync time to 15 seconds
Baseten redesigned its delta weight-sync path to use model weights already resident on GPUs as the parent copy, eliminating the need for a second full checkpoint in CPU memory. The company reports that syncing an approximately 770 GB GLM-5.3 checkpoint now takes 15 seconds with a 0.8-second request pause, versus 36 seconds and six seconds previously. Baseten also reports up to 3.2× higher output throughput per GPU at concurrency 48, while its Kimi K3 tests include an 8.4 GB delta applied across two GB300 nodes.
GPU-resident parent weights can reduce update latency and free host memory for KV-cache offloading, making the approach directly relevant to large-model reinforcement-learning rollout infrastructure; however, the performance results are vendor-reported.
Published
October 30, 2026
In our last technical blog, we showed delta weight syncs for a ~770 GB GLM-5.3 checkpoint in 36 seconds, with 6 seconds of request pause. The same sync now takes 15 seconds, with a 0.8-second pause, enabling throughput improvements by as much as 3.2x.
We rebuilt the weight sync path around the weights already on the GPU. This removes the need for us to maintain a second full copy in CPU memory. This new design lets us:
- Verify updates by reading only changed GPU positions
- Minimize pause time by preparing GPU writes beforehand
- Use freed host memory for KV cache, improving throughput
- Unlock Kimi K3 RL rollout on 8:GB300, previously blocked by CPU memory
#Baseten#GLM-5.3#Kimi K3#NVIDIA GB300#Baseten Delivery Network