Baseten将增量权重同步缩至15秒
Baseten重构了增量权重同步路径，直接以GPU中已有的模型权重作为父版本，不再需要在CPU内存中保留完整检查点。Baseten称，约770 GB的GLM-5.3检查点现可在15秒内完成同步，请求暂停仅0.8秒，此前分别为36秒和6秒。公司还称在并发数为48时，单GPU输出吞吐最高提升3.2倍，并在两个GB300节点上测试了Kimi K3的8.4 GB增量更新。
以GPU常驻权重作为父版本可降低更新延迟并释放主机内存用于KV缓存卸载，直接关联大模型强化学习rollout基础设施；但这些性能数据均由厂商自行测试。
Published
October 30, 2026
In our last technical blog, we showed delta weight syncs for a ~770 GB GLM-5.3 checkpoint in 36 seconds, with 6 seconds of request pause. The same sync now takes 15 seconds, with a 0.8-second pause, enabling throughput improvements by as much as 3.2x.
We rebuilt the weight sync path around the weights already on the GPU. This removes the need for us to maintain a second full copy in CPU memory. This new design lets us:
- Verify updates by reading only changed GPU positions
- Minimize pause time by preparing GPU writes beforehand
- Use freed host memory for KV cache, improving throughput
- Unlock Kimi K3 RL rollout on 8:GB300, previously blocked by CPU memory
#Baseten#GLM-5.3#Kimi K3#NVIDIA GB300#Baseten Delivery Network