Streaming DiLoCo
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Streaming DiLoCo Brings WAN-Based LLM Training to MaxText
Google Developers Blog says MaxText v0.2.5 introduces Streaming DiLoCo, which lets independently located learner groups train across datacenters while reducing cross-slice communication through periodic global updates and fine-grained parameter streaming. In tests using Qwen3-32B on two TPU v5p-32 slices, Google reported about 64–65 times less cross-slice traffic than data parallelism; the worked example says the 10 Gbit/s link required roughly 1.3 Gbit/s for Streaming DiLoCo. The feature is available through MaxText v0.2.5 on PyPI, its Docker images and the GitHub main branch.
Google Developers Blog · 🔥 0
Experience and discussion from the community
Share my Streaming DiLoCo experienceAsk about Streaming DiLoCo
Nobody has shared their experience with Streaming DiLoCo yet.