Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Streaming DiLoCo Brings WAN-Based LLM Training to MaxText

AI summary

Google Developers Blog says MaxText v0.2.5 introduces Streaming DiLoCo, which lets independently located learner groups train across datacenters while reducing cross-slice communication through periodic global updates and fine-grained parameter streaming. In tests using Qwen3-32B on two TPU v5p-32 slices, Google reported about 64–65 times less cross-slice traffic than data parallelism; the worked example says the 10 Gbit/s link required roughly 1.3 Gbit/s for Streaming DiLoCo. The feature is available through MaxText v0.2.5 on PyPI, its Docker images and the GitHub main branch.

In-depth insight

Streaming DiLoCo gives AI infrastructure teams a released MaxText option for pooling TPU slices over constrained or geographically distributed networks without relying on one monolithic high-bandwidth cluster.

Distributed LLM Training Across Metros: Streaming DiLoCo in MaxText - Google Developers Blog

Distributed LLM Training Across Metros: Streaming DiLoCo in MaxText

NOV. 19, 2026

Mohit Khatwani Software Engineer

Chris Zuo Software Engineering Intern

Nova Fallen Software Engineer

#Streaming DiLoCo#MaxText#DiLoCo#Qwen3-32B#Google DeepMind#Google Research#Google Cloud#TPU v5p

Original · 10 min readRead the original
Group chat

No comments yet. Start the conversation.