Cloud TPU
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- Google's Tunix Zero-TIM targets bitwise-zero train-inference mismatch in TPU RL
Google announced an implementation of zero train-inference mismatch (Zero-TIM) developed in Tunix, its open-source JAX-native library for LLM reinforcement learning and post-training on Cloud TPU. The approach makes the sampler and trainer assign bitwise-identical logprobs to the same action and token history, verified at multi-host scale across 64 TPU v5p chips with a DP8×TP8 mesh, by locking kernel accumulation order, ordering cross-shard reductions, sharing the action-logprob path and differentiating serving kernels with custom rules. In single-seed FrozenLake runs with a public Qwen3-8B checkpoint, Zero-TIM logged zero mean absolute sampler-trainer logprob difference at every step and reached higher observed solve rates than standard training with trainer-old logprobs and token-level truncated importance sampling.
Google Developers Blog · 🔥 0 - MaxText v0.2.4 adds long-context training for Cloud TPU
Google announced that MaxText v0.2.4 supports long-context training on Cloud TPU through four context-parallel strategies: All-gather, A2A (DeepSpeed Ulysses), P2P (Ring Attention), and A2A + P2P (USP). The framework can extend context to 14 million tokens across TPU 5p and TPU 7, and Google reports that Llama 3.1 8B reached 70% Model FLOPs Utilization on TPU 5p for sequences up to 4 million tokens. The release is available on PyPI, Docker images, and GitHub main.
Google Developers Blog · 🔥 0
Experience and discussion from the community
Share my Cloud TPU experienceAsk about Cloud TPU
Nobody has shared their experience with Cloud TPU yet.