Google's Tunix Zero-TIM targets bitwise-zero train-inference mismatch in TPU RL
Google announced an implementation of zero train-inference mismatch (Zero-TIM) developed in Tunix, its open-source JAX-native library for LLM reinforcement learning and post-training on Cloud TPU. The approach makes the sampler and trainer assign bitwise-identical logprobs to the same action and token history, verified at multi-host scale across 64 TPU v5p chips with a DP8×TP8 mesh, by locking kernel accumulation order, ordering cross-shard reductions, sharing the action-logprob path and differentiating serving kernels with custom rules. In single-seed FrozenLake runs with a public Qwen3-8B checkpoint, Zero-TIM logged zero mean absolute sampler-trainer logprob difference at every step and reached higher observed solve rates than standard training with trainer-old logprobs and token-level truncated importance sampling.
Google says the mismatch stems from floating-point reduction order differing between rollout decoding and batched trainer passes, a hidden off-policy gap that compounds over multi-turn trajectories; the modular kernel patterns and three-way verification contract are designed for adoption in existing multi-host JAX/TPU serving and training pipelines, with support for linear-attention and Mixture-of-Experts architectures being extended.
Unlocking Stable On-Policy RL on TPU with Bitwise-Zero Train–Inference Mismatch - Google Developers Blog
Unlocking Stable On-Policy RL on TPU with Bitwise-Zero Train–Inference Mismatch
NOV. 16, 2026
Yuxuan Zhang Software Engineer
Lin Chai Senior Software Engineer
Lance Wang Senior Software Engineer