TorchTitan is a PyTorch-native platform for rapid experimentation and large-scale pre- and post-training of generative AI models. It is designed as a minimal, clean-room implementation of PyTorch-native techniques, with flexible extension points for users’ needs. This has enabled TorchTitan to find widespread adoption across research and production in academia and industry.

The TorchTitan team landed massive improvements in the past year, without losing its minimal, extensible ethos. Enhancements include support for SOTA models, dramatic improvements of out-the-box performance at scale, RL support and new communication features.

TorchTitan is for everyone – easy to use, providing SOTA performance and allowing users to start small and scale up seamlessly. We will present all these new improvements in the following sections, and we look forward to your feedback!

New model support

SOTA model support is critical for the success of training frameworks, and TorchTitan aims to always provide the latest advancements when it comes to open-weights architecture. TorchTitan added PyTorch-native support for Muse Spark Open, Muse Glimmer, Qwen3-VL and Qwen3.5, Kimi K2.7, Kimi K3, and DeepSeek-V4. These were full training-stack integrations with support across TorchTitan’s 5D parallelism stack. This work also introduced a composable configuration system built around typed Python recipes, model-config transforms, and targeted component overrides. Users can change parallelization strategies, model components, or hardware-specific kernels without maintaining separate model forks. The same interfaces integrate optimized kernels from Attention Gym, including Gated DeltaNet, Kimi Delta Attention, and sparse-attention kernels, as well as opt-in Helion kernels such as fused RoPE, while remaining compatible with torch.compile.