TorchTitan unifies PyTorch-native generative AI training workflows
TorchTitan is a PyTorch-native platform for pretraining and post-training generative AI models, spanning supervised fine-tuning and reinforcement learning. Its documentation says FSDP2, tensor, pipeline, context, and expert parallelism can be composed through DeviceMesh and DTensor, while TitanRL shares model definitions and kernels between the trainer and vLLM generation. The supplied report is a project-maintained repository snapshot and does not independently validate the stated performance.
The project provides a readable, customizable Python codebase that brings large-scale pretraining, post-training, and reinforcement learning into one stack and allows different parallelism strategies to be composed without rewriting model definitions.
A PyTorch native platform for training generative AI models
A PyTorch-native platform for pretraining and post-training generative AI models at scale
Quick start · Models · Features · RL · Docs · Contributing
torchtitan is a clean-room, PyTorch-native implementation of large-scale training. One readable codebase takes a model from pretraining through supervised fine-tuning to reinforcement learning, composing FSDP, tensor, pipeline, context, and expert parallelism without rewriting the model.