DeepSpeed ZeRO-3
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Developer releases sdft_zero3 for self-distillation under ZeRO-3
Developer ifiaposto released sdft_zero3, a plug-and-play GitHub repository combining self-distillation with an EMA teacher, ZeRO-3, and PEFT. The author says it includes LoRA adapter switching for teacher and student passes, resumable EMA teacher checkpoints, and DeepSpeed configurations intended to address memory spikes. The author reports testing the method with Qwen3.5 and ToolAlpaca.
r/MachineLearning · 🔥 0
Experience and discussion from the community
Share my DeepSpeed ZeRO-3 experienceAsk about DeepSpeed ZeRO-3
Nobody has shared their experience with DeepSpeed ZeRO-3 yet.