sdft_zero3
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Developer releases sdft_zero3 for self-distillation under ZeRO-3
Developer ifiaposto released sdft_zero3, a plug-and-play GitHub repository combining self-distillation with an EMA teacher, ZeRO-3, and PEFT. The author says it includes LoRA adapter switching for teacher and student passes, resumable EMA teacher checkpoints, and DeepSpeed configurations intended to address memory spikes. The author reports testing the method with Qwen3.5 and ToolAlpaca.
r/MachineLearning · 🔥 0
Experience and discussion from the community
Share my sdft_zero3 experienceAsk about sdft_zero3
Nobody has shared their experience with sdft_zero3 yet.