EGGROLL
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Q Labs introduces Dust for Transformer pretraining without backpropagation
Q Labs introduced Dust, a zeroth-order optimization method that estimates update directions from random, token-level activation perturbations instead of standard backpropagation. The company reported training Transformers on FineWeb up to 243M parameters and 20M tokens, while a SOTAAZ Lab reproduction found basic reproducibility but no clear advantage over backpropagation. Dust also remained far more computationally expensive than backpropagation.
机器之心 · 🔥 0
Experience and discussion from the community
Share my EGGROLL experienceAsk about EGGROLL
Nobody has shared their experience with EGGROLL yet.