grokking
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- PBT rescued grokking but did not speed it up
A Reddit experiment tested Population Based Training (PBT) on modular addition mod 97 using a small MLP, with learning rate and weight decay adapted across 20 members. In a reported “dead zone” where no fixed setting grokked, fast PBT succeeded in 3 of 3 runs at about 6,850 steps, while fixed and slow PBT succeeded in 0 of 3. Under equal compute, however, the best fixed grid cell reached 95% validation accuracy in 317 ± 29 steps, compared with 2,667 ± 2,244 for PBT.
r/MachineLearning · 🔥 0
Experience and discussion from the community
Share my grokking experienceAsk about grokking
Nobody has shared their experience with grokking yet.