tinybrain
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- tinybrain opens Chinese model training code and 60.26B-token dataset
The tinybrain project author open-sourced code and a cleaned Chinese pretraining dataset assembled from 26 public sources, with the full tier containing 60.26 billion tokens. The project documents model designs including mHC residual streams, Engram lookup memory, sliding-window attention, MoE, and a post-training workflow spanning SFT, distillation, and reinforcement learning. The author says usable model weights are not yet complete, while the reported loss and throughput figures are preliminary and self-reported.
掘金 · 人工智能 · 🔥 0
Experience and discussion from the community
Share my tinybrain experienceAsk about tinybrain
Nobody has shared their experience with tinybrain yet.