Puro-2B
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Tsinghua open-sources Puro-2B: 2B pretraining on RTX 5090s for about $4,400
A Tsinghua University team led by professors Chen Wenguang and Lyu Kaifeng released Puro-2B, a fully open pretraining recipe that publishes the training data, framework, configuration and model weights. The roughly 2B-parameter model was trained from scratch on a cluster of consumer RTX 5090 GPUs using up to 1.4 trillion tokens; on a 15-task evaluation, a version with about $4,400 of pretraining compute beat Qwen2-1.5B on average, while the $6,900 best version came close to Qwen2.5-1.5B on average. The team also fit a Puro Cost Scaling Law to estimate achievable quality at a given budget, combining hardware choice, FP8 training, the MuonH optimizer, Curriculum Model Averaging and data-slice evaluation.
36氪 人工智能 · 🔥 8
Experience and discussion from the community
Share my Puro-2B experienceAsk about Puro-2B
Nobody has shared their experience with Puro-2B yet.