Crowd-v1推进社区协作式10亿参数模型训练
CrowdGPT项目称,已有85名注册贡献者在一周内完成130亿个token的训练,目标是推进无需数据中心的开源、社区协作式Crowd-v1。项目希望该模型在基准测试中达到或超过Qwen3-0.6B,但目前尚未展示这一结果。项目仍处于开发阶段,并邀请社区参与训练机制和数据集建设。
这是一项社区规模化训练实验,但参与人数和token数量均为项目方自行报告,基准性能目标也尚未得到验证。
Processing img 0ybf7gn4c90i1...
Hello r/LocalLLaMA :D
This is yet another follow-up post on the project CrowdGPT, which aims at creating the first collaboratively trained 1 billion parameters transformer, Crowd-v1, without any datacenters, and for 0$. The project's goal is to show we don't need huge Nvidia GPU clusters nor millions of dollars to train a competitive Large Language Model.
The goal of the project is to create a 1B parameters model, that would match or exceed Qwen3-0.6B's performance on benchmarks as a proof-of-concept, we could then scale with a 7B A1B MoE or even a 30B A3B MoE once we get enough contributors. The size of the models trained is community driven.
This project is obviously 100% non-profit and open-source. The goal is to propose a new paradigm of AI where we don't wait on Chinese/American AI labs to provide us open-weight models, and we just create the models ourselves.
From the previous post we achieved to get 85 contributors registered on the project, leading to 13B tokens trained in a week 🤯 Thanks to every contributor who trusted the early-stage project, and imagine what 1000+ people with idle GPUs could train!