Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Crowd-v1 Advances Community-Led 1B-Parameter Model Training

AI summary

The CrowdGPT project reports that 85 registered contributors trained 13 billion tokens in one week as work continued on Crowd-v1, an open-source, community-driven transformer designed to run without datacenters. The project aims to match or exceed Qwen3-0.6B on benchmarks, but that performance target has not been demonstrated. CrowdGPT describes the effort as work in progress and invites contributions to its training mechanics and datasets.

In-depth insight

The update provides a notable community-scale training experiment, although its participation and token figures are self-reported and its benchmark goal remains unverified.

Processing img 0ybf7gn4c90i1...

Hello r/LocalLLaMA :D

This is yet another follow-up post on the project CrowdGPT, which aims at creating the first collaboratively trained 1 billion parameters transformer, Crowd-v1, without any datacenters, and for 0$. The project's goal is to show we don't need huge Nvidia GPU clusters nor millions of dollars to train a competitive Large Language Model.

The goal of the project is to create a 1B parameters model, that would match or exceed Qwen3-0.6B's performance on benchmarks as a proof-of-concept, we could then scale with a 7B A1B MoE or even a 30B A3B MoE once we get enough contributors. The size of the models trained is community driven.

This project is obviously 100% non-profit and open-source. The goal is to propose a new paradigm of AI where we don't wait on Chinese/American AI labs to provide us open-weight models, and we just create the models ourselves.

From the previous post we achieved to get 85 contributors registered on the project, leading to 13B tokens trained in a week 🤯 Thanks to every contributor who trusted the early-stage project, and imagine what 1000+ people with idle GPUs could train!

#CrowdGPT#Crowd-v1#Qwen3-0.6B

Original · 1 min readRead the original
Group chat

No comments yet. Start the conversation.