CrowdGPT众包训练十亿参数模型
AI 总结
开发者Vxtzq1介绍了CrowdGPT,该项目计划让参与者使用个人电脑协作训练一个十亿参数LLM,而非依赖数据中心。开发者称,开源项目将公开训练数据和模型权重,接受干净数据提交,并允许参与者参与项目方向决策;数据提交流程据称已准备就绪。当前模型被定位为概念验证,项目细节与相关能力尚未获得独立验证。
深度见解
CrowdGPT提出了一种去中心化、社区协作的LLM训练路线,但其技术可行性与开放数据主张仍有待验证。
| Hello, I'm developing CrowdGPT, a project aiming to create an LLM without datacenters and without the usual millions of dollars spent by AI companies. The project aims to train a 1 billion parameter model collaboratively and openly, with a protocol where everyone gets a say in the direction the project takes. Anyone can use their PC to train the AI independently, then add their contributions to the global model. Unlike most American and Chinese AI companies, the project is not only open-source, but also makes 100% of its training data public (along with the weights). Anyone can submit their own clean data to train the model, and even help steer the project in new directions. This creates the paradigm of an AI we all have a stake in. The project is open to as many people as possible, which is why I'd like to ask: what's stopping you from contributing to the project? I want to hear your answers so I can improve it. Are people simply willing to dedicate some of their GPU time to training a cool new open-source model? Or do they want something in return (like free inference credits on the crowdgpt model for example)? Although the current model is somewhat small (1 billion parameters), I plan to make it bigger / better in the future, and its current size is only meant as a proof of concept. Any advice / feedback on the project is welcome. Here's the project website: https://www.crowdgpt.net/ Or the GitHub if anyone wants to check out the code 😂 https://github.com/Vxtzq/CrowdGPT Thanks in advance. PS: I hope this post follows the subreddit's rules! My main goal is to gather criticism and understand how to make the project more accessible, not to advertise it 😅 EDIT: The data submission pipeline for the project (i'm mentioning in some replies) is ready!! Come check it at https://www.crowdgpt.net/#submit submitted by /u/Vxtzq1 [link] [comments] |