Garlic LLM让破解PS4离线运行Qwen3.5-9B
AI 总结
Leonardo Alegría发布开源项目Garlic LLM,可在已破解PS4的GPU上离线运行90亿参数的Qwen3.5-9B。该项目将权重重新打包为定制的4位和3位格式,占用约4.5 GiB,并使用Qwen3.5的MTP头进行推测解码;开发者称文本聊天速度为每秒31至39个token,代码生成最高为每秒43个token。Tom’s Hardware报道称,开发者测试了运行11.52固件的base或Slim PS4,但性能测试未说明提示词和方法,PS4 Pro也尚未测试。
深度见解
Garlic LLM展示了通过定制量化和推测解码,让90亿参数模型完全运行在旧游戏主机GPU上的具体开源方案。
| I made this, so a quick note: it’s my open-source project. Qwen3.5-9B runs on a stock PS4 at 31–39 tok/s. The MTP head that ships with Qwen3.5 made the biggest difference: it drafts up to two tokens and the full model checks them in one pass, which took it from 22 to 31–39 tok/s on this old GPU. Weights are re-packed into custom 4- and 3-bit formats to fit about 4.5 GiB. Your old PS4 can now run a 9B AI, offline, on its own GPU. It talks, listens, and writes code. No cloud, no subscription, no Linux. Free and open source. Code: https://github.com/leonardo-alegria/garlic-llm Direct download link: https://huggingface.co/garlic-llm/garlic-llm-ps4 submitted by /u/leonardo_alegria [link] [comments] |
#Garlic LLM#Qwen3.5-9B#PlayStation 4#GoldHEN#Chatterbox#Parakeet TDT 0.6B v3#OpenOrbis PS4