创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

Reddit 用户用 Strata 运行 Qwen Flash Next

AI 总结

一名 Reddit 用户称,在配备 64GB 内存的 M1 Max MacBook Pro 上,通过 Strata 本地运行 Qwen Flash Next 的速度达到每秒 24 个 token。其表示该模型完成基础任务表现出色,但无法与更大的前沿模型竞争;帖子还提到 MoEShift 和将大型模型卸载至 SSD。

深度见解

这为 Strata 与 SSD 卸载技术扩大消费级硬件上的本地模型选择提供了早期部署信号,但结论来自个人体验,尚未获得受控基准测试佐证。

I have been looking for a local model to do some of the typical work I used AI subscriptions for.

I’ve tested almost every popular model. I have an M1 Max 64gb MacBook Pro as the inference machine and because of the memory size, I wasn’t able to load anything very effective in the tasks I use it for.

Found out about strata and have been testing it out. It’s awesome! I’m kinda surprised more people don’t talk about strata on here. I feel like a lot of us are in that limbo area where we have hardware that is more powerful than needed to run a sub 30b model like Qwen 27b, but not large enough to run models like Qwen Flash Next.

But thanks to open source research, I’m now getting Qwen Flash Next to run at 24 tok/s! And it’s doing phenomenal work! I know it’s not going to compete with the larger frontier models, but I don’t think I necessarily “need” frontier model intelligence. I just need enough to do basic things while I can handle the rest.

Seriously, check out Strata and also MoEShift that someone posted about on here. It’s awesome to be able to offload larger models onto SSD. I hope this offloading technique grows and more research is done on it because I genuinely think this could be an amazing way to get these frontier level open source models to work on regular consumer hardware.

#Qwen Flash Next#Strata#MoEShift#Qwen 27B#M1 Max#MacBook Pro

原文约 1 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。