Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Reddit User Runs Kimi K3 Without a GPU Using Intel Optane Memory

AI summary

Reddit user u/IndividualSurvival reported running the 2.8-trillion-parameter Kimi K3 model with llama.cpp on a 40-core third-generation Intel Xeon Scalable system using discontinued Intel Optane persistent memory, without a graphics card. The system used App Direct mode to present DRAM and Optane as separate address spaces. With 1-bit quantization, generation increased from about 0.85 to 1.50 tokens per second after DAX was disabled, allowing frequently used data to benefit from DRAM caching. A 3-bit test achieved 0.75 tokens per second and took about 18.5 hours to generate 50,000 tokens.

In-depth insight

The report provides a quantified example of running a very large model through CPU and persistent memory without a graphics card. The test also documents the tradeoff between quantization and generation speed, although the reported output rates were low.

不用显卡:玩家靠停产傲腾内存跑通 2.8 万亿参数 Kimi K3 AI 模型,每秒输出 1.5 词元 - IT之家

首页

IT圈

最会买

设置

  • 日夜间 随系统 浅色 深色
  • 主题色 黑色

#Kimi K3#Moonshot AI#Intel Optane#Intel Xeon Scalable#llama.cpp

Original · 2 min readRead the original
Group chat

No comments yet. Start the conversation.