Intel Optane
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 2
- RTX 3090s and RAM drive boost Kimi K3 on second-gen Optane
A homelabber added two RTX 3090s to a second-generation Intel Optane system running the quantized 2.8-trillion-parameter Kimi K3 model. The builder reported 3.11 tokens per second for generation and 16.8 tokens per second for prompt evaluation with 45 of 94 shards in a tmpfs RAM drive, versus 2.20 and 13.3 with Optane alone. Tom’s Hardware noted that persistent Optane memory still required additional cooling fans because it became very hot.
Tom’s Hardware · 🔥 6 - Reddit User Runs Kimi K3 Without a GPU Using Intel Optane Memory
Reddit user u/IndividualSurvival reported running the 2.8-trillion-parameter Kimi K3 model with llama.cpp on a 40-core third-generation Intel Xeon Scalable system using discontinued Intel Optane persistent memory, without a graphics card. The system used App Direct mode to present DRAM and Optane as separate address spaces. With 1-bit quantization, generation increased from about 0.85 to 1.50 tokens per second after DAX was disabled, allowing frequently used data to benefit from DRAM caching. A 3-bit test achieved 0.75 tokens per second and took about 18.5 hours to generate 50,000 tokens.
IT之家 AI · 🔥 0
Experience and discussion from the community
Share my Intel Optane experienceAsk about Intel Optane
Nobody has shared their experience with Intel Optane yet.