ReplaySSM

Headlines · 2

1. 3d
   RossiLH demos a lightweight KIMI K3 serving module

   RossiLH presented a CPU-based interactive demonstration of parts of the KIMI K3 serving module across 14 tabs. It covers KDA speculative replay, replay-consistency checks, ReplaySSM ring buffering, unified paging for MLA and KDA storage, two-stage prefix-cache lookup, engine comparisons, and Fleet scheduling with consistent-hash routing and budget isolation. The author describes the components as reduced-scale versions or approximations and presents the project as part of his learning process.

   掘金 · 人工智能 · 🔥 0
2. Nov 2
   LlamaIndex reports nearly 10x faster token generation for OCRia-1 on one B200

   LlamaIndex reports that its optimizations raised OCRia-1 token generation from about 285 to 2,800 tokens per second on a single NVIDIA B200 GPU, a vendor-reported result not independently verified here. The stack combines workload-aware mixed-precision quantization, DFlash2 speculative decoding, ReplaySSM, custom kernels and 17 additional optimizations. LlamaIndex says OCRia-1 is available through LlamaParse across all tiers and plans to contribute generally applicable improvements to vLLM.

   LlamaIndex Blog · 🔥 0

Experience and discussion from the community

Share my ReplaySSM experienceAsk about ReplaySSM