DeepSeek has open-sourced DeepInfer, the inference engine behind its service, together with kernels used by the system. LMSYS tested SGLang against DeepInfer using DeepInfer’s benchmarks on the same hardware, reporting up to 1.04x prefill throughput on four NVIDIA GB300 or four B300 GPUs and decode steps up to 7.4% shorter on 64 GB300 GPUs. The comparison is not fully equivalent because request-to…
DeepSeek announced on October 20 that it is fully open-sourcing DeepInfer, its internal inference framework, following earlier releases of a series of large models, the TileLang compiler tooling and inference operator libraries. The company says DeepInfer bundles high-performance operators and a deeply optimized inference engine supporting distributed inference, speculative decoding, SWA Bounded R…
Tencent Hunyuan released Hy Memory 2.0, an agent memory framework, and open-sourced three layers: the framework SDK, the local MemHub memory app, and a dedicated memory-writer model, Hy-Memory-30B-A3B (~3B active parameters, Apache-2.0). According to Tencent and its accompanying paper, the model scores a 55.49% macro average across four harnesses (Mem0, MemOS, Hy-Memory and held-out EverMemOS) on …
Meta released Muse Spark Open, an open-weight mixture-of-experts model with roughly 816B total parameters and about 42B active per token, accepting text, image, audio and video input while generating text, with a context window of up to one million tokens. Meta says it is trained for agentic coding and long-horizon work such as reading codebases, writing and debugging code, calling tools, planning…
Technology Innovation Institute (TII) has introduced Falcon-ASR, a 1.6B-parameter speech recognition model focused on Arabic, including the Emirati dialect. TII says it supports English, French, Spanish and Portuguese, offers word-level timestamps, and achieved a 20.92% average WER across six Arabic test sets in its evaluation.
Why it matters: It adds a multilingual speech model with a specific f…