Ling-3.0-flash

Headlines · 2

1. 4d
   NVIDIA Boosts Ling-3.0-flash MoE Inference on DGX Spark

   An NVIDIA Technical Blog post details three optimizations for Ling-3.0-flash on NVIDIA DGX Spark: the Humming MXFP4 MoE backend, online FP8 quantization of the LM head, and NEXTN multi-token-prediction speculative decoding. Profiling showed batch-size-1 decoding was memory-bound, with weight reads dominating, and the Mixture-of-Experts computation plus the output-layer projection standing out as the costliest operations. On a synthetic workload the baseline ran at 27.71 ms TPOT and 36.09 tok/s, while NEXTN MTP reached 9.72 ms and 102.88 tok/s; across the full 1,319-question GSM8K evaluation the three optimizations together raised decode throughput 42.5% and cut mean end-to-end latency 24.5%, with no statistically detectable accuracy change (96.89% to 96.82%). The post also warns that combining online FP8 LM-head quantization with MTP requires the draft model to share the entire LM-head module, otherwise the framework silently falls back to a configuration slower than baseline.

   NVIDIA Technical Blog · 🔥 2
2. Oct 22
   InclusionAI releases Ling-3.0-flash-Sante, a 124B healthcare model

   InclusionAI has released Ling-3.0-flash-Sante, a healthcare-specialized model built on Ling-3.0-flash, according to a model card reposted on r/LocalLLaMA. The model has 124B total parameters with 5.1B active per token, a 256K-token training context, and a hybrid-linear MoE architecture combining Kimi Delta Attention, gated MLA and sparse MoE layers. Weights are available on Hugging Face and ModelScope; InclusionAI says Sante performs well on healthcare reasoning, knowledge and safety benchmarks, but those are vendor claims without independent verification, and the company describes it as a research model, not a medical device, whose outputs may be incomplete or incorrect.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my Ling-3.0-flash experienceAsk about Ling-3.0-flash