Generating tokens faster starts with identifying what limits inference performance. In the batch-size-1 evaluation of Ling-3.0-flash from Inclusion AI on NVIDIA DGX Spark, profiling showed that decoding was memory-bound: reading model weights drove execution time. Two operations stood out—the mixture-of-experts (MoE) computation and the output-layer projection that produces scores across the model’s vocabulary.

Those findings pointed to two ways to improve performance: reduce the weight traffic in each decode step and generate more tokens per verification round. This post examines three techniques that address those costs:

• •Humming MXFP4 MoE kernels to improve execution of the routed-expert computation.
• •Online FP8 quantization of the output layer, or LM head, to reduce its weight-read volume.
• •Multi-token prediction (MTP) speculative decoding with NEXTN to draft and verify multiple tokens together.

This post explains how each optimization works, measures its effect, and examines an integration issue that can make the combined configuration slower than the baseline. Ling-3.0-flash serves as the implementation example, with code and deployment instructions in the Ling cookbook.

Model and baseline configuration

The evaluation used Ling-3.0-flash-fp4, a mixture-of-experts model with 124 billion total parameters and 5.1 billion active parameters per token. The model has 42 layers: 35 Kimi Delta Attention (KDA) linear-attention layers and seven multi-head latent attention (MLA) layers. Each MoE layer routes each token to eight of 512 routed experts, alongside one shared expert.