MTP
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 5
- Community tester: DFlash2 for Qwen3.8-Flash-Next gives +17% decode speed in llama.cpp v0.8.0
A Reddit tester in r/LocalLLaMA reports that llama.cpp v0.8.0 is the first release supporting DFlash2 drafters for Qwen3.8-Flash-Next, after quantizing their own GGUFs of the drafter. Testing on an Apple M5 Max with Q4_K_XL showed a geometric mean of 87.9 tps for DFlash2 versus 75.1 tps for MTP, a 17% gain, with code prompts at 95.1 tps (+18%) and list prompts at 127 tps (+39%) but prose slightly lower at 51.9 tps (-3%). At around 25K context DFlash2 roughly equalled MTP with some drop-off, and the tester noted its Q4_0 draft head is only about 350MB versus 2.2GB for the ggml MTP head, though the required compute buffer offsets the VRAM savings.
r/LocalLLaMA · 🔥 2 - UkisAI releases two Swift 1.5 reasoning-efficiency models
UkisAI announced Swift 1.5 GLM-5.3-Flash and Swift 1.5 Ornith-9B, with GGUF and other quantized weights available for local inference. The company says GLM-5.3-Flash cut thinking-token usage by 61.4% on MMLU-PRO with a 0.46% average accuracy loss, while Ornith-9B reduced thinking-token usage by 33.5% without an accuracy loss. These results are vendor claims based on UkisAI's own benchmarks and have not been independently verified.
r/LocalLLaMA · 🔥 0 - Embedl reports Qwen3.8-27B deployment at 40 tok/s on Jetson Thor
Embedl reported a Qwen3.8-27B deployment on NVIDIA Jetson Thor using INT4 weights, an MTP head for speculative decoding, FlashHead, and TensorRT Edge-LLM 0.10.1. Under its stated batch-one test setup, the company reported 39.6 tokens per second for decoding, 23.6 end to end, 926 tokens per second for prompt processing, and peak memory of 24.8 GiB. The r/LocalLLaMA post also reports near-FP16 benchmark results, but the deployment and performance figures are vendor claims without independent confirmation.
r/LocalLLaMA · 🔥 0 - Developer reports 190 tok/s for Qwen3.8-27B on two RTX 3090s
A developer reports that the Geezuz dual-GPU CUDA kernel ran Qwen3.8-27B on two power-limited RTX 3090s at 190 tok/s for one request and an aggregate 622 tok/s across eight requests. The open-source project splits computation across two GPUs over PCIe, supports tensor-core quantization paths and MTP drafting, and includes an OpenAI-compatible server, but its speed and output-quality comparisons have not been independently verified.
r/LocalLLaMA · 🔥 0 - vLLM brings adaptive verification to draft models without a confidence head
vLLM's blog says Adaptive Verification now supports all draft-model-based speculator types, not just DSpark drafters that ship with a confidence head. For drafters lacking one, such as EAGLE, MTP, DFlash and some DSpark checkpoints, vLLM uses a lightweight logistic regression that maps the log-odds of the drafter's most likely token to the target model's acceptance probability, with one shared slope plus one intercept per draft position (K+1 parameters) fitted online from the accept/reject decisions produced during normal verification. The feature landed in PR #52228 and is enabled with enable_adaptive_verification; on DeepSeek-V4-Flash with DSpark the estimator beat the trained confidence head on AUC and log-loss but had worse ECE, and the two were nearly tied on calibration once the first draft position was excluded. Stated limitations include requiring a target attention backend that supports variable-length verification, and smaller gains for MTP and EAGLE than for DFlash and DSpark.
vLLM Blog · 🔥 0
Experience and discussion from the community
Share my MTP experienceAsk about MTP
Nobody has shared their experience with MTP yet.