SYCL

Headlines · 1

1. Oct 22
   llama.cpp XMX matrix engine refactor nearly doubles B70 speculative decoding speeds

   A Reddit user on r/LocalLLaMA reports that llama.cpp pull request #29864, which refactors Intel XMX matrix engines and is attributed to budiony, made a huge improvement to speculative decoding speeds on an Intel B70 GPU running SYCL. In before/after runs on Muse Glimmer Q6_K (30B) with DFlash2 across nine prompts, throughput rose from 38.0 to 73.0 tok/s on code_python and aggregate wall time fell from 46.86s to 26.68s. The poster notes the acceptance rate dropped from 0.7845 to 0.363 because cheaper token verification allowed n-max to be increased; the numbers are a single, unverified user report.

   r/LocalLLaMA · 🔥 0

Experience and discussion from the community

Share my SYCL experienceAsk about SYCL