Intel B70 GPU
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- llama.cpp XMX matrix engine refactor nearly doubles B70 speculative decoding speeds
A Reddit user on r/LocalLLaMA reports that llama.cpp pull request #29864, which refactors Intel XMX matrix engines and is attributed to budiony, made a huge improvement to speculative decoding speeds on an Intel B70 GPU running SYCL. In before/after runs on Muse Glimmer Q6_K (30B) with DFlash2 across nine prompts, throughput rose from 38.0 to 73.0 tok/s on code_python and aggregate wall time fell from 46.86s to 26.68s. The poster notes the acceptance rate dropped from 0.7845 to 0.363 because cheaper token verification allowed n-max to be increased; the numbers are a single, unverified user report.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my Intel B70 GPU experienceAsk about Intel B70 GPU
Nobody has shared their experience with Intel B70 GPU yet.