AWQ
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- Embedl reports Qwen3.8-27B deployment at 40 tok/s on Jetson Thor
Embedl reported a Qwen3.8-27B deployment on NVIDIA Jetson Thor using INT4 weights, an MTP head for speculative decoding, FlashHead, and TensorRT Edge-LLM 0.10.1. Under its stated batch-one test setup, the company reported 39.6 tokens per second for decoding, 23.6 end to end, 926 tokens per second for prompt processing, and peak memory of 24.8 GiB. The r/LocalLLaMA post also reports near-FP16 benchmark results, but the deployment and performance figures are vendor claims without independent confirmation.
r/LocalLLaMA · 🔥 0
Experience and discussion from the community
Share my AWQ experienceAsk about AWQ
Nobody has shared their experience with AWQ yet.