Llama-3.3-70B
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 1
- SparseDecoding: decoding-aware pruning for faster LLM inference
The paper introduces SparseDecoding, a training-free network pruning framework aimed at the decoding stage of LLM inference. It builds calibration matrices from layer-wise activations collected during dense-model autoregressive generation (excluding prefill) so the pruning objective matches decoding activations, and pairs this with an N:M sparse matrix-vector kernel using bitmask indexing and fixed-step traversal. The authors report that on Llama-3.1-8B, Llama-3.3-70B and Qwen3-14B/32B it outperforms fixed-text calibration on long-form generation benchmarks, with up to 1.48x end-to-end wall-clock decoding speedup on A100 GPUs.
Hugging Face · Papers · 🔥 0
Experience and discussion from the community
Share my Llama-3.3-70B experienceAsk about Llama-3.3-70B
Nobody has shared their experience with Llama-3.3-70B yet.