910B

Headlines · 1

1. Nov 3
   Roofline for LLM Inference: Performance Budgets from Compute and Bandwidth

   A personal-blog article, presented as the first installment of an eight-part “Inference INFRA Methodology” series, applies the Roofline model to GPU inference. It estimates latency with T ≥ max(W/F, Q/BW) and effective performance with P ≤ min(F, I×BW), using arithmetic intensity, the hardware ridge point and actual HBM traffic to diagnose GEMM and attention bottlenecks. H20 examples and the article’s own measurements distinguish when quantization, batching and operator fusion may help. A series schedule covers communication, distributed inference, hardware, attention, MoE, PD/EPD and core optimizations. The benchmarks are self-reported and have not been independently verified here.

   掘金 · 人工智能 · 🔥 0

Experience and discussion from the community

Share my 910B experienceAsk about 910B