H20

AI overview

Headlines · 3

Nov 3
Trade-offs in Distributed Inference for Attention and MoE
The report analyzes distributed-inference strategies across tensor, data, expert, context, and pipeline parallelism using weight, KV-cache, activation, communication, capacity, and latency budgets. Its author argues that GQA can shard KV heads up to the KV-head count, while MLA's shared latent cache must be replicated under tensor parallelism, and compares MoE tensor parallelism with expert parallelism in terms of communication, memory use, load balancing, and implementation complexity. The report title labels it installment 2, while the body calls it the third of eight installments in an inference-infrastructure methodology series. Its comparative figures and conclusions are the author's analysis and are not independently corroborated by the supplied evidence.
掘金 · 人工智能 · 🔥 0
Nov 3
Roofline for LLM Inference: Performance Budgets from Compute and Bandwidth
A personal-blog article, presented as the first installment of an eight-part “Inference INFRA Methodology” series, applies the Roofline model to GPU inference. It estimates latency with T ≥ max(W/F, Q/BW) and effective performance with P ≤ min(F, I×BW), using arithmetic intensity, the hardware ridge point and actual HBM traffic to diagnose GEMM and attention bottlenecks. H20 examples and the article’s own measurements distinguish when quantization, batching and operator fusion may help. A series schedule covers communication, distributed inference, hardware, attention, MoE, PD/EPD and core optimizations. The benchmarks are self-reported and have not been independently verified here.
掘金 · 人工智能 · 🔥 0
Oct 13
Nvidia H200 China orders surge; reported TSMC capacity talks
TechNews reported that Chinese tech companies' demand for Nvidia chips remains strong despite a recent easing of some U.S. H200 sales restrictions, with unnamed sources saying Chinese firms have ordered more than 2 million H200 chips for 2026 against Nvidia inventory of about 700,000 units. The report said Nvidia is sounding out TSMC about expanding H200 capacity, and that an eight-chip H200 module costs about 1.5 million yuan, offers roughly six times the performance of H20 and is about 15% cheaper than gray-market alternatives. It also reported that ByteDance plans to raise its 2026 Nvidia chip budget to about 100 billion yuan, conditional on Chinese approval of H200 imports, while Tencent is reportedly issuing $5 billion in bonds and has a five-year, roughly $7 billion Oracle lease for about 100,000 high-end AI chips via Southeast Asian data centers; Trump has said he would allow H200 sales to approved Chinese customers with a 25% fee, but Beijing is still reviewing imports and requires new state-funded data centers to use domestic AI chips.
科技新报 · 🔥 0

Experience and discussion from the community

Parallel Agent Teams: Small-Team Collaboration, Multi-Team Exploration

Optimizing a GPU kernel or completing a set of formal proofs often requires repeated trial and error: try a different implementation, run an evaluation, and keep iterating in whichever direction proves effective. When giving a model more test-time compute, the most direct approach is to let more agents try simultaneously; if the task only needs one best result, the value of the other attempts lies…

Agent PAT 测试时计算多智能体 Benchmark