Analysis: "Tokens per Watt" pitched as an edge-AI metric beyond TOPS
A Juejin article argues that peak TOPS no longer predicts on-device AI experience, proposing tokens per watt as a better measure of sustainable inference efficiency. It compares Qualcomm's heterogeneous CPU/GPU/NPU orchestration with MediaTek's efficiency-first APU scheduling and cloud offloading. The piece also frames edge AI as shifting from bursty, on-demand response to always-on, multimodal agent workloads, and discusses the power-performance-cost trade-off in edge chip design.
If sustained token throughput per watt becomes the headline metric for phone and PC chips, chip marketing, model deployment and quantization choices could all be evaluated on a different axis than today, though the article's forecast is editorial rather than an industry consensus.
1. 引言:当端侧 AI 开始「算账」
过去几年,端侧芯片的宣传口径几乎被一个数字统治——TOPS(每秒万亿次运算)。从手机 SoC 到 PC 处理器,再到各类 AI 加速卡,厂商们乐此不疲地把 TOPS 越堆越高,仿佛只要算力足够大,端侧 AI 就能水到渠成。
但真正把大模型塞进手机、PC、车机和可穿戴设备之后,行业很快发现:TOPS 高不等于体验好。一台标称 40 TOPS 的设备,跑起 7B 模型可能只有个位数 token/s,风扇狂转、电量狂掉;而另一台标称 20 TOPS 的设备,反而能流畅地长时间对话。
于是,一个新的指标开始被反复提及——每瓦 Token 数(Tokens per Watt)。它衡量的不是「峰值能算多快」,而是「每一瓦功耗能稳定产出多少有效 token」。这个看似简单的换算,正在把端侧芯片的设计逻辑从「堆算力」推向「算效率」。
本文围绕四个问题展开:
- 「每瓦 Token 数」为何会取代 TOPS 成为端侧 AI 的核心指标;
- 高通、联发科等厂商的端侧芯片路线如何围绕这一指标演进;
- 设备从「被动响应」走向「主动服务」,对算力提出了哪些新要求;
- 端侧芯片如何在功耗、性能、成本这个三角中寻找平衡。