Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Analysis: "Tokens per Watt" pitched as an edge-AI metric beyond TOPS

AI summary

A Juejin article argues that peak TOPS no longer predicts on-device AI experience, proposing tokens per watt as a better measure of sustainable inference efficiency. It compares Qualcomm's heterogeneous CPU/GPU/NPU orchestration with MediaTek's efficiency-first APU scheduling and cloud offloading. The piece also frames edge AI as shifting from bursty, on-demand response to always-on, multimodal agent workloads, and discusses the power-performance-cost trade-off in edge chip design.

In-depth insight

If sustained token throughput per watt becomes the headline metric for phone and PC chips, chip marketing, model deployment and quantization choices could all be evaluated on a different axis than today, though the article's forecast is editorial rather than an industry consensus.

1. 引言:当端侧 AI 开始「算账」

过去几年,端侧芯片的宣传口径几乎被一个数字统治——TOPS(每秒万亿次运算)。从手机 SoC 到 PC 处理器,再到各类 AI 加速卡,厂商们乐此不疲地把 TOPS 越堆越高,仿佛只要算力足够大,端侧 AI 就能水到渠成。

但真正把大模型塞进手机、PC、车机和可穿戴设备之后,行业很快发现:TOPS 高不等于体验好。一台标称 40 TOPS 的设备,跑起 7B 模型可能只有个位数 token/s,风扇狂转、电量狂掉;而另一台标称 20 TOPS 的设备,反而能流畅地长时间对话。

于是,一个新的指标开始被反复提及——每瓦 Token 数(Tokens per Watt)。它衡量的不是「峰值能算多快」,而是「每一瓦功耗能稳定产出多少有效 token」。这个看似简单的换算,正在把端侧芯片的设计逻辑从「堆算力」推向「算效率」。

本文围绕四个问题展开:

  • 「每瓦 Token 数」为何会取代 TOPS 成为端侧 AI 的核心指标;
  • 高通、联发科等厂商的端侧芯片路线如何围绕这一指标演进;
  • 设备从「被动响应」走向「主动服务」,对算力提出了哪些新要求;
  • 端侧芯片如何在功耗、性能、成本这个三角中寻找平衡。

#Qualcomm#MediaTek#Hexagon

Original · 10 min readRead the original
Group chat

No comments yet. Start the conversation.