Artificial Analysis以每项任务耗时为核心API指标
Artificial Analysis宣布,其API性能基准测试将以“每项任务耗时”取代流式输出速度,作为核心指标。此次更新适用于约1000个无服务器端点。端到端输出速度从提交请求计至收到最终令牌,涵盖预填充、隐藏推理和生成过程。每项测量均加入短提示、文档长度及长上下文提示,以及多轮编程和知识工作会话;核心数据为过去5天的平均值。基准测试还会在关键模型上同时追踪各厂商API的默认努力档与最高努力档,因为不同实验室的努力档名称不可直接比较。
Artificial Analysis表示,随着工作负载日益偏向智能体应用,完整任务耗时比单看输出速度更具参考价值。新的端到端指标纳入了单看解码速度可能掩盖的首令牌前等待、隐藏推理时间及冗长输出的影响。
Announcing the next version of Artificial Analysis API Performance Benchmarking: Time per Task is now our headline performance metric, measuring how long a model takes to complete a task rather than how fast it streams
The update applies to the ~1,000 serverless endpoints we benchmark multiple times a day for speed, latency and caching.
Key changes: ➤ Time per Task is now the headline metric to compare models: as workloads become more agentic, the time to complete a task matters more than speed per token. A model that is fast per token but verbose can take longer than a slower model that uses fewer tokens. Time per Task divides a model's output tokens per Intelligence Index task by its End-to-End Output Speed.
➤ End-to-End Output Speed covers the full request: speed is now measured from sending the request to receiving the final token. That includes the wait before the first token (prefill, and reasoning for models that hide it) as well as generation. Output tokens now come from each provider's API-reported counts instead of the o200k_base tokenizer, and we monitor for discrepancies in reported counts.
#Artificial Analysis#GPT-6 Astra#GPT-6.1 Sol#Claude Opus 5.5#Gemini 4 Carbon#OpenAI#Anthropic#Google