Artificial Analysis makes Time per Task its headline API metric
Artificial Analysis announced that Time per Task is now the headline metric in its API performance benchmark, rather than streaming speed. The update applies to roughly 1,000 serverless endpoints. End-to-end output speed is measured from request submission to the final token, including prefill, hidden reasoning and generation. Every measurement now includes short, document-length and long-context prompts, plus multi-turn coding and knowledge-work sessions; headline figures are five-day averages. The benchmark tracks each provider’s API default effort alongside highest effort for key models, because effort labels are not comparable across labs.
Artificial Analysis says task-completion time is more relevant as workloads become more agentic. Its new end-to-end measure captures delays that decoding speed alone can obscure, including time before the first token and the effect of verbose output.
Announcing the next version of Artificial Analysis API Performance Benchmarking: Time per Task is now our headline performance metric, measuring how long a model takes to complete a task rather than how fast it streams
The update applies to the ~1,000 serverless endpoints we benchmark multiple times a day for speed, latency and caching.
Key changes: ➤ Time per Task is now the headline metric to compare models: as workloads become more agentic, the time to complete a task matters more than speed per token. A model that is fast per token but verbose can take longer than a slower model that uses fewer tokens. Time per Task divides a model's output tokens per Intelligence Index task by its End-to-End Output Speed.
➤ End-to-End Output Speed covers the full request: speed is now measured from sending the request to receiving the final token. That includes the wait before the first token (prefill, and reasoning for models that hide it) as well as generation. Output tokens now come from each provider's API-reported counts instead of the o200k_base tokenizer, and we monitor for discrepancies in reported counts.
#Artificial Analysis#GPT-6 Astra#GPT-6.1 Sol#Claude Opus 5.5#Gemini 4 Carbon#OpenAI#Anthropic#Google