Artificial Analysis Endpoint Accuracy Index v1.1 expands coverage
Artificial Analysis launched Endpoint Accuracy Index v1.1, expanding coverage to six open-weight models, including GLM-5.3 Flash and Muse Spark Open. The index compares provider endpoints with a self-hosted reference using tool calling, scientific reasoning and long-context recall; its updated AA-LCR v1.1 uses 25 repeats and narrows the average 95% confidence interval from ±4.7 to ±2.6 points.
The measurements provide developers with an accuracy comparison across more than 100 serverless endpoints, highlighting how output caps, reduced reasoning and tool-argument handling can affect deployed model performance.
Launching Artificial Analysis Endpoint Accuracy Index v1.1: a minor version upgrade alongside expanded coverage for GLM-5.3 Flash and Muse Spark Open. Across the 100+ serverless endpoints we benchmarked, nearly 1 in 4 score below the reference
Providers trade off accuracy to optimize for speed and cost. They quantize weights, write custom kernels, tune their inference stacks and cap output lengths, and sometimes they simply ship bugs. The Endpoint Accuracy Index measures how much of a model’s accuracy each endpoint preserves, so developers can pick providers on accuracy, not just price and speed.
We score each endpoint as a percentage of our self-hosted reference deployment of the official weights, across tool calling (BFCL-500), scientific reasoning (HLE-250) and long context recall (AA-LCR-25), equally weighted.
What’s changed:
➤ Upgraded AA-LCR-25 to AA-LCR v1.1 and increased repeats from 10 to 25: AA-LCR v1.1 corrects errors and ambiguities in answer keys and moves grading to GPT-5.6 Luna (medium). Running each question 25 times, up from 10, reduces the average 95% confidence interval from ±4.7 to ±2.6 points
➤ Expanded coverage to six open weights models: We add GLM-5.3 Flash (max) and Meta’s Muse Spark Open (xhigh), alongside DeepSeek V4.1 Flash 1017 (max), GLM-5.3 (max), Kimi K3 (max) and gpt-oss-120b (high)
#Artificial Analysis#Endpoint Accuracy Index v1.1#AA-LCR v1.1#GLM-5.3 Flash#Muse Spark Open#DeepSeek V4.1 Flash 1017#GLM-5.3#Kimi K3#gpt-oss-120b#GPT-5.6 Luna