Artificial Analysis
AI overview
Sign in and the AI will write an overview from our coverage.
Headlines · 119
- DeepSeek V4.1 Flash 1017 debuts as open-weight value leader
Latent Space's AINews roundup reports that DeepSeek V4.1 Flash 1017 is a 552B-total, 16B-active MoE model with a 1M-token context window under an MIT license, priced unchanged at $0.30/$1.20 per million input/output tokens at peak, with off-peak pricing half that. Artificial Analysis scores it 45 on its Intelligence Index, up six points and tied with GLM-5.3 for second among open-weight models, while noting MiMo-V2.6-Pro and GPT-6.1 Sol (medium) score higher at lower cost per task. The model is available through DeepSeek's API, Fireworks, Baseten, DeepInfra, Cloudflare and Together, and ranks first among open models in Code Arena WebDev at No. 12 overall with 1,670 points; Together says it was trained with self-feedback RL, and release figures cite Terminal-Bench 4.0 rising from 31.2 to 42.3.
Latent Space · 🔥 0 - MiniMax M3.1 Flash benchmark gains spark training-data allegations
MiniMax M3.1 Flash earned an Artificial Analysis Intelligence Index score of 42, 13 points above M3, but researchers questioned whether the gain reflected genuine capability or memorized benchmark questions. Researcher Binkg reported that the model retained the original answer after a public AA-Omniscience question was modified, and TMTPost reproduced a similar English-language failure, although the model answered correctly in Chinese. MiniMax opened the model weights on October 23, enabling independent testing, but no request logs or training records show that the company accessed or used private evaluation questions.
cnBeta · 🔥 0 - MiniMax releases M3.1 Flash, confirms Space Bunny identity
MiniMax released M3.1 Flash on October 19 and confirmed that Space Bunny, an anonymous model that had drawn attention in developer communities, was an early version; MiniMax said Space Bunny processed more than 200 trillion tokens while unnamed. The company describes M3.1 Flash as a reasoning model with a one-million-token context window that accepts text and images, and says it runs nearly three times faster than M3 while cutting agent task costs by 70%. Reported results include first place on Zapier's Automation-Bench v1.0.6, 76.2% on BankerToolBench (tied with Claude Opus 5), 82.2% on CyberGym and 29.6% on Agents' Last Exam. The Decoder reported a 42-point Artificial Analysis Intelligence Index score, 13 above M3, while Chinese outlets reported 43. The Decoder listed API pricing at $0.15 per million input tokens and $0.60 per million output tokens; Chinese reports listed 1 yuan input and 4 yuan output per million tokens. MiniMax said it plans to release open weights on October 23.
The Decoder · 🔥 0 - Tomasz Tunguz: Anthropic looks like a cloud, and should be valued like one
In a blog post, venture investor Tomasz Tunguz argues that Anthropic should be assessed as a cloud business rather than only an AI model company, comparing its Q2 2026 revenue of $12 billion with AWS at $42 billion, Azure at $39 billion and Google Cloud at $25 billion. He cites 1,374% year-over-year growth, a $46 billion annual run rate and $518 billion in data-center commitments, while noting that two customers accounted for nearly 25% of 2025 revenue, per Reuters reporting from an unfiled prospectus. He also cites Ramp vendor data showing 87% of businesses buying a coding agent use Claude Code, and Artificial Analysis figures putting Claude Opus 5.5 at $5.98 per task versus $3.26 for GPT-6 Astra and $1.99 for Gemini 4 Argon.
Tomasz Tunguz · 🔥 0 - Mistral Large 4 preview tops non-Chinese, non-US models on Intelligence Index
The Batch reports that Mistral trained Mistral Large 4 on 3,800 GPUs in its own data centers, using data spanning more than 160 languages, and mixed many task types — single-turn chat, science problems, safety, factual accuracy and long-running tool use — into one reinforcement learning run with rewards from unit tests, reward models, LLM judges and code inspection without execution. On Artificial Analysis' Intelligence Index, the API preview in reasoning mode scores 38 at $1.13 per task, tying GPT-6 Luna and trailing MiMo-V2.6-Pro (46), GLM-5.3-Flash (42) and DeepSeek V4.1 Flash (39). On Artificial Analysis' Cyber Index, which measures finding and fixing vulnerabilities in code repositories, the preview reaches 50 percent, tying GLM-5.3-Flash. The Batch notes the RL run that produced the preview is still in process.
The Batch · DeepLearning.AI · 🔥 0 - Claude Fable 5.5 leaks in Azure catalog ahead of an expected launch
36Kr and TestingCatalog report that Claude Fable 5.5 appeared in Microsoft Azure's East US 2 model catalog, with a LiteLLM entry listing a 1M-token context window, up to 128K output tokens and the same $10/$50 per-million-token pricing as Fable 5.1. TestingCatalog cautions that the entry was removed a day later when Azure queries came back empty and that the values may have been carried over from Fable 5.1. A leaker said partners had been given access to the model and expected a release next week, while a "claude-onigiri-eap" codename briefly surfaced in Artificial Analysis's frontend; Anthropic has confirmed nothing.
量子位(原生 RSS) · 🔥 0 - Prime Intellect and Nvidia post-train Nemotron 3.5 Super VL for enterprise agents
Prime Intellect and NVIDIA built 2,500 verified training tasks aimed at teaching long-horizon enterprise agent work. According to the Prime Intellect blog, 100 GRPO steps — about 11 hours of training on 128 NVIDIA GB200 GPUs — took NVIDIA Nemotron 3.5 Super VL from 34.0% to 57.7% on AutomationBench-AA (public split), a 23.7-point gain over its starting SFT checkpoint. Each task is scored on actual changes to simulated business applications and files, with objectives the agent must complete and guardrails it must not break. The environment library, trainer and templating used for validation are open source, and the same loop runs as a hosted service on Prime Intellect Lab.
Prime Intellect Blog · 🔥 0 - Tom's Hardware tests RTX Spark in Surface Laptop Ultra: strong local AI, weak gaming, $5,899
Tom's Hardware tested Nvidia's RTX Spark platform in Microsoft's Surface Laptop Ultra, a system with the full GB10 Superchip, 20 CPU cores, 6,144 CUDA cores and 128GB of LPDDR5X-9400 memory. Launch embargoes limited this round to platform-wide experience and comparative performance, with battery life, build and display details promised later. For AI, the outlet used Artificial Analysis's AgentPerf-local benchmark, reporting time-to-completion across agentic tasks instead of time-to-first-token and tokens-per-second. In gaming, Gears of War: E-Day averaged 77.7 fps with 39.9 fps 1% lows near 1080p, which the reviewer called not a great experience; Control Resonant averaged 109.2 fps at 2560x1440 but with input latency above what the tester considers acceptable. The tested unit lists at $5,899 with 1TB of storage, with lead times already into late December.
Tom’s Hardware · 🔥 0 - Ant Group's Ling team open-sources 560B Ling-3.1-flash
On October 15, Ant Group's Ling team released Ling-3.1-flash as open weights, publishing an FP8 checkpoint under the MIT license on Hugging Face and ModelScope. The model has about 560B total parameters and activates about 25B per token; Ant says it supports up to 1M context, though 36Kr notes the repository's max_position_embeddings is 262,144 and 1M is listed as a target, while Artificial Analysis measured an intelligence index of 41 and about 210 tokens per second. Ant says a mixed MXFP4/block-FP8 scheme cuts weights from about 534GB to 292GB and its Hopper test configuration from eight GPUs to four; 36Kr also notes vendor benchmark scores were run on the unreleased LingCode harness, and independent testing found flaws in long-chain agent tasks.
智东西 · 🔥 0 - Artificial Analysis launches AA-Robotics v0.1; GPT-6 Astra leads robot-arm tests
Artificial Analysis released AA-Robotics v0.1, a research-preview benchmark in which frontier models write code through coding agents to control a Hiwonder NexArm six-axis robot arm across eight tabletop tasks such as pouring, writing and stacking blocks. GPT-6 Astra led the first results with 69.4 points, ahead of GPT-6.1 Sol (64.6), Gemini 4 Argon (55.3), Claude Opus 5.5 (48.8) and Claude Fable 5.1 (46.3). Models averaged about 30 minutes per task against roughly 20 seconds for the human baseline, and Astra cost $16.36 per task on average versus $1.93 for Sol, which scored only 4.8 points lower. The reports also noted that models often kept working after meeting the criteria, as when Sol built a qualifying block tower and then ruined it with further adjustments.
机器之心 · 🔥 0 - Meta releases Muse Spark Open, an 816B open-weight multimodal model
Meta released the weights of Muse Spark Open on 15 October, an open-weight mixture-of-experts model with about 816 billion total and 42 billion active parameters and a context window of up to one million tokens. It accepts text, images, audio and video as input and produces text, and Meta trained it for agentic coding, tool use and long-horizon tasks. Meta, which calls it the most capable US open-weight model to date, worked with Apple so the model can run locally on two M5 Ultra Mac Studio desktops via MLX; it is also offered through third-party APIs including Together AI, Baseten, Fireworks AI, Groq, Modal, Nebius and OpenRouter, with weights on Hugging Face under the Muse Model License. Arena.ai ranked it the top US open model on its Code Arena: WebDev leaderboard at 1511 points and 13th among open models overall, while an Artificial Analysis score of 39 cited in community discussion placed it behind GLM 5.3 and Kimi K3.
Hugging Face · 🔥 0 - SpaceXSI releases Grok Voice 3.0, topping Artificial Analysis speech-to-speech index
On Oct 14, SpaceXSI, formerly xAI, released Grok Voice 3.0, a speech-to-speech model it calls "the world's most intelligent voice model," with sub-one-second latency, available in its Voice Agent API at $0.08 per minute, the same price as Grok Voice 2.0. Artificial Analysis' Speech-to-Speech Index ranks it first at 86.9, ahead of OpenAI's GPT-Live-1 at 83.2 and Google's Gemini 3.8 Live at 82.6; it also scored 97% on Full Duplex Bench and 74% on ServiceNow's EVA-Bench for enterprise voice agents. In the τ-Voice test simulating accents, background noise and interruptions, its task success rate was 71.6%, slightly below GPT-Live-1's 74.5%, though SpaceXSI says its latency is 40% lower. The company says Grok Voice already handles millions of in-car conversations at Tesla and about 20,000 multilingual Starlink support calls a day, roughly 70% resolved without a human.
xAI News · 🔥 0 - AI video prices fall from yuan to fen as Kling, Vidu and Seedance compete
An analysis by Guangzhui Zhineng, published by 36Kr and syndicated by TMTPost, reports that Kuaishou's Kling AI opened internal testing of Kling 4.0 on September 28, extending single native clips to 30 seconds at 4K, and launched a lightweight Kling 4.0 Flash that emphasizes fast generation and lower cost, though its API pricing has not been disclosed. On October 8, Shengshu Technology released Vidu Q4 Preview, supporting up to 15 reference images, three reference audio clips and 4K output, with launch pricing from 0.09 yuan per second and a company estimate that the same budget can produce up to about five times more content. The article also cites list prices showing ByteDance's Seedance 2.5 at 70 yuan per million tokens for 720p generation without references versus 46 yuan for Seedance 2.0, and argues that flagship video entry prices have moved from yuan to fen per second, shifting competition toward cost-performance and deliverable quality.
36氪 人工智能 · 🔥 0 - Custom silicon and the inference price war: Google's Argon undercuts rivals
Citing Artificial Analysis data, Tomasz Tunguz notes that five frontier models sit within five intelligence points of one another while their cost per task differs by 3.8x; Google's Argon comes in at $1.99 a task, 74% below the priciest model, Fable 5.1, and 67% below Opus 5.5, 56% below Muse Wonder and 39% below Astra. He attributes the gap to differences in architectures and GPU costs, pointing to Google's TPU program begun in 2013 and citing Google's own TPU v4 measurements versus Nvidia's A100. The essay also cites Microsoft's Maia 300 and OpenAI's first inference chip, Jalapeño, both company-reported efficiency claims without independent verification, and estimates Google could serve tokens at 44% less than peers — a figure derived from the midpoint of a published range, not a measured result.
Tomasz Tunguz · 🔥 0 - Vidu Q4 Preview launches from ¥0.09/second as China's video-model trio splits paths
According to Zhi Dongxi and 36Kr, China's leading video-generation players refreshed their line-ups around the National Day holiday: Kuaishou's Kling AI announced Kling 4.0 on Sept 28, and Shengshu Technology released Vidu Q4 Preview on Oct 8, the first preview of its next flagship video model, priced from ¥0.09 per second. The article says Vidu Q4 Preview centres on expressive character acting, camera language and complex VFX, supports up to 4K output and 16-second single generations, and accepts up to 15 reference images plus three reference audio clips. It also claims the preview overtook Seedance 2.0 to rank third globally on Artificial Analysis, with MaaS image-to-video pricing of ¥0.59/s at 720p, ¥0.75/s at 1080p and ¥2.43/s at 4K, described as the lowest in the industry.
智东西 · 🔥 0 - Utopai opens MIPCOM as AI-native studio; Banderas to star in Cortés
On October 13, AI-native film studio Utopai took the opening keynote stage at MIPCOM in Cannes, pitching a studio model built around original IP, human creativity and its own film infrastructure, and disclosing that a partnership with Korea's CJ ENM Studios is being advanced. The same day, Antonio Banderas announced he will star in Utopai's film Cortés, which the report describes as his first theatrical feature with AI involvement. The report also says Utopai plans three theatrical releases in 2027 — Cortés, Half Moon and The Most Serious Fart — and that its custom video model Utopai X ranks second worldwide with a score of 1149 on Artificial Analysis's blind text-to-video-with-audio leaderboard.
钛媒体 · 🔥 0 - StepFun's Step 5 Preview lands on Tencent WorkBuddy with two weeks free
StepFun announced on October 12 that its flagship Step 5 Preview model is now available on Tencent's all-scenario AI office workspace WorkBuddy, with exclusive free access from October 12 to October 25, 2026. WorkBuddy users can use the model for deep research, information retrieval, data analysis, and document and PPT generation, including sustained execution of multi-step tasks. Step 5 Preview launched on September 20 with a sparse MoE architecture, 600B total parameters and 27B activated, a 1M-token context window and native text and vision input; StepFun says it plans to open-source the full weights on October 15.
IT之家 AI · 🔥 0 - Mistral releases trillion-parameter Large 4, the best model outside the US and China
European AI lab Mistral has released Large 4 (also called ML4 and nicknamed "Le Chonk" on social media), a trillion-parameter model. Testing lab Artificial Analysis scored it 38 on its overall intelligence index, according to Tom's Hardware, leaving it behind Chinese models such as Kimi K3 (max), GLM-5.3-Flash and MiMo-V2.6.Pro, though it performs strongly on cybersecurity tasks. Tom's Hardware reported that independent tests rank ML4 as the best AI model outside the U.S. and China, and said Mistral expects the exact weights to be released before the end of October.
Tom’s Hardware · 🔥 0 - Xiaomi's MiMo-V2.6 report details $2.6M single-run RL post-training
Xiaomi's MiMo team published a technical report, "MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement," covering two sparse MoE models: MiMo-V2.6-Pro (1.02T total parameters, 42B activated) and MiMo-V2.6-Flash (310B total, 15B activated). The report says each RL step samples 1,568 prompts and roughly 25,000 agent trajectories totalling 2.7–3.7B tokens, with RL post-training costs of about $2.6M for Pro and $900K for Flash. The team also open-sourced more than 7,000 RL environments plus an end-to-end RL framework.
InfoQ 中文 AI&大模型 · 🔥 0 - Artificial Analysis teases AA-Robotics v0.1 benchmark for robot-arm control
Artificial Analysis said on X that AA-Robotics v0.1 is coming soon, describing it as a benchmark for whether today's frontier language models can control a real robot arm zero-shot, with no coaching. The post gave no release date, model list, task design or scoring method, so it is a teaser rather than a published result. If it ships as described, the benchmark would extend language-model evaluation from text and software tasks into physical robot control.
ArtificialAnlys (X) · 🔥 0 - 30-step image edit test: GPT Image 2.5 Sunburst drifts most, Ideogram 4.5 holds up
Artificial Analysis ran the same photo through four frontier image-editing models for 30 consecutive edits (light the fire, add a sofa, repaint the walls, swap day for twilight, and more) to compare multi-turn consistency. It says Ideogram 4.5 and FLUX 3 edit locally, leaving 95% or more of the image essentially untouched on small changes such as adding a vase of tulips, while leaderboard-topper GPT Image 2.5 Sunburst re-renders most of the image on every edit, keeping only about a fifth unchanged so colour and detail shifts compound over turns. Nano Banana 2.1 sits in between: edits stay local but the rest of the image shifts slightly and gradually darkens. The group notes its leaderboard scores single edits, so this was its own test of what happens when 30 changes stack up.
ArtificialAnlys (X) · 🔥 0 - Utopai pushes AI-native films into theatres and Hollywood unions
Per ifanr, AI-native studio Utopai Studios is advancing several theatrical projects: the family animated feature The Most Serious Fart (planned for summer 2027), the eight-episode sci-fi series Space Nation (in production under a Hollywood guild framework), the historical epic Cortés (full theatrical release expected 2027), a Journey to the West project co-developed with Huace Film & TV, and the 2027 drama Half Moon. The company says it has presold some international distribution rights for Space Nation and Cortés to traditional buyers including Brazil's Globo system and German broadcasters at standard non-AI market prices, with combined presale revenue potentially reaching about $110 million. Its in-house video model Utopai X, specially trained for film production on top of MiniMax H3, ranks first in audio sync and physics according to Artificial Analysis and feeds into its PAI production platform.
爱范儿 · 🔥 0 - Anthropic launches Claude Haiku 5.5 at $0.10 per million input tokens
Anthropic released Claude Haiku 5.5 on October 7, describing it as the fastest, cheapest and most capable model in its Haiku small-model line. For requests up to 100,000 tokens, it costs $0.10 per million input tokens and $0.50 per million output tokens, rising to $0.50 and $2.50 above that threshold; Anthropic says it costs about 75% less to run on average than Haiku 4.5. Anthropic-reported figures show OSWorld 2.1 offline subset rising from 15.7% to 72.4% and GDPval-AA v2.1 at 1620, while Terminal-Bench 4.0 is reported at either 39.2% or 33% across sources, still well below Sonnet 5.5. Haiku 5.5 is the first Haiku model with adjustable effort settings and is positioned as a subagent for Opus 5.5 and Sonnet 5.5; Anthropic also cut Sonnet 5.5 cache-read pricing from $0.20 to $0.10 per million tokens.
IT之家 AI · 🔥 0 - Utopai X's No.2 video ranking questioned: post-training on MiniMax H3 under scrutiny
On Sept 30, 2026, Artificial Analysis's text-to-video blind-voting leaderboard placed Utopai Studios' Utopai X second worldwide with 1150 points, behind only Wan 3.0 and the top-ranked US model, prompting a wave of Chinese-language coverage calling it an "AI video dark horse" with a "proprietary custom model." The article says roughly 90% of that coverage ignored the leaderboard footnote "Utopai X (based on MiniMax H3)" — it is not trained from scratch but post-trained on MiniMax's open-source base with film-production data. It raises three questions: cost-effectiveness (base scored 1136 vs 1150, and no training compute or inference configuration has been disclosed), whether the fine-tuning can migrate to a new base model, and whether its iteration speed can match native models such as Seedance's continuous data flywheel.
钛媒体 · 🔥 0 - Why OpenRouter’s Rankings May Not Measure Global Model Preference
An analysis argues that OpenRouter’s rankings measure token volume routed through one intermediary rather than global developer preference, and that prices, discounts, free access, batch jobs and self-testing can heavily influence the results. It also questions the comparability of Artificial Analysis scores and ARC-AGI results when test composition, weighting, scoring anchors, interfaces and versions change.
钛媒体 · 🔥 0 - Utopai X ranks second on AI video leaderboard as PAI targets film production
Utopai Studios' video model Utopai X scored 1,150 Elo on Artificial Analysis's text-to-video-with-audio leaderboard (dated Sept 29, 2026), placing second behind Wan 3.0 and ahead of Dreamina Seedance 2.5. The studio pairs the post-trained model with PAI, a production system that keeps scripts, character and scene assets, shot lists and revision history in one workspace. It says three features — Cortés, Half Moon and The Most Serious Fart — are planned for 2027 theatrical release, and it has partnered with Japan's DeNA to trial PAI in animation production.
36氪 人工智能 · 🔥 0 - Mistral launches Mistral Large 4 preview, a 1T-parameter open-weight model
Mistral AI has opened a public preview of Mistral Large 4 (nicknamed Le Chonk), a 1-trillion-parameter natively multimodal open-weight model with roughly 49 billion active parameters; the weights are due by the end of the month. Mistral claims state-of-the-art open-model performance on enterprise workloads such as cybersecurity, finance and law, and says it beats even closed frontier models on visual grounding — though it notes that some cyber-test leads come from Claude and GPT-6 refusing the tasks. Independent measurement is more sobering: Artificial Analysis's Intelligence Index puts ML4 at 38 points, a big jump from Mistral Large 3's 9, but well behind leader Claude Opus 5.5 (Max) at 58.
Mistral AI News · 🔥 0 - Utopai X Reportedly Ranks Second in Artificial Analysis Video Test
Utopai Studios’ Utopai X reportedly scored 1,150 (±10) in an Artificial Analysis blind video evaluation, ranking second globally and first in the United States. The article says the model was post-trained from MiniMax H3 for film production and led the evaluation’s audio-sync and physics categories, but these claims are presented through a promotional report and are not independently verified here.
新智元 · 🔥 0 - Utopai X reportedly ranks second in Artificial Analysis Video Arena
A Machine Intelligence report says Utopai X ranked second globally, with an Elo score of 1150, in Artificial Analysis's Video Arena in late September. The article attributes the result to custom post-training built on the MiniMax H3 architecture and integration with Utopai's Production Intelligence platform, while noting that the claims come from the reported evaluation and company materials.
机器之心 · 🔥 0
Experience and discussion from the community
Share my Artificial Analysis experienceAsk about Artificial Analysis
Nobody has shared their experience with Artificial Analysis yet.