Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Artificial Analysis

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 119

  1. Tencent releases Hy4 with vision and computer use

    Tencent Hunyuan released the 770B-parameter Hy4 model, expanding pre-training and post-training over Hy4 preview while adding vision perception, computer use, improved long-context understanding and stronger long-horizon task performance. Tencent says Hy4 is available through Tencent Cloud, OpenRouter and products including WorkBuddy and CodeBuddy. Model weights and a system card are still planned, so the release does not yet constitute an open-weight publication. Artificial Analysis scored Hy4 at 35 on its Intelligence Index, placing it second among Chinese lab models but behind Kimi K3.1 Pro, while Tencent acknowledged a remaining gap with leading proprietary models.

    Hacker News 首页(100+ 分) · 🔥 0
  2. AA-PersonalityBench compares open-weight and proprietary model behavior

    Artificial Analysis compared the 10 open-weight and 10 proprietary models with the highest scores in its Intelligence Index using AA-PersonalityBench. It percentile-ranked each model from 1 to 100 across 30 personality traits and found that the leading open-weight models averaged 32 points below leading proprietary models on truthfulness. Open-weight models also ranked lower on pragmatism and structure, but higher on slop and bottom-line summarizing.

    ArtificialAnlys (X) · 🔥 0
  3. Artificial Analysis Updates Six Capability Indexes to v2.0

    Artificial Analysis said it updated its six Capability Indexes to v2.0, aligning them with evaluations from the Artificial Analysis Intelligence Index v5.0 and placing greater emphasis on Non-Hallucination and Agentic Tool Use. The indexes map O*NET occupational tasks to representative benchmarks, weighting each benchmark by how often the capability appears in the occupation. They cover Finance & Accounting, Strategy & Ops, Legal, Healthcare & Medical, Engineering, and Economics. The reported changes include upgraded HLE and AA-Omniscience versions, removal of GDPval-AA from Agentic Knowledge Work, a new Science capability for Engineering using CritPt v2 and Terminal-Bench-Science 0.1, and FrontierCode 2.0 for Agentic Coding. The report said Claude Opus 5.5 (max) leads all six indexes. These details and rankings come from a single supplied report.

    ArtificialAnlys (X) · 🔥 0
  4. Artificial Analysis plans API release for improved CritPt v2

    Artificial Analysis said it worked with the CritPt authors to finalize the CritPt v2 grading server and planned to make it available through its API the following day. The authors improved the robustness of the grading-server logic, and Artificial Analysis incorporated those changes in version 5.0.1 of its Intelligence Index. It then regraded every model, finding small score increases for most models while frontier-model rankings remained unchanged, with scores moving by at most one point.

    ArtificialAnlys (X) · 🔥 0
  5. FrontierPhysics benchmark launched for end-to-end AI physics research

    A paper submitted to arXiv on 4 November 2026 introduces FrontierPhysics, an end-to-end benchmark with 56 tasks made by 42 physicists based on their own research, covering nine domains across theoretical, computational, and experimental physics. Each task requires agents to review the literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts. Scoring combines deterministic verifiers with expert-authored rubrics to judge whether agents' research is correct and publishable. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts. Manual review of agent trajectories shows that frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors. The benchmark team, drawn from physics labs at Berkeley, Caltech and Harvard, said it recruited 42 physicists from 26 institutions; Artificial Analysis said it was excited to partner to host the evaluation on its platform soon.

    Hugging Face · Papers · 🔥 0
  6. HeyGen Voice Turbo reportedly free on OpenRouter through November 30

    An OpenRouter social-media announcement says HeyGen Voice Turbo is free through November 30, with no credits required, and supports 39 languages. The same announcement describes it as a lower-latency version of HeyGen’s text-to-speech model that previously topped the Artificial Analysis TTS leaderboard, and says it offers more than 2,900 voice options, requests of up to 5,000 characters, and streamed 44.1 kHz audio. The offer and specifications have not been independently verified.

    OpenRouter (X) · 🔥 0
  7. Utopai Studios appoints Hans Ku as chief product officer

    Utopai Studios announced that former Disney AI executive Hans Ku has joined as chief product officer, reporting to CEO Cecilia Shen. He will oversee product, engineering, research and business development, with a focus on expanding PAI across original productions and industry partnerships. The reports also say he previously helped Disney's collaboration with OpenAI around Sora.

    品玩 实时要闻 · 🔥 0
  8. AA-AgentPerf Adds Daily GPU Rental Pricing and Throughput-per-Dollar Metric

    Artificial Analysis announced a collaboration with Silicon Data to add market-based GPU rental prices to the AA-AgentPerf datacenter benchmark. The new “Throughput per $/hr” metric divides a system's serving throughput by its hourly rental cost, with current indices covering H100, H200, B200 and B300 across cloud providers. AA-AgentPerf results show NVIDIA's B300 generally leading among systems with rental data, while the B200 matches it at faster output speeds on some models.

    ArtificialAnlys (X) · 🔥 0
  9. Open-weight models gain token share while leading closed models score higher

    OpenRouter reports that open-weight models rose from 27% to 72% of the top 50 models’ tokens on its platform between October 2025 and late September 2026. Its token-weighted pricing data lists open-weight models at $0.28 per million input tokens and $1.62 per million output tokens, compared with $1.27 and $6.36 for closed-weight models. On the Artificial Analysis Intelligence Index, Claude Opus 5.5 scored 57.6, compared with 46.3 for MiMo-V2.6-Pro, the leading open-weight model in the comparison. The report says these scores depend on reasoning-effort settings, the 11-point gap is a snapshot, and token counts use providers’ own tokenizers. OpenRouter recommends testing an open-weight model on a team’s tasks and using closed-weight models for steps where tests show extra quality matters.

    OpenRouter Blog · 🔥 0
  10. NVIDIA Revises Vera Rubin Specifications for Volume Production

    NVIDIA said the revised Vera Rubin specifications represent what its systems sustain after qualification and volume production, rather than the preliminary architectural targets published during development. The changes lower the per-Rubin GPU HBM4 bandwidth from 22 TB/s to 19.2 TB/s and NVLink 6 bandwidth from 3.6 TB/s to 3 TB/s across the platform. NVIDIA also said Vera Rubin NVL72 is shipping and that published third-party agentic benchmarks used the production specifications, but the supplied reports do not independently verify those claims.

    Wccftech · 🔥 0
  11. Runway shifts toward model aggregation and enterprise licensing

    36氪 reports that Runway is shifting its strategy toward Aleph 2.0 video editing, a multi-model app, Agent-centered workflows, and enterprise model licensing. Its app now defaults to Seedance 2.0 for video and GPT Image 2 for images, while Gen-4.5 remains available as an option. Runway Dev separately provides API access to multiple models, including Seedance and Google Omni Flash, with routing based on developer preferences, candidate models, and price limits. Through its model-licensing business, Runway grants enterprises such as Gen-4.5 for private deployment and fine-tuning with their own data, charging licensing fees. CEO Anastasis Germanidis said ARR was approaching $200 million; the report did not independently verify that figure. AppMagic estimated $2.7 million in mobile revenue over the latest 30 days, while Artificial Analysis ranked Gen-4.5 outside its top 10.

    36氪 人工智能 · 🔥 0
  12. Google Releases Gemini 4 Carbon for Coding and Cybersecurity

    爱范儿 reports that Google DeepMind released Gemini 4 Carbon, a flagship model focused on long-horizon coding and cybersecurity. Artificial Analysis Intelligence Index v5.0 reportedly scored Carbon (High) at 59, ahead of Claude Opus 5.5 (Max) at 58, while Google reported results on SEC-bench Pro, ExploitGym and CWE-bench. Access is initially limited to selected cybersecurity defenders and trusted partners through the Fairwind Program, with broader availability unannounced.

    爱范儿 · 🔥 0
  13. Google launches Gemini 4 Carbon for cybersecurity and long-horizon tasks

    Google DeepMind announced Gemini 4 Carbon as its next frontier model, emphasizing cybersecurity, coding, and long-horizon capabilities. Artificial Analysis measured it at 59 on Intelligence Index v5, placing it first, though it trailed Claude Opus 5.5 on the Coding Agent Index. Access initially goes to selected cybersecurity defenders through the Fairwind Program, with no general-availability timetable announced.

    Google DeepMind Blog · 🔥 0
  14. TinyLeague removes Artificial Analysis, adds KV cache efficiency metric

    TinyLeague maintainer nunodonato announced that the small-model ranking project will stop tracking Artificial Analysis scores. The maintainer said many small models lack such scores and that, in his view, Artificial Analysis is moving toward closed benchmarks, which could result in fewer scores for small and open-weight models. TinyLeague added a KV cache efficiency metric that estimates memory use in gigabytes for a 128K context and rates models on a 0–100% scale. The maintainer said the metric is intended for people running local models with limited VRAM. Work on a separate TinyLeague-Index continues, but the maintainer said a fair methodology has not yet been found because models use different benchmarks.

    r/LocalLLaMA · 🔥 0
  15. Artificial Analysis overhauls Intelligence Index with private evaluations

    Artificial Analysis launched Intelligence Index v5, with 60% of its evaluations now based on private test sets or held-out answers. The update adds FrontierCode 2.0 and Terminal-Bench-Science, upgrades three existing benchmarks, and introduces Progress Index v1 to track model capability over time; Claude Opus 5.5 leads the initial v5 ranking with 58 points.

    cognition (X) · 🔥 0
  16. Can Cheaper OpenAI Models Dislodge Anthropic’s Enterprise Lead?

    The Information reports that OpenAI is positioning itself as a cheaper and more flexible enterprise provider to challenge Anthropic, with companies such as Rippling reportedly switching from Anthropic models to cost-efficient OpenAI models. Artificial Analysis evaluations cited by The Information found that OpenAI released the least expensive models this year among OpenAI, Google and Anthropic for completing the evaluated tasks. The report also says Anthropic’s newly filed IPO prospectus indicates that lower prices and flexibility alone may not substantially weaken its enterprise position.

    The Information · 🔥 0
  17. Microsoft releases MAI-Image-2.7-Preview for controllable image editing

    Microsoft announced MAI-Image-2.7-Preview on October 27, 2026, a model that edits an existing image using text instructions rather than generating images from text. Artificial Analysis placed it second on AA-Image-Editing v2.1 with 1,193 Elo, while Arena ranked it third with 1,470 points; Microsoft also claimed improvements over MAI-Image-2.6 across all 10 tested cases and seven editing-action categories. The preview is available on Arena, Artificial Analysis, and MAI Playground, but Microsoft has not announced a text-to-image release date, public API, pricing, or general commercial availability.

    IT之家 AI · 🔥 0
  18. Ant open-sources Ling-3.1-flash-1019 and Ling-3.1-flash-base

    Ant’s Bailing team released the Ling-3.1-flash-1019 checkpoint for personal-agent workloads and simultaneously published BF16 Ling-3.1-flash-base weights under the MIT license. Both use a 560B-total, 25B-active mixture-of-experts design. The 1019 checkpoint supports a context window of up to 1 million tokens and scored 44 on Artificial Analysis Intelligence Index v4.3.2, three points above the previous version. Bailing says the base model achieved the highest score in 32 of 47 tests against DeepSeek-V4-Pro-Base and MiMo-V2.5-Pro-Base, but the supplied reports do not establish independent reproduction of that comparison.

    r/LocalLLaMA · 🔥 0
  19. Kimi K3.1 Pro Ranks Fourth, Then Apologizes for Quota Issues

    Moonshot AI released Kimi K3.1 Pro as an upgrade to its July K3 model, with free-to-download weights planned by Nov. 4. Artificial Analysis reportedly scored it 55 on its Intelligence Index, placing it fourth overall and first among open models, while some users reported slow performance and occasional wrong paths. Within 48 hours, Kimi apologized for rate limits and unnecessary quota consumption among paying users, offering compensation equal to one week of usage.

    36氪 人工智能 · 🔥 0
  20. Artificial Analysis changes make benchmark scores harder to compare

    Artificial Analysis revised its benchmark methodology several times from v4.1 through v4.3.2, changing test weights, replacing or adding evaluations, retiring GPQA Diamond, and recalibrating GDPval Elo scores, according to its official changelog and methodology. Huxiu AI notes that Kimi K3 fell from 57 at its July evaluation to 44 on the current model page, while Kimi K3.1 Pro received 55 and was expected to lead the open-model ranking once weights were equalized. Because the service did not publish a crosswalk between old and new scores, results from different versions or capture dates should not be compared directly.

    虎嗅 AI · 🔥 0
  21. Moonshot AI launches Kimi K3.1 Pro with 1M-token context

    Moonshot AI released Kimi K3.1 Pro, a post-trained version of Kimi K3 with 2.8 trillion total parameters, 104 billion active parameters, and a 1 million-token context window. Artificial Analysis reported a score of 55 on its Intelligence Index, while the model is available through Kimi web and mobile services, Kimi Work, Kimi Code, the API, and OpenRouter. Moonshot AI said it would release the full model weights by November 4, but the supplied reports do not confirm that release occurred.

    MIT科技评论中文 · 🔥 0
  22. DeepSeek V4.1 Flash revision 1017 boosts efficiency

    DeepSeek upgraded its open-weight V4.1 Flash model to revision 1017, reducing HBM and SSD KV-cache requirements to one-quarter and one-eighth of V4 Flash, respectively. The 552B-parameter Mixture-of-Experts model activates 16B parameters, accepts text and images, and supports a 1-million-token context window; DeepSeek’s internal benchmarks place it between GPT-5.6 Sol and Claude Opus 5, though independent Artificial Analysis testing flags frequent hallucinations. Tom’s Hardware reported that DSF-1017 ranks second among large open-weight models on its intelligence index, but is less reliable when answers lack supporting facts.

    Tom’s Hardware · 🔥 0
  23. Muse Spark Open and Qwen3.8-Flash-Next benchmark comparison

    A Reddit analysis of Artificial Analysis results gives Muse Spark Open a score of 39 and Qwen3.8-Flash-Next a score of 40, comparing open-weight models by total parameter count. The post also reports Unsloth UD-Q4_K_XL GGUF sizes of about 506 GB for Muse Spark Open and 111 GB for Qwen3.8-Flash-Next. It relays Meta’s recommendation to run Muse Spark Open locally on two M5 Ultra Mac Studios with 512 GB of memory each.

    r/LocalLLaMA · 🔥 0
  24. Baseten releases mammoth-1 and elephant-1 decision models

    Baseten released mammoth-1 and elephant-1, open-weight models from Base Labs that return probability distributions for boolean, multiple-choice, and ordered-score questions instead of generating text. The 27B mammoth-1 and 9B elephant-1 were post-trained from Qwen3.5 models on 6.21 million examples; Baseten reports mammoth-1 first among 208 models on Jev Decision Index 0.4, while Artificial Analysis scored elephant-1 65.2, tying d1-ultra. A r/LocalLLaMA report says elephant-1 uses an MIT license and costs about $8 per million tasks, compared with roughly $31 for d1-ultra, while both models are available through Baseten Model APIs with zero data retention during the reported trial.

    Baseten Blog · 🔥 0
  25. MiniMax M3.1 Flash lands on Runware with 1M context, multimodal input

    MiniMax said its M3.1 Flash model is now available on inference platform Runware, accessible via the model string "minimax:m3.1@flash", thanking Runware for shipping it quickly. In the same post, MiniMax said the model accepts text, image and video input, supports a 1M-token context window with up to 524K output tokens, and offers adjustable reasoning depth. The vendor also cited a score of 42 on the Artificial Analysis Intelligence Index, 72.5% on AutomationBench-AA and 73.7% on DeepSWE, and described the model as cheaper per token than MiniMax; these figures come from MiniMax's own post and have not been independently confirmed.

    MiniMax_AI (X) · 🔥 0
  26. Reddit user plots Artificial Analysis Intelligence Index against model size

    A Reddit user posting as /u/xrailgun on r/LocalLLaMA has built an interactive site that plots Artificial Analysis Intelligence Index scores against total or active parameter counts, with closed frontier models shown as optional horizontal dashed lines. The site accounts for mixture-of-experts and lookup-table models such as Qwen3.8-Flash-Next, offers a rough model recommendation guide based on a user's hardware, and refreshes its data every three days through a GitHub Action. The author says the motivation was that Artificial Analysis charts the index against price and speed, while the axis wanted for local use was parameter size; incorrect or missing sizes can be corrected in an overrides file.

    r/LocalLLaMA · 🔥 0
  27. Ant Group open-sources Ming-3.0-tts, tops two open-source TTS leaderboards

    Ant Group has released Ming-3.0-tts, an open-source text-to-speech model whose weights are published on Hugging Face under the MIT license, according to AIbase AI新闻. The release includes a 0.5B dense model and a 7.9B mixture-of-experts variant called Ming-3.0-tts-tiny, and the model offers preset voices plus stateless voice cloning. AIbase, citing Artificial Analysis leaderboards, reports that Ming-3.0-tts scored 93.2% on pronunciation robustness to tie for first among 31 models ahead of Eleven v4 at 91.7%, ranked first among open-source models in the Controlled Voice Arena with a 1,069 Elo over 1,827 evaluations and 13th overall, and placed third among open-source models in Provider Voice with 1,188 Elo over 1,957 evaluations and 19th overall.

    AIbase AI新闻 · 🔥 0
  28. Perplexity's open pplx-decider-v1.2 claimed 10x cheaper than GPT 6 Luna

    A Reddit post in r/LocalLLaMA says Perplexity's pplx-decider-v1.2 costs one tenth of GPT 6 Luna with low reasoning while producing identical-quality decisions, and that pplx-decider-flash-v1.2 does the same within the Decisions API. The poster, /u/Balance-, also says v1.2 is 2.5x cheaper than Jev, links to an Artificial Analysis decision index page, and states that both pplx-decider models are open source under Apache 2.0 with repositories on Hugging Face. The supplied material includes no official Perplexity announcement and no independent verification of the pricing or quality claims.

    r/LocalLLaMA · 🔥 0
  29. ElevenLabs launches Eleven Realtime v1 Mini and Max speech-to-speech models

    ElevenLabs released Eleven Realtime, a speech-to-speech model line with v1 Mini for lower-cost natural conversation and v1 Max for customer support and multi-step tasks, on October 20, 2026. Both models are available through the ElevenAgents platform and an OpenAI-compatible Realtime API, priced per minute plus token costs, with Max costing more than Mini. According to Artificial Analysis independent testing reported by 智东西, Max scored 98.8% on Big Bench Audio with a 0.57-second average first-audio response, placing third behind StepAudio 3 Realtime at 99.7% and Qwen Audio 3.0 Realtime Plus at 99.2%, while Mini scored 75.9%; ElevenLabs separately claimed no model on the Artificial Analysis leaderboard is both faster and more accurate, and 智东西's hands-on test of the Architect setup assistant found voice editing convenient but noted a noticeable Chinese accent, long replies and latency.

    The Decoder · 🔥 0
  30. Xiaomi MiMo-V2.6-Pro Launches on WorkBuddy

    IT之家 reported that Xiaomi Technology announced Xiaomi MiMo-V2.6-Pro is now available on WorkBuddy. Citing the announcement, the report says the model supports 1M-token context, multi-agent collaboration, and joint understanding across images, video, audio, and text.

    IT之家 AI · 🔥 0

← PreviousNext →

Experience and discussion from the community

Share my Artificial Analysis experienceAsk about Artificial Analysis

Nobody has shared their experience with Artificial Analysis yet.