Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Stirrup

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 5

  1. Parallel Search advanced tops Artificial Analysis Search Index

    Artificial Analysis benchmarked Parallel Search API’s fast and advanced modes across 1,700 search-focused tasks using its Stirrup harness and GPT-5.6 Luna. In the November runs, advanced scored 83 and ranked first among 26 entries on the Search Index, while fast scored 82 and ranked fourth. Both modes scored 89 F1 on DeepSearchQA, with fast averaging about $0.014 per task and advanced about $0.030.

    ArtificialAnlys (X) · 🔥 4
  2. Google and Artificial Analysis launch MKA agent benchmark

    Google DeepMind and Artificial Analysis launched MKA, a benchmark for testing agents on visually complex, real-world knowledge work such as blueprints, charts, reports, and spreadsheets. The evaluation set contains 200 private tasks developed with experts across 10 industries and 83 occupations, while 10 sample tasks are public on Hugging Face. Google reported that Gemini 4 Carbon ranked second in quality, with an average task cost about one-sixth that of the leading model.

    IT之家 AI · 🔥 0
  3. Google DeepMind and Artificial Analysis launch MKA benchmark

    Google DeepMind and Artificial Analysis launched the Multimodal Knowledge Agent (MKA) evaluation, which tests agents on 200 private visual tasks drawn from 10 industries, including architectural drawings, engineering schematics, and clinical scans. Tasks are scored against expert rubrics averaging about 70 weighted criteria, with Climb WSR as the headline metric; Artificial Analysis runs them in its Stirrup harness with code execution, web access, and image-viewing tools. Artificial Analysis reports Claude Fable 5.2 leading with a 57% Climb WSR, followed by Gemini 4 Carbon at 54% and Claude Opus 5.5 at 53%, alongside substantial differences in cost per task.

    ArtificialAnlys (X) · 🔥 0
  4. Keenable Search 1.0 leads Artificial Analysis search ranking

    Artificial Analysis benchmarked Keenable Search 1.0's pro and realtime modes across 1,700 search-focused tasks using GPT-5.6 Luna and the Stirrup agent harness. The pro mode scored 83 and realtime scored 82 on the Artificial Analysis Search Index, ahead of Exa and Perplexity at 80, while average search time was 3.0 and 2.3 seconds per task against a 12.5-second board median. Public pay-as-you-go pricing was reported at $4 per 1,000 searches in both modes.

    ArtificialAnlys (X) · 🔥 0
  5. Artificial Analysis launches VIALS-AA visual-reasoning leaderboard

    Artificial Analysis launched VIALS-AA, a leaderboard for Handshake AI’s VIALS benchmark, with 161 tasks across eight life-science and biotech domains. Models must interpret scientific artifacts—including gel blots, flow cytometry plots, plasmid maps, and phylogenies—and produce specific conclusions, with each task run three times in the Stirrup agent harness. Gemini 4 Carbon led with a 64% pass^3 score, followed by GPT-6 Astra at 63% and Claude Opus 5.5 at 61%, while Kimi K3.1 Pro led the open-weight models with 45%.

    ArtificialAnlys (X) · 🔥 0

Experience and discussion from the community

Share my Stirrup experienceAsk about Stirrup

Nobody has shared their experience with Stirrup yet.