Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Hybrid Jev and Claude Opus routing cuts identity-resolution costs

AI summary

A vendor-authored benchmark tested Jev and GPT-6 Luna on identity-resolution grading, where both delivered acceptable out-of-the-box accuracy and were faster and cheaper than frontier models in the authors’ setup. On the harder test, routing low-confidence Jev cases to Claude Opus 5.5 produced 95.3% accuracy at $2.94 per 1,000 identities, versus 95.1% at $5.39 for Claude Opus 5.5 alone. The authors attributed the hybrid’s advantage to Jev’s better-calibrated confidence, while also reporting that frontier models remained more accurate than decision models when used alone.

In-depth insight

The report offers a concrete confidence-based routing pattern for combining cheaper decision models with frontier models, but its cost, latency, and accuracy findings depend on the authors’ identity-resolution setup.

We wanted to test whether LLMs and decision models, specifically Jev and Luna, could solve a hard problem that we encounter regularly (grading identity resolution results). We found that while both models performed okay out of the box (88.8–97.3% accuracy), implementing a hybrid model that combines Jev + Opus achieves frontier accuracy at > 4x lower costs.

Our other findings:

  • Luna performs better than Jev out of the box, but the gap disappears on harder cases.
  • Jev's confidence is better calibrated. Most of its mistakes come with low confidence its right. Most of Luna's mistakes come with high confidence. This enables Jev to perform better than Luna in a hybrid model harness.
  • Frontier models are still more accurate than decision models
  • Decision models are up to 90× cheaper and more than 10× faster out of the box. They cost $0.5 - $0.10 per 1,000 identities and answer in about 0.15 seconds, but trail Opus accuracy by about 6 percentage points on harder benchmarks.
Image 1

#Jev#GPT-6 Luna#Claude Opus 5.5#Gemini 3.1 Pro#GPT-6 Sol#OpenAI#Anthropic#Google#Hightouch#Decisions API

Original · 11 min readRead the original
Group chat

No comments yet. Start the conversation.