Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Amazon swaps Sonnet 4.6 for Strands Decider in ticket labeling

AI summary

Amazon engineers reported that Amazon and AWS teams replaced Claude Sonnet 4.6 with the Strands Decider E4B decision model for high-volume internal ticket labeling. Amazon's measurements showed comparable results, average call time falling from 4,215 ms to 897 ms, and estimated list-price cost dropping about 98%; these figures and the claimed annual savings of millions of dollars were not independently verified.

In-depth insight

The deployment shows how a purpose-built decision model can replace a general-purpose LLM for closed-set classification, with Amazon reporting lower latency, fewer tokens, and substantially lower serving costs.

For a while, Amazon and AWS teams relied on Claude Sonnet 4.6 to label tickets in an internal system without breaking the bank.

Sonnet 4.6 fit the goldilocks choice of balancing intelligence with cost. Considering it dealt with one of the world’s largest internal ticketing systems, the nine month old model did well. However, two Amazon engineers recently swapped out Sonnet 4.6 for Strands Decider, a decision model we launched a month ago, that performed just as well at 98% less cost. This change will save millions of dollars of token costs in a year.

Carr and Mehul, who work on the internal ticketing system, walk through how this happened below.

Replacing an LLM’s job with a decision model

We had Sonnet 4.6 labeling tickets at enormous scale. The tickets could be customer escalations, capacity requests, automated alarms, security findings, or anything else across the company that needed to fit within 20 category types. The agent ran with a long system prompt on every ticket, generating JSON and reasoning against a fixed set of labels.

A decision model can’t generate text. It’s first given some state and a question with a set of options, then the decision model returns a probability for each option plus a confidence score. We saw this method fit perfectly with ticket labeling: the state is the ticket, the options are the label catalog.

#Amazon#AWS#Strands Decider E4B#Claude Sonnet 4.6#Gemma 4#Amazon SageMaker

Original · 4 min readRead the original
Group chat

No comments yet. Start the conversation.