Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider, a decision model, to label tickets in an internal ticketing system. The team estimates the change cuts per-ticket cost by roughly 98% and average call time by 78%, and says E4B outperformed the production Sonnet setup on a separate test set. The approach suits closed-choice tasks with high volume.
Amazon has replaced one of its own large language model workflows with a much smaller system, and the company says the switch cuts costs by 98% without hurting accuracy. Two Amazon engineers, identified in the write-up as Carr and Mehul, swapped Claude Sonnet 4.6 for Strands Decider, a decision model the company launched a month ago. The change applies to an internal ticketing system that handles customer escalations, capacity requests, automated alarms and security findings.
Sonnet 4.6 had been labeling those tickets for a while, and the engineers say it did the job well. The model is nine months old, and the team considered it a good balance of intelligence and cost. But the volume was enormous, and each ticket required a long system prompt, a JSON response and reasoning against a fixed set of 20 labels. That adds up quickly.
How does a decision model differ from an LLM?
The engineers found a different approach in a decision model. Instead of generating text, a decision model receives a state and a question with a set of options. It then returns a probability for each option along with a confidence score. For ticket labeling, the ticket is the state and the label catalog becomes the options. The team started with the original Strands Decider 2B model, but a Gemma 4 build known as E4B produced clearly better results on a test set, so they switched to it. They then adjusted how confident the model needed to be before applying a label. On a separate set of the company's tickets, E4B outperformed the production Sonnet setup.
The engineers did not simply flip a switch and hope for the best. Their labeling function reads a three-way setting from AWS AppConfig on every message. In BEDROCK mode, Sonnet labels every ticket and Strands Decider is not called. In DUAL mode, both models run in parallel, the Sonnet results are applied to the ticket, and the two outputs are compared. In DECIDER mode, Strands Decider labels every ticket and Bedrock is not called. Running both models on real traffic and reviewing every disagreement gave the team more useful information, they wrote, than an offline evaluation would have.
The performance gains were substantial. Average time per call dropped from 4,215 milliseconds with Sonnet 4.6 to 897 milliseconds with E4B, a 78% improvement. Total tokens per ticket fell from 6,724 to 2,010, a 70% reduction. The team attributes much of that drop to the output format. Strands Decider produces a single choice rather than a JSON answer, and its prompt is far shorter. Production label descriptions alone exceeded the model's 4,096-token window, so the engineers rewrote each of the 20 labels as a one- or two-sentence criterion. On long tickets, the decision model drops the oldest comments to fit, though the engineers said the ticket data otherwise matches what Sonnet receives. Average Lambda execution duration dropped 74% on the day the team turned off Sonnet usage.
How much cheaper is a decision model than Claude Sonnet?
Cost is where the numbers become striking. At published us-east-1 on-demand prices, Sonnet 4.6 costs about $0.0222 per ticket, or roughly $22,200 per million tickets. The E4B model runs on SageMaker ml.g5.xlarge instances, which cost $1.408 per instance-hour. The engineers estimate the model costs about $0.0005 per ticket, or about $500 per million tickets when they keep headroom for bursts. That works out to about 44 times cheaper than Sonnet. The estimate leaves out Lambda costs, though the team noted that Lambda also runs for less time per ticket now.
The engineers also flagged a quirk that surprised them. Sending identical content with JSON keys in a different order shifted the output probabilities by as much as 0.19. The team now sorts keys on every request, and they advised anyone serving a decision model through an API to check for the same issue before comparing results.
When is a decision model not a replacement for an LLM?
The engineers were careful to say the approach will not suit every task. It worked because the work is a closed choice from a known catalog, the volume is high and latency under the old setup mattered. If labels require explanation text, if the catalog changes weekly or if the useful context cannot fit into a few thousand tokens, the team wrote that a decision model probably cannot replace an LLM. The engineers said the three-way switch is the part they would reuse regardless of which model they choose.
Questo articolo è stato scritto con l'assistenza dell'IA.
News Factory APP - notizie agentiche per potenziare il tuo SEO e AEO.