In October, more than one in four requests on Vercel’s AI model router went to decision models rather than large language models. For a type of AI that was invented only about a month earlier, that is a large share of the work.
What are decision models?
Decision models are LLMs that don’t produce output tokens. Instead, they use their general intelligence to choose quickly and cheaply between preset options, and they report how confident they are in the choice. That makes them useful in production systems, where a fast decision can matter more than the detailed reasoning a slower, more expensive frontier model would provide.
TypeSafe built its business on that idea. The company became a $7.5 billion company after releasing its Jev family of models, which still dominates usage where outside metrics are available. OpenRouter, which tracks traffic to several model providers, handled more than 1.5 billion queries to TypeSafe’s models last week.
Others have noticed. Dozens of decision models have appeared since Jev launched, and the barrier to entry is low. One Amazon engineer built one over a weekend, and soon every major cloud provider had its own option. On Tuesday, Microsoft CEO Satya Nadella said the company is rolling out its decision model, Microsoft-Decision-1.1, for wide use on Azure immediately.
Smaller players are also jumping in. Sqwish, a U.K.-based AI startup, behind a decision model called kettei fat, announced a $3 million pre-seed round. Co-founders Ushnish Sengupta and Federica Freddi met as graduate students at Cambridge. They had been developing small LLMs when Jev arrived, and they concluded that their architectures could be repurposed as a capable decision model.
Why are startups betting on customization?
Data is the common thread. TypeSafe CEO Diogo Almeida told TechCrunch that the hardest part of building Jev was calibrating its judgment, which the company did with statistically adjusted synthetic data. Sqwish also works with synthetic data, but it is pitching something more specific: built-in post-training tools that let a customer’s own data improve the model’s judgment over time.
“What makes us not a commodity, I think, is the fact that we are offering our customers the same power…they can keep uploading their production data, and just the same way that we trained our models, their models can also keep getting better at the tasks that they care about,” Sengupta told TechCrunch.
Can benchmarks reliably rank decision models?
Ranking these models is difficult. Sqwish’s kettei fat has performed well on some benchmarks, but many builders say the real test happens on the job. Developers can generally swap one model for another with little effort and see how it performs in their own setting. Sengupta framed benchmark results as an invitation to try the product, with continued tuning as the reason to stay.
Microsoft’s team takes a similar view. Microsoft-Decision-1.1 was built to balance cost, latency, intelligence and calibration. Achint Srivastava, the engineering vice president who led the project, said too much focus on benchmarks is a mistake.
“What we see is that ‘benchmaxed’ models fall flat under real enterprise usage very quickly,” Srivastava wrote in an email. “Our testing underscores just how challenging it is to deliver accuracy, reliability, speed and cost efficiency in a single model. Microsoft-Decision-1.1 and Jev are among the few that perform strongly across all four dimensions, even when evaluated on ‘held out’ or unseen benchmarks reflective of real use cases.”
For startups, the crowded field is also a chance. Freddi expects future AI systems to pair what she calls System One intelligence, such as decision models, with System Two intelligence, such as frontier LLMs. She sees a large market for the faster layer and described it in stark terms compared with the largest AI companies.
“Even in that future, System One is maybe a $10 billion, $50 billion [opportunity], and OpenAI is doing this IPO with you know $1 trillion, $2 trillion market value in mind,” she told TechCrunch. “This is all we care about; whereas for them, this is just a side project.”
This article was written with the assistance of AI.
News Factory APP - agentic news to boost your SEO & AEO.
