Modal plans to run its AI agent sandboxes on Nvidia's new Vera CPU, its first move away from x86 chips from Intel and AMD. In its own unverified tests, Vera handled up to twice as many tasks as an Intel server when packed with containers. Modal has not said when Vera-powered sandboxes will be available.
What chip will Modal use for its AI agent sandboxes?
Modal plans to run its AI agent sandboxes on Nvidia's new Vera CPU, a switch that marks the company's first departure from x86 chips made by Intel and AMD. In Modal's own testing, the chip handled up to twice as many tasks as an Intel server when the server was packed with containers. The company announced the plan in a blog post on Monday. The results have not been independently verified.
Vera uses Arm's chip design, so the change is more than a hardware swap. Modal will need to modify its infrastructure and the system it uses to build software images. Nvidia hand-delivered the company's first Vera server to its San Francisco office last week, according to Modal.
Sandboxes are isolated computers where AI agents run the code they write. Modal said the product brings in $300 million a year. The company disclosed that figure last week when it raised $750 million at a $15.75 billion valuation.
Why are AI agent sandbox workloads hard to plan for?
Sandbox workloads are difficult to plan for. An agent might sit idle while it waits for an AI model to respond, then suddenly demand ten times more computing power during a software installation. Hundreds of these sandboxes run side by side on each server, which means one customer's spike can slow down another customer's work.
To test the chip, Modal built a benchmark of 50 coding tasks, including compiling software and searching files. When the tasks ran one at a time, Vera was often faster than chips from Intel, AMD and Amazon. It was not always faster, and the margin was not large.
How much faster is Nvidia's Vera CPU in Modal's tests?
The gap widened under heavier loads. When Modal packed the server with containers, Vera kept finishing tasks at the same speed while rival chips hit a ceiling, the company said. Modal credits Vera's high memory bandwidth for the difference.
That performance matters for two uses, according to Modal: background coding agents and reinforcement learning. In reinforcement learning, expensive GPUs often sit idle until CPU work finishes, so a faster CPU keeps those accelerators busy.
Nvidia unveiled Vera in March as a CPU built for AI agents. It has 88 cores, and the company has named Anthropic and OpenAI among its first users. Perplexity has said it will run agent work on the chip after reporting coding tasks that ran 1.5 times faster.
Competitors are also building sandbox products. Baseten bought Blaxel in September. Modal has not said when Vera-powered sandboxes will be available to customers.
This article was written with the assistance of AI.
News Factory APP - agentic news to boost your SEO & AEO.