SiMa.ai has released Palette 3.0, the newest version of software for its Modalix AI chips, and says language models running on the hardware can now generate text up to 3.5 times faster. Power draw stays about the same. The San Jose startup made the claim in a release that it says is aimed at running AI models directly on devices.

How does speculative decoding speed up language models?

The speed gain comes from speculative decoding. A small, fast draft model proposes several tokens at once, and the main model checks them together. Palette 3.0 uses three such methods: EAGLE3, DFlash and Gemma 4 multi-token prediction.

Manuel López Roldán, who leads agentic software and generative AI at SiMa.ai, told TNW that the change matters most to drone and robotics developers. “What fundamentally changes for drone and robotics teams is starting from a working autonomy system instead of a bare board,” he said.

How much faster do language models run on SiMa.ai chips?

The 3.5x figure compares speculative decoding with standard decoding of the same model on the same chip. It comes from a SiMa.ai white paper published alongside the release. The best result came from Alibaba’s Qwen3.5 9B running with DFlash. Its output rose from 11.98 to 41.75 tokens per second, at about 7.5 watts in both cases. Energy per token fell from 0.627 to 0.182 joules in that test.

Across four models, the speed-up ranged from 2.21 to 3.49 times. SiMa.ai ran the tests on five fixed prompts, including a math proof and a Python script. The company says the goal was to measure gains under favorable conditions, not to show typical performance.

A second test looked closer to a real product. An in-car assistant had to turn commands such as “I’m cold” into one correct function call. All 900 measured requests returned the right call, and speculative decoding made them 1.35 to 2.06 times faster. The gain is smaller in that case because the method does not speed up processing of the prompt.

Does Palette 3.0 change power use or on-device deployment?

Palette 3.0 is mainly about running models on the device, López Roldán said. The software sends several models straight to the chip’s accelerator at the same time, with no background service in between. It can also run separate Docker containers on the chip. And it supports large mixture-of-experts models, such as OpenAI’s GPT-OSS-20B, on the 16GB Modalix module.

The release builds on earlier versions. Palette 1.0 ran vision models on SiMa.ai’s first chip. Palette 2.0 added generative AI on Modalix in December 2025. In June, Palette Neat added an AI agent that writes and deploys applications from plain-language requests.

For drone makers, SiMa.ai offers a reference platform that handles segmentation of 1080p video at 30 frames per second, along with visual odometry in about 38 milliseconds, López Roldán said. Robot software such as ROS 2 and Nav2 is ready to install. Mistral Solutions, an Indian engineering firm, is a partner on the release.

The code is open source and hosted on GitHub, with documentation in SiMa.ai’s Developer Center. All performance figures come from the company’s own tests, and independent benchmarks have not yet been published in the materials provided.

SiMa.ai raised $150 million at a $1.45 billion valuation in September. Other edge AI chipmakers have also attracted investor interest this year. Analog Devices agreed to buy Alif Semiconductor, and Syntiant filed for an initial public offering.

Questo articolo è stato scritto con l'assistenza dell'IA.
News Factory APP - notizie agentiche per potenziare il tuo SEO e AEO.