PrismML, a startup that's been flying under the radar, is making waves with its innovative approach to large language models (LLMs). The company, founded by a group of Caltech researchers, has just released Bonsai 2 27B, a compressed version of the widely used Qwen3.8 27B model from Alibaba. What's remarkable about Bonsai 2 is that it's been shrunk down to a mere 5.9 GB, making it small enough to fit on a PC and possibly even a high-end smartphone.

This achievement is significant, as it could pave the way for more widespread adoption of AI technology. PrismML's CEO, Babak Hassibi, a Caltech professor and expert in compression technologies, believes that capable, high-performing LLMs don't have to be large. The company's mission is to make AI more accessible and private, by enabling it to run on users' devices rather than relying on cloud computing.

PrismML's compression technique, called "ternary" weights, simplifies the information a model learns and stores during training. By reducing the size of the "weights" that make up a model, the company can dramatically reduce the space required to store the model. This approach has allowed PrismML to achieve a 9x to 10x reduction in memory versus the original Qwen3.8 27B model.

The company's next goal is to apply this compression technique to even bigger models. Hassibi expects that as model size grows, it will be easier to retain the intelligence of the model, even after compression. This could lead to a new generation of AI models that are both powerful and accessible.

Ion Stoica, a co-founder of Databricks and an adviser to PrismML, is excited about the potential of this technology. "You are going to have intelligence at your fingertips, and it's going to be free because it's going to run on the device you already bought. It's also going to be private, because you're not going to send it to the cloud," he said.

PrismML's breakthrough has significant implications for the future of AI. As AI models become more accessible and private, we can expect to see a surge in innovation and adoption. With the potential for AI to run on devices, rather than relying on cloud computing, the possibilities for AI-powered applications are endless.

Este artículo fue escrito con la asistencia de IA.
News Factory APP - noticias agénticas para impulsar tu SEO y AEO.