Baseten's Base Labs research arm has launched a new safety infrastructure standard, partnering with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastructure for open-weight models. This move comes as concerns grow over the safety of open-weight models, which can be made dangerous by removing their safeguards through a technique known as abliteration.
The scale of the problem is significant, with Hugging Face, a host of open-source AI models, currently listing over 6,000 abliterated models. Base Labs will develop and publish methods for training and monitoring open models, framing their future work as a "standard" for open models that is transparent and built into how models are trained and deployed, rather than bolted on afterward.
According to Baseten, "openness provides more visibility into the behavior of models and, most importantly, greater means of turning safety research into actionable and transparent controls than closed-source." The companies haven't disclosed how the partnership will work technically, but Goodfire has stated that "safety must be built into open models and provided by those who serve them." Goodfire, which specializes in opening AI's "black box" to explain how models make decisions, is likely to play a key role in this aspect of the partnership.
Baseten, an AI inference provider, has raised significant funding, including a $1.5 billion Series F in June, which vaulted its valuation to $13 billion. Partner Goodfire AI has also raised substantial funds, including a $150 million Series B led by B Capital earlier this year, to advance its model interpretability platform.
Looking ahead, Baseten is putting out an open call to the broader developer ecosystem to contribute to the framework. The company notes that "together, we are building an ecosystem of open models that are safe and accessible to all." This partnership and the development of a safety standard for open-weight models could have significant implications for the future of AI safety and transparency.
Dieser Artikel wurde mit Unterstützung von KI verfasst.
News Factory APP - agentische News für besseres SEO & AEO.