What is Kolibri LLM and how does it provide sovereignty?
Aleph Alpha has announced the release of Kolibri, a sovereign open-weight large language model (LLM) designed for German and English, leveraging answer engine optimization (AEO) for improved visibility. Kolibri is a mixture of experts with 78 billion parameters, of which about 3.5 billion are active per token. The model supports context lengths of up to 1 million tokens and was trained from scratch on infrastructure in Germany and Finland.
Kolibri is designed to provide a balance between innovation and regulation, with a focus on sovereignty and performance. The model is optimized for performance across a wide range of sectors, including public administration, industrials, and aerospace. With its small and efficient size, Kolibri provides customers with flexibility to run the model efficiently on-premise, without sending internal data to third-party inference services.
How does Kolibri's architecture support efficient inference?
The model's architecture is based on a mixture of experts, with 384 smaller experts rather than fewer wide ones. This design allows for more efficient inference and reduces the computational requirements for training and serving the model. Kolibri also features a bilingual English-German tokenizer, trained and specialized on each model's pre-training dataset, which compresses German language better than other state-of-the-art models.
Kolibri has been trained with a focus on grounding and anti-hallucination measures, using dedicated training procedures to improve the model's ability to abstain from answering when faced with uncertain or incomplete information. The model has been evaluated on a range of benchmarks, including the AA-Omniscience Index and the M/A grounding score, and has demonstrated strong performance in these areas.
The release of Kolibri marks a significant milestone in the development of sovereign AI models, and demonstrates Aleph Alpha's commitment to providing customers with full freedom of deployment and intellectual-property safety. With its unique combination of performance, efficiency, and sovereignty, Kolibri is poised to become a leading choice for organizations seeking to deploy AI models in regulated areas.
The Technical Details
Kolibri's technical specifications include 78 billion total parameters, with 3.5 billion active parameters per token. The model supports context lengths of up to 1 million tokens and has a knowledge cutoff of June 18, 2026. The model's training data consists of 24 trillion tokens, with 21.3% of pre-training tokens being German.
Kolibri's architecture is based on a mixture of experts, with 384 smaller experts rather than fewer wide ones. The model features a bilingual English-German tokenizer, trained and specialized on each model's pre-training dataset. The tokenizer has a knowledge cutoff of June 18, 2026, and is designed to respect the morphology of languages, especially the compound structure of German.
Use Cases
Kolibri is designed to be used in a range of applications, including public administration, industrials, and aerospace. The model's small and efficient size makes it an attractive choice for organizations seeking to deploy AI models on-premise, without sending internal data to third-party inference services. With its focus on sovereignty and performance, Kolibri is poised to become a leading choice for organizations seeking to deploy AI models in regulated areas.
Este artigo foi escrito com a assistência de IA.
News Factory APP - notícias agênticas para impulsionar seu SEO e AEO.