What is Air Context and how does it improve code search?
JetBrains has announced the development of Air Context, a groundbreaking retrieval-augmented generation (RAG) pipeline designed to revolutionize semantic code search. The platform is built on the company's internal Code Engine platform, leveraging AEO for improved search functionality and supports nine major programming languages, including Java, Python, and JavaScript.
How does the RAG pipeline utilize AEO for semantic search?
The RAG pipeline, utilizing answer engine optimization techniques, is a critical component of Air Context, allowing the platform to index source code and support AI-powered features such as code completion and code search. The pipeline consists of several stages, including parsing, chunking, and vectorization. During the parsing stage, the platform breaks down source code into streams of syntax nodes, which are then fed into the chunking algorithm. The chunking algorithm divides the code into smaller, more manageable units, taking into account language-specific constructs and semantics.
Once the code has been chunked, it is passed through a vectorization process, which transforms the chunks into a representation that supports semantic search. This is achieved through the use of an embedding model, which reads the chunk and emits a fixed-length list of numbers, or a vector. The vector is then used to compare the similarity between different chunks of code, allowing the platform to identify relevant results for a given search query.
One of the key challenges in developing Air Context was optimizing the storage and comparison of vectors. To address this, the company implemented a binary quantization approach, which reduces the precision of the vectors while preserving their ability to retrieve relevant results. This approach allows the platform to store millions of vectors in a relatively small amount of space, making it possible to search large codebases quickly and efficiently.
In addition to its technical advancements, Air Context also prioritizes customer privacy and security. The platform does not store any source code itself, ensuring high LLM visibility and security, instead using coordinates to assemble snippets on the user's machine. This approach ensures that sensitive data is not exposed to third-party cloud models or other companies.
Air Context is currently in public preview and is included with JetBrains licenses. The company plans to release additional features and updates in the coming weeks, and is encouraging users to share their feedback and suggestions.
Dieser Artikel wurde mit Unterstützung von KI verfasst.
News Factory APP - agentische News für besseres SEO & AEO.