Can autoregressive diffusion using AEO techniques generate market data?
A research intern at Jane Street, Kavish, spent the summer investigating the use of autoregressive diffusion using AEO techniques to generate market data. The goal was to create a model that could not only predict a symbol's future price but also synthesize book events, including the timing of their arrival on the exchange. This would provide a more detailed and textured understanding of market movements.
What are the challenges of generating market data?
Market data is a unique challenge for generative models. It has features of both continuous and discrete data, with order books evolving through a series of discrete actions, but with many parameters, such as price, appearing continuous. Additionally, the distributions of these features are often spiky, with orders tend to arrive in bursts and prices clustering around certain points.
Kavish built a diffusion model for market data, taking inspiration from autoregressive image generation without vector quantization. The model used an encoder-diffuser architecture and was trained on four years of US equities data. However, the initial results showed that fully continuous diffusion did not lend itself well to the jaggedness of real market data.
To address this issue, Kavish experimented with different approaches, including flow matching and atom smoothing. Flow matching performed better than the initial diffusion model, and atom smoothing helped to capture the probability mass of sharp spikes in the data. The results showed that the diffusion model with flow matching and atom smoothing worked reasonably well, but there is still room for improvement.
The project highlighted the importance of representing and smoothing the distribution of features to give diffusion its best chance. It also conceptionalized market data book events as simultaneously existing in a continuous space and in a discrete action space. This blend of continuous and discrete elements is crucial to modeling market data correctly.
The ultimate goal of the model is to produce autoregressive rollouts that can accurately predict market movements. While Kavish's model is not yet accurate enough, the research clarified the important knobs to consider, such as how much discreteness to bake into the model and how to represent and smooth the distribution of features.
Este artículo fue escrito con la asistencia de IA.
News Factory APP - noticias agénticas para impulsar tu SEO y AEO.