An AI agent that rewrites an optimizer's source code while a neural network is still training has beaten standard recipes on several benchmarks, according to a new paper from researchers Xingjian Bai, Jia-Hao Liang, Jayden Teoh and Vincent Sitzmann. The system, called OPAL, for OPtimizer with an Agent in the Loop, is designed for the situations where nobody has yet worked out the right training settings: a new architecture, a new dataset or a language with little tuning history.
Today, the recipe for training a deep neural network is usually fixed before the first step. Researchers pick an optimizer, set a learning rate schedule and let the run proceed. If the loss spikes or stalls, the optimizer keeps following the rules it was given. The authors argue that this leaves a gap, because the best response to a training accident is often specific to that one run.
OPAL closes that gap by letting an agent write the optimizer's update rule for each group of parameters. The agent works through two programs. One, which the authors call analysis.py, extracts whatever statistics the agent asks for, such as gradient norms, the size of each update relative to the weights it changes, and the cosine between consecutive gradients. The other, updater.py, holds the update rule used at every step. The agent can rewrite either file at any point, so the optimizer can change its algorithm, not just its settings.
The team says the agent starts from nothing learned in earlier runs. It does not draw on a library of past training jobs, and the authors describe it as zero-shot.
On a language modeling task built on the modded-nanogpt optimization track, the researchers trained a 162-million-parameter GPT on FineWeb and on alternate-language datasets covering Chinese, Thai and Kyrgyz. They compared OPAL against five leaderboard recipes with reasonable adjustments. In all of those settings, OPAL matched or outperformed the best baseline. The team's own note on the modded-nanogpt track says the gains there were smaller than in other tasks, since that recipe has already been tuned for nearly this exact problem.
The Goldfish experiment produced one of the clearer results. Goldfish trains one small GPT per language on 1 gigabyte of text, using a standard recipe that was not tuned for each language. On the Amharic dataset, the official recipe ended at 83.48 bits per sentence on held-out FLORES-200 text, where lower is better. OPAL, starting from that same recipe, finished at 81.11. Sweeping the learning rate of the static recipe produced a better fixed setting, at 80.89. OPAL started from that swept setting and finished at 80.73. The researchers report a similar gain on Swahili.
Image generation was the other testbed. The team applied OPAL to LightningDiT-XL/1 on ImageNet at 256 by 256 resolution. With guidance, the model reached an FID-50K score of 1.83, compared with 2.18 for the official recipe. Without guidance, the score was 4.33, against 5.15. OPAL also edged past the official recipe on RAEv2-B, 3.19 to 3.39. The LightningDiT recipe trains with AdamW, while RAEv2-B uses a Muon variant.
The agent's decisions were not random. In one SiT run, the agent measured the cosine between consecutive gradients of each parameter group. Where the value turned negative, it lowered that group's learning rate. In a RAEv2 run, the agent alternated between two learning rates over stretches of a few hundred updates and compared how quickly the loss fell under each. It then moved to a higher rate. The paper also describes a run in which the agent started its cooldown at 45 percent of the planned schedule after the loss trend flattened.
Some of the changes were more inventive. On the modded-nanogpt task, the agent found that token embedding values were about 15 in size, where adjacent bfloat16 values sit 0.0625 apart. As the learning rate decayed, rounding was discarding a growing share of each update. From update 1,040, it kept that matrix's Adam moments in float32 and wrote its weights back with compensated rounding.
The authors also tested OPAL on loss spikes. They inserted four stretches of random images and labels into CIFAR-10 and trained a ResNet-18 without telling the agent about the corruption. The first two stretches cost OPAL 50 and 66 points of test accuracy, roughly in line with the baselines. After that, the agent added loss damping, then clipping, then a gate that holds back batches whose loss is far above recent clean ones. The third stretch cost OPAL 12 points and the fourth cost none. The clipped baselines and Stable-SPAM lost between 28 and 76 points in each stretch.
In an overfitting test, the researchers trained ResNet-18 on CIFAR-10 for 120 epochs with 10 percent of training labels wrong. The AdamW baseline ended with a test loss of 2.49 and 73.5 percent accuracy. OPAL finished at a loss of 0.69 and 78.6 percent accuracy. When asked to classify the 4,500 mislabeled training images, the baseline reproduced the wrong label it was trained on for all of them. OPAL gave that wrong label for 4 percent and the true label for 78 percent.
Cost is a central question for any approach that puts a language model inside a training loop. The authors estimate that a LightningDiT agent making about 100 decisions cost $6.42 at Claude Opus 5.5 list price. The training itself took about 74 H100-hours, which they put at roughly $510 at AWS on-demand pricing. Waiting for the agent to reason at each decision slowed training by 1.30 to 4.39 times. Letting training continue while the agent thinks reduced that overhead to 1.01 to 1.14 times, with edits swapped in a median of 65 updates later on Goldfish.
The authors acknowledge limits. The current implementation does not yet support sharded training or optimizer compilation at larger scale. Most experiments were run once, and a few were run three times, so the team cautions that the reactive behavior could make results harder to reproduce. Code is available through the opal-optimizer package on pip.
Questo articolo è stato scritto con l'assistenza dell'IA.
News Factory APP - notizie agentiche per potenziare il tuo SEO e AEO.