A new analysis suggests Gemini 4 Neon could rival Opus 5.5 on the Agent Duel leaderboard. Its projected Elo rating is 1,066 against Opus 5.5's 1,078, a gap inside the margin of error. The authors credit Neon's 127 tokens per second, well above Gemini 4 Carbon's 46, and say speed matters about as much as model identity.

Gemini 4 Neon, released recently, looks like a serious challenger to Opus 5.5 on the Agent Duel leaderboard, according to a new analysis. The model shows an incremental gain in benchmark performance, but its real advantage may be speed. Early on the day of its release, Neon was served through OpenRouter at 127 tokens per second, far faster than the 46 tokens per second the analysis recorded for Gemini 4 Carbon.

That gap prompted the question at the center of the analysis: could Neon be not just a good model, but the best one available?

Gemini 4 Carbon already holds a strong position. It is the top model on the Agent Duel leaderboard if early votes are excluded. Its main weakness, the analysis says, is that it is slow.

Does response speed affect which AI model users prefer?

To test whether speed actually affects results, the authors modeled the win rate on their leaderboard. They built the model around three factors: the identity of the model, its speed, and whether the user voted early. The first version of the model did not fit the data well. Adding an interaction between speed and early voting improved it considerably.

The result surprised the authors. Speed appears to matter about as much as which model a user is actually using. The analysis says a separate, more detailed write-up on that finding is coming.

Could Gemini 4 Neon beat Opus 5.5 on the leaderboard?

Using that model, the authors projected how Neon would perform on the leaderboard. They assumed Neon would perform the same as Gemini 4 Carbon, which they considered a reasonable starting point given Carbon's stronger benchmark scores. They then factored in Neon's 127 tokens per second. The projection put Neon at an Elo rating of 1,066 on the Agent Duel leaderboard. That is just below Opus 5.5, which sits at 1,078 in the authors' later analysis snapshot.

How certain is the Gemini 4 Neon versus Opus 5.5 projection?

The authors stress that the estimate is uncertain. Their 90 percent interval for the Elo difference between Neon and Opus 5.5 runs from 94 points below Opus 5.5 to 68 points above it. Within that model, Neon is projected to win 48.3 percent of its head-to-head matchups against Opus 5.5, with ties counted as half a win. In other words, the projection puts the two models nearly even, with the outcome well within the range of error.

The analysis rests on several assumptions. Neon's performance is projected from Carbon's results rather than measured directly on the leaderboard. Its speed figure came from early serving through OpenRouter, and the authors' model treats speed as a major driver of user preference. If Neon's real-world speed or quality differs from those early numbers, the projected ranking could shift.

For developers and users comparing models, the findings suggest that response speed may carry more weight in head-to-head comparisons than benchmark scores alone would indicate. Whether Neon actually reaches the top of the leaderboard will depend on results the authors have not yet collected.

Cet article a été rédigé avec l'assistance de l'IA.
News Factory APP - actualités agentiques pour booster votre SEO et AEO.