AI labs have built their reputations on benchmarks, the test results that are supposed to show whether one model is better than another. But the methods behind those tests are complicated and still maturing, and the industry needs to make them more consistent before anyone can rely on them, according to Irene Solaiman, Hugging Face’s chief policy officer. She spoke at TechCrunch Disrupt last week.
Can AI benchmark results be trusted?
So can the tests being released today be trusted? Solaiman said the word “trust” carries a lot of weight. For her, a key question is whether an evaluation can be reproduced. Do outside researchers have the access they need to the model itself? “This is where open weights come in in a big way,” she said.
Why is reproducibility hard in AI model evaluation?
Reproducibility has long been one of the thorniest problems in model evaluation, and it remains a subject of heavy debate across the industry. Even when a single test is sound, a harder question follows: how does it compare with other evaluations? Solaiman said the field has not reached that point. “They are an evolving science — where we don’t know how to meaningfully compare them to each other,” she said.
Tests also lose their usefulness quickly. “They start to get ineffective relatively quickly, and we don’t have as much of a thriving ecosystem as we need,” she said. “There’s this hunger for an embedded evaluator ecosystem now, but the qualifications that determine who is an effective evaluator across what tasks need a lot more standardization.”
What is an embedded AI evaluator?
The embedded evaluator idea has gained ground over the past several months. Under the model, third-party observers, who are supposed to be neutral, work inside AI organizations. The push has coincided with a run of safety incidents that have raised concerns about the potential harms of the technology. Hugging Face was itself the target of one of those incidents.
Instead of pushing for active government regulation to identify and reduce safety risks, many labs have backed the third-party approach. Doubts remain about how neutral or effective those evaluators will be. Anthropic’s decision to hire Accenture as its first evaluator drew significant backlash, and some critics mocked the idea that the consulting firm was qualified for the job.
Evaluation is also changing in scope. Testing is moving beyond the model alone to cover the entire system built around it, including the harness and the agent. The harness is the tool a user relies on to access the model, and it can shape performance a great deal. Solaiman said a weaker model paired with a better harness could easily outperform a frontier model.
That shift creates its own problems. “Agentic evals are really hard and really expensive,” Solaiman said. “We have just more evaluations for models. … We don’t have as many evaluations for harnesses, and we are developing better evaluations for the holistic agent — because that’s what a lot of folks are getting value out of,” she added.
For now, the evaluation space is largely unregulated in practice, with each group setting its own approach. If benchmarks are going to remain useful to industry leaders, the field will likely need a more uniform, industry-wide approach, one where everyone is working from the same playbook.
Cet article a été rédigé avec l'assistance de l'IA.
News Factory APP - actualités agentiques pour booster votre SEO et AEO.