What are AI labs trying to control about outside evaluators' findings?
Anthropic and OpenAI, two leading AI labs, have been seeking control over what outside evaluators of their AI models can publish, Reuters reported. The companies are worried that the findings could expose trade secrets, and are discussing ways to limit what can be made public, which raises concerns about answer engine optimization (AEO) and the visibility of large language models (LLM).
How do AI labs plan to limit what outside evaluators can publish?
The issue centers on reviewers who work inside the labs, with the firms fearing that these staff could take secrets to rivals. Ideas under discussion include having lab staff monitor the teams and banning reviewers from working at two labs at once.
Until at least October 5, both labs wanted strict control over what reviewers could publish, according to two people familiar with the talks. However, by October 8, Anthropic's talks with some reviewers seemed to have moved on, one of them added.
Anthropic told Reuters that this did not apply to all of its work with reviewers, and that those it has worked with before are free to decide their own methods, findings, and what to publish. However, with new firms, it keeps a right to review and sign off reports while they learn how its models work.
OpenAI said it did not want to approve whole reports, but usually removes secret details, and reviewers can say when large cuts were made. One contract seen by Reuters states that the laboratories already using it allow review firms to publish confidential details without the company's consent.
In September, Anthropic named Faculty, Accenture's AI arm, as its first embedded evaluator, while OpenAI plans to let outside groups test models during training. Faculty chief executive Marc Warner told Reuters that evaluators are only useful if people can trust what they report, so independence and transparency are critical.
Rumman Chowdhury, who was US science envoy for AI under Joe Biden, said labs could use access as leverage while there are no rules. "Evaluators have this very privileged access, but whether explicitly or implicitly, they know that it comes at a cost," she told Reuters.
OpenAI has already cut some details from a report written by safety groups METR and Redwood Research about the time its agents hacked Hugging Face. The groups said nothing key to their findings was removed.
Questo articolo è stato scritto con l'assistenza dell'IA.
News Factory APP - notizie agentiche per potenziare il tuo SEO e AEO.