How many vulnerabilities has Project Glasswing found?
Seven months after Anthropic launched Project Glasswing, the evidence for its vulnerability-hunting claims is thin, according to an independent tracker that has followed the project since shortly after its April 7 launch. The tracker counts 442 vulnerabilities attributable to Anthropic's AI models, a figure that works out to about 0.5% of the 93,450 CVEs published this year. Only two of those 442 have been exploited in the wild.
The gap is widest in Anthropic's own public ledger. The company reports 37,966 findings, but only 7,349 of them, or 19.4%, have been made public. The ledger lists just 217 as fixed. Since the tracker's September update, Anthropic has added nearly 12,000 findings and recorded only 15 more fixes.
Duplicates are the more worrying number, the tracker argues. Roughly 1,600 findings, about 4.4% of the total, are marked as duplicates, which is nearly eight times the number of fixes. Another 419 have been withdrawn. In September, the duplicate count stood at two. The tracker's explanation is simple: the longer a flaw sits unreported, the more likely someone else finds and fixes it first.
A rising accuracy rate on the ledger
Is Anthropic's claimed true positive rate accurate?
Anthropic's claimed true positive rate has climbed from 90.8% at the ledger's May launch to 94.2% in its most recent update on November 2. The tracker says that is hard to square with the record. More findings have been withdrawn or marked as duplicates than have been fixed, which suggests the triage data that feeds the true positive figure may be incomplete. The tracker also says the ledger contains a bug that inflates the number of findings marked as disclosed, and it urges readers to check using the "disclosed" filter.
The company has acknowledged that manual review cannot keep pace. In its October 8 post announcing OSS Scanner, Anthropic said it had found more than 29,000 candidate vulnerabilities but had reviewed only about 6,000 by hand. It said it remained "bottlenecked on our human capacity to validate these findings," and had already sent nearly 5,000 unverified reports to maintainers.
Who checks Anthropic's AI vulnerability reports?
OSS Scanner is a self-service tool that open source projects can opt into. Maintainers request a scan and triage the results themselves. Anthropic states that the reports are "fully model-generated, without human review or triage" and that "it is possible reports will be incorrect or invalid." The tracker's view is that the bottleneck has not gone away. It has moved to the maintainers.
What the curl batch shows
The clearest test so far came from curl. Its maintainer, Daniel Stenberg, announced that the project had received 45 vulnerability reports from Anthropic, with one rated medium severity and the rest rated low. Stenberg is publishing them at a deliberate pace "to soften the impact for the team."
Of the 34 reports published so far, 19 have been judged not to be vulnerabilities, six have been confirmed as low severity, and nine are still awaiting assessment. That puts the false positive rate at 55.9% and the confirmed rate at 17.6%. Stenberg wrote in May that five Mythos findings had amounted to a single confirmed vulnerability.
The tracker also points to a mismatch in severity. In September, it found that Claude rated 91.5% of ledger findings as critical or high, while maintainers rated only 51.3% that way. Glasswing launched with a focus on high- and critical-severity flaws.
The tracker says it is not dismissing AI's value in vulnerability research. Its conclusion is narrower: seven months in, the results do not yet match the launch message. The CVEs are a small share of the year's total, almost none are being exploited, fixes have stalled, and the work of checking findings has shifted to the open source volunteers the project said it would help.
Cet article a été rédigé avec l'assistance de l'IA.
News Factory APP - actualités agentiques pour booster votre SEO et AEO.