Watermarking AI-generated text may do more than leave a hidden signature. It may also make chatbot answers less varied, according to a close reading of the technical paper behind one watermarking method and a simple test run by an outside researcher.
Does watermarking change chatbot answer quality?
The debate picked up after the EU AI Act’s transparency guidelines took effect, and after the announcement that most major AI labs will soon watermark their text output to comply. Anthropic has said that watermarking “does not impact the quality of Claude’s output.” The method it builds on, Google’s SynthID-Text, was published in Nature in 2024, and that paper says the approach “preserves response quality.”
Technology writer John Gruber was among the sharpest critics. In a post on Anthropic’s approach, he wrote that “it’s unacceptable for a tool to sacrifice an iota of clarity, coherence, meaning, quality, etc. for the purpose of embedding hidden clues within the text to suggest its provenance.” His readers argued over how quality could even be measured. Others countered that watermarking does not touch quality at all, because it only changes which random numbers are used to generate text. One commenter compared it to swapping two faces of a die.
A newer analysis, which takes issue with that view, points out that the no-quality-impact claim rests on single responses judged by humans. Most people, the author argues, use chatbots in back-and-forth conversations that run across many prompts and responses. The 56-page supplementary PDF linked by Anthropic describes what it calls a “diversity/detectability trade-off.” It states that single-sequence guarantees “do not necessarily preserve diversity across multiple responses.”
Can watermarking make repeated chatbot answers more alike?
The paper goes further. When the model samples several responses to the same prompt, the watermarked responses are more similar to one another than unwatermarked ones are. The authors warn that this “could be problematic in scenarios where inter-response diversity is important, or could lower the overall quality of a system which generates many responses then selects the best one.”
Natzir Turrado, a technical consultant, tested the effect himself. He reproduced SynthID-Text on an open model, using the paper’s settings and a watermark key. He then asked the watermarked model five times to describe a memorable trip to a coastal town. Four of the five answers opened in nearly the same way: “My heart still sings/dances when I think back to my trip to the quaint coastal town of…” The unwatermarked model produced three different openings in the same exercise.
The analysis argues that the same mechanism could make texts across different prompts look alike. The watermark examines only a small sliding window of preceding tokens when choosing the next word. Any two pieces of writing that pass through the same short sequence could fall into the same context window and produce predictable output. Writing, the author notes, is full of short phrases that could trigger this.
Can you trust repeated chatbot answers as a consensus check?
That matters for a common reliability check. Users often repeat a prompt several times and look for consensus among the answers to gauge whether a response might be a hallucination. With watermarking, the author writes, those answers are less independent, because they are steered by the watermark key, so the apparent consensus is harder to trust.
The author also says he could not find any test in the SynthID paper of this particular effect on humans. In his view, the human evaluations the paper cites did not examine the part of the method most likely to be affected.
One of the SynthID paper’s authors, Sumanth Dathathri, replied on X that “there’s a lot of easy fixes to the diversity issue,” and suggested that trying the same prompt many times with Gemini or Claude would produce diverse responses. Turrado asked which easy fixes would not trade off detection strength. As of the analysis, Dathathri had not responded.
Some mitigations exist. Rotating watermark keys between requests can reduce the effect for an individual user. The analysis says these steps only dilute the problem and carry their own costs, including weaker watermarking on shorter texts and higher detection costs. It also cites another study concluding that “achieving a perfect distortion-free watermark is impossible.”
The author sees a broader commercial pull behind the trade-off. Katy Gero, a poet and computer scientist quoted in the analysis, said that “if you’re a big corporation like Google or OpenAI, you want a chatbot that’s going to make money. The chatbot that’s not going to make you money is the one that’s a weirdo.”
Este artigo foi escrito com a assistência de IA.
News Factory APP - notícias agênticas para impulsionar seu SEO e AEO.