🚨【GPTZero CTO 深度拆解 AI 文本水印技术:原理、缺陷与监管真相】
随着 Anthropic 宣布在其 Claude 模型中全面嵌入不可见的文本水印与元数据,AI 内容的追踪与鉴别再次引发行业热议。AI 检测工具 GPTZero 的首席技术官(CTO)Alex Cui
@alexcdot 发文,详细拆解了 Anthropic
@AnthropicAI 、Google
@Google 及 OpenAI
@OpenAI 等顶级 Labs当前采用的文本水印生成原理、工程挑战,并对未来 AI 水印的破局与监管走向作出了深度预测。
一、 核心原理:KGW 算法如何“神不知鬼不觉”打水印?
Alex Cui 指出,目前各大前沿 Labs 采用的高效且低成本的水印方案,本质上都遵循 KGW 算法(Kirchenbauer 等人提出),其运作分为生成与检测两步:
水印生成:系统在生成第 $n+1$ 个 Token(词/标记)时,会结合前面已经生成的 $n$ 个 Token 以及一个“私钥”(Secret Key),通过哈希算法将整个词库随机划分为“红集(Red Set)”和“绿集(Green Set)”(例如各占 50%)。随后,算法会人为提高绿集中词汇的出现概率,优先从中采样。
水印检测:检测器无需对照原始文本,只需利用相同的私钥,根据前文重新推演每个位置对应的红绿词集。如果一段文本中选择“绿集词汇”的比例显著高于 50% 的随机概率,检测器即可在统计学意义上判定该文本带有 AI 水印。
二、 辟谣与澄清:关于水印的三个核心疑问
改写能否破坏水印?答案是肯定的,普通的改写很容易打乱哈希结构。但目前可以通过引入“自适应水印(Adaptive Watermark)”或 SIR 等统计模型,用概率分布替代确定性函数,使水印在一定程度的改写后依然保持可检测性。
打水印会降低 AI 文本质量吗?会有轻微牺牲,但人类几乎感知不到。 语言表达的方式呈指数级丰富,微调词汇概率不会破坏整体语义。不过,在遇到像“2+2=4”这种答案高度确定、或代码等严谨文本时,水印机制往往会失效。
红绿词集会被黑客推算破解吗?难度极高,但并非不可能。 想要反向推导出红绿词集的分布,需要收集海量的水印样本。然而,如果 Labs 将检测器直接向公众开放,就会面临“水印窃取(Watermark Stealing)”的被逆向破解风险。
三、 顶级 AI 实验室面临的四大工程挑战
逐字流式传输限制: 用户需要实时看到生成的文字,因此水印必须逐个 Token 实时嵌入。而像 SemStamp、PostMark 等需要在生成整句或整段后重新编辑文本的稳健水印方案,目前根本无法在生产环境中部署。
密钥泄露风险: 私钥一旦泄露,整个水印体系即告崩溃。因此必须设计复杂的多密钥轮换机制以降低损失。
代码与特例文本的兼容: 代码中的变量名或固定语法不能随意改动,否则程序会直接报错。水印系统必须学会“聪明地”只在容忍同义替换的区域(如变量命名)嵌入水印。
误判与漏报的引导成本: 检测器必然存在误报(False Positive)与漏报(False Negative),如何教育公众合理看待检测结果,是 Labs 耗费巨资解决的难题。
四、 未来 6 个月趋势预测:合规走过场,检测器才是主战场
对于 AI 水印的未来发展,Alex Cui 提出了 5 大预测:
开源检测器等于“自自杀”: 若 Anthropic 向公众开放检测工具,攻击者会迅速建立对抗测试,找到 100% 绕过水印的方法;相反,若像 Google 一样仅向政府和特定机构开放,安全性会更高,尽管目前学术界已出现无需数据即可零样本(Zero-shot)破解水印的论文。
高强度改写依然是“杀手锏”: 测试表明,只要结合词汇替换与句法重构进行深度改写,或直接融入人类作者的表达,现有的 AI 水印(包括 Google DeepMind 的 SynthID)都会被轻易绕过。
“足够好”即可,合规大于实效: 顶级 Labs 并不期待普通用户不去攻击水印,欧盟等监管机构的要求也仅停留在“有防范措施”即可。这更像是一场应对监管的“合规过场”,而非真要穷尽手段暴露 AI 的使用。
终端用户并不关心: 普通用户一是察觉不到水印的存在,二是迫于监管政策(如欧盟 AI 法案)只能被动接受。
真正的重点在 AI 检测器: 相较于容易被破坏的水印本身,开发更精准、具备对抗训练能力的 AI 检测器才是当务之急。
Claude's watermark probably doesn't work how you think. As the CTO of GPTZero, I'll explain how Anthropic, Google and OpenAI are building text watermarking in this brief explainer and whether it can be defeated.
Almost all forms of watermarking that are fast and cheap enough for a frontier lab have the same formula, following the KGW method:
In generation:
1. Let's say you've generated n tokens so far. Take those n tokens + a secret key to generate a random hash
2. Use that hash to randomly reweight the probabilities for the n+1 token, and then sample from that new distribution. In the simple case, you could split 50% of all English words into a green or red set based on your hash, and boost the probability of words in the green set.
For watermark detection:
1. For each token, see if it was in the green or red set.
2. To do this, recreate the hash based on the secret key and the text preceding the current token. Then, recreate the green and red set of words.
3. Once you've checked all the words in the text, if the next token is selected disproportionally from the green set more than 50% of the time, you claim the text has the watermark.
I can tell you want to ask the following:
1) Isn't it easy to mess up the hash if you paraphrase the text? The answer is mostly yes, however, you can use a statistical model to get your hash instead of a deterministic function (SIR, Adaptive Watermark). Since the entire watermark is probabilistic, this is fine.
2) Doesn't this make the text much worse? The answer is yes, it does - Yes, it does – but for most people, it's imperceptible (Google claims in human feedback study with 20,000 texts), since there are exponentially many ways to write the same paragraph. DiPmark does something more sophisticated to avoid shifting the text distribution on average. Of course, watermarks fail on short text or highly predictable texts like "2+2=4".
3) Shouldn't it be easy to figure out the green and red sets? The answer is no. You would need an exponentially large number of samples from the watermarker to reconstruct those sets exactly, but it's a risk if the detector is open to the wild (Watermark Stealing)
Still, there are couple challenges that a frontier lab needs to overcome:
1. Their watermark needs to work token-by-token because they are streaming their text to users. Many watermark methods plan sentences or paragraphs at a time, or change the text after its entirely written, in order to make their watermark robust to paraphrasers, and a frontier lab cannot afford to do this yet (SemStamp, PostMark)
2. If the secret key leaks, the watermark is busted. To avoid a large blast damage from this, you need to have a couple secret keys in rotation.
3. There are some texts, like code, that cannot be arbitrarily changed, otherwise the code will break. In those cases, the watermark needs to selectively change words in parts of the text that can tolerate synonyms (i.e. like variable naming) - see SWEET, EWD, Invisible Entropy.
4. They will need to educate their users on how to deal with false positives and false negatives of a detector, which is a big challenge (one we put a lot of effort into)
So, how do I see this playing out in the next 6 months?
1. If Anthropic releases the watermark detector publically, I think they defeat their own watermark. People find reliable watermark removal strategies by testing against Anthropic (AI detectors like GPTZero have an advantage here because they can train against these adversaries once they become popular).
2. If they keep the detector private to the government, like Google has done, it's "safer". However, there are some papers showing trained approaches that work robustly to zero-shot break watermarks without any data, simply because they try to write the text just like a human (Zhang et al. 2024, Watermarks in the Sand). Also, making your detector makes it battle-tested and stronger long-term (my experience).
3. In my testing, the watermarks don't survive intense paraphrasing (especially if you combine word choice and syntax attacks), or human text substitution (rewrite your AI text by plagiarizing human authors). The free paraphrasers I've tried have quickly bypassed Google Deepmind's SynthId for what it's worth.
4. All-in-all, frontier labs are likely okay with this because they expect most users to not attack the watermark, and also because they + European regulators likely don't care past a certain point - its good enough.
5. Overall, I think users of frontier LLMs will not really care about this, because 1) they don't realize watermarks are there, 2) EU will force everyone to conform, 3) this seems more like regulatory hoop-jumping than an earnest effort from frontier labs to expose LLM use
Lastly, people's first concern shouldn't be watermarking, it should be AI detectors!
If you're posting, "its not X, its Y!!", I don't think the watermark is going to make a difference :)
顯示更多