A Privacy-Preserving Watermark for Measuring AI Text in Aggregate

Language-model watermarks usually aim to make individual generated texts recognizable. Elise L. Han and Tatsunori Hashimoto, of Stanford University, ask whether that objective is necessary when the real question concerns aggregate usage. Their paper, Reconciling Privacy with Provenance for Language Model Watermarks, introduces a watermark designed to estimate prevalence without making each document a strong detection signal. Submitted to arXiv on November 4, 2026, the supplied version is a preprint; its source materials do not report peer-reviewed acceptance.

A bounded signal instead of a fingerprint

The authors draw inspiration from randomized response, a survey technique that adds controlled randomness to sensitive answers. An individual response remains ambiguous, but researchers can recover population frequencies because they know the randomization process.

Their Red-Green Sequence watermark, or RGS, applies a related principle to generated text. A secret keyed hash assigns each sequence a reproducible red or green classification. The generator favors green sequences, while an analyst with the key estimates watermark prevalence from the observed green fraction.

The privacy guarantee bounds how much watermarking can change the probability of any output relative to the same underlying model without watermarking, even for a fixed key. The parameter epsilon controls that bound: smaller values provide stronger privacy but leave less signal for estimation. This protects watermark status relative to a reference generation process; it is not a general guarantee that a document conceals its contents or author.

How the watermark works

Conventional red-green watermarks bias token choices repeatedly during generation. RGS instead encodes one red-green bit per sequence, avoiding the accumulation of token-level evidence that makes long texts increasingly identifiable.

The paper presents two generation methods. A parallel sampler draws multiple candidates and reweights their probabilities within the privacy constraints. A cheaper rejection sampler draws candidates sequentially, accepts green candidates immediately, and accepts red ones with probability exp(−epsilon). If it reaches its candidate limit, it returns the final candidate.

Experiments primarily use rejection sampling with a maximum of 20 candidates and a 128-bit keyed BLAKE2b hash. Depending on the task, the hashed sequence is a generated prefix rather than the entire response. At epsilon values of 0.5 and 1, the authors report fewer than 1.5 candidates examined on average across the tested models; at 2 and 3, the average is around two. These are candidate counts, not direct measurements of production latency.

To estimate prevalence, the analyst subtracts the expected unwatermarked green fraction from the observed fraction, then divides by the nominal separation between watermarked and unwatermarked green probabilities. Estimates are clipped to the interval from zero to one.

What the experiments found

The evaluation uses Qwen3 models with 1.7, 4 and 8 billion parameters, alongside Llama-3.1-8B-Instruct. Experiments average over 20 randomly sampled keys and 50 mixture repetitions per key.

For C4 web-text prompts, the main setup uses mixtures of 2,000 texts, with watermark generation and detection based on the first 50 generated tokens. The authors report mean squared error of 0.0019 and mean absolute bias of 0.0038 at epsilon 1. Mean absolute bias describes systematic deviation in the estimates, not the error readers should expect from every individual mixture.

Harder experiments use OUTFOX essay prompts and LiveCodeBench coding problems. Each mixture contains only 100 texts responding to a single prompt, reducing both sample size and prompt diversity. At epsilon 2, the reported essay results include mean squared error of 0.0113 and mean absolute bias of 0.004. Coding produces higher bias and variance than essay writing.

These mixtures compare watermarked outputs with unwatermarked outputs from the same base model. They therefore test watermark prevalence estimation, not unrestricted identification of all AI writing against human-authored text.

Why diversity and robustness matter

Estimation depends on how many different sequences the underlying distributions can produce. When outputs are repetitive, a fixed key can yield green frequencies that differ from the nominal baseline. Watermarked estimation bias can persist even as sample size grows. The authors show that increasing the nominal green fraction can reduce normalized bias, at the cost of greater sampling variance. Their analysis also relies on idealized pseudorandom coloring and conditional independence assumptions.

Two deployment limitations remain explicit. Minor edits can disrupt sequence-level hashing, so RGS does not yet provide meaningful edit robustness. Its privacy guarantee also does not extend to unlimited queries by one user.

For AI monitoring, the contribution is a change of objective: measuring how common a watermark is, rather than maximizing evidence against individual documents. The experiments support that direction under controlled conditions, while leaving robust, longitudinal deployment unresolved.