Learning to Persuade When Users Can Game the Learning Process

A model of competing interests

Li, affiliated with Boston University, and Lin, affiliated with Microsoft Research New England and Harvard University, investigate a theoretical problem in information design: choosing what information to disclose to influence someone else's decisions. Their paper is a preprint, not evidence from a deployed platform or a study of human participants.

The researchers motivate the problem with a hypothetical language-model API provider. The provider has an internal assessment of whether a request can be handled successfully and chooses what to communicate. A business user then decides whether to continue consuming paid tokens. The provider benefits from continued use, while the user wants useful answers rather than unwanted output. This is an illustrative incentive conflict, not an empirical claim about any particular service.

In the model, a designer repeatedly sends signals about a state of the world to one long-lived receiver. The designer knows the true state distribution and both parties' payoff functions, but not the receiver's subjective prior belief. That belief can differ from the true distribution and remains fixed throughout the interaction.

Why simply watching is insufficient

Counting how often states occur cannot reveal a subjective belief that differs from reality. Instead, the researchers design signaling experiments that make the receiver's actions informative about that belief.

The basic intuition resembles binary search. With two states and two actions, a carefully chosen signal creates a threshold at which the receiver is indifferent between actions. Observing the chosen action helps locate the receiver's belief relative to that threshold. Repeated experiments narrow the possibilities. The paper extends this approach to finite sets of states and actions by estimating probability ratios.

A strategic receiver, however, may deliberately choose an action that sacrifices an immediate reward to obtain more favorable information later. Li and Lin address this by delaying when an observed action can change future signaling.

For each experiment, the algorithm records the action taken when a designated signal first appears. It then keeps the signaling rule unchanged for additional periods and ignores actions during that waiting interval. Because the receiver values distant rewards less, the delay reduces the attraction of manipulation. It need not eliminate every deceptive action: deviations with very small immediate costs can remain worthwhile, but the resulting estimation error is bounded.

Two standards for success

The researchers measure performance using regret: the gap between a benchmark payoff and the designer's cumulative expected payoff over T periods.

The static benchmark repeats the best single-period outcome available when the receiver's belief is known. The stronger dynamic benchmark allows the informed designer to optimize across the entire interaction, using future information to influence current decisions.

For each benchmark, the authors report an algorithm with cumulative regret bounded by O(log² T), for fixed problem parameters. Dividing by T gives an average shortfall of O(log² T/T). This is a payoff guarantee, not a claim that computation takes logarithmic time or that every individual interaction improves.

Learning alone is insufficient. Small belief-estimation errors can change which action a receiver prefers, sharply reducing the designer's payoff. The algorithms therefore modify the strategies computed from estimated beliefs to remain effective for nearby beliefs.

For the dynamic benchmark, this modification must also preserve the value of future cooperation. The researchers introduce a procedure that strengthens incentives while weakly increasing the receiver's expected payoff from following recommendations, compared with the unmodified signaling rule under the same belief.

Future information as leverage

The authors characterize the dynamic optimum using strategies that recommend actions while the receiver complies, then permanently withhold information after a deviation. These strategies require commitment by the designer.

In a numerical illustration with three states, three actions, 20 periods and a receiver discount factor of 0.7, the paper reports an average designer payoff of approximately 0.7279, versus 0.7 for the static benchmark. This illustrates the model's potential gains from alternating designer-favorable recommendations with later compensation through valuable information.

Where the guarantees stop

The results assume independently drawn states, finite choices, full-support beliefs, a known positive lower bound on receiver state probabilities and specific regularity conditions on preferences, including a unique best action in each state. They also assume a fully patient designer and a receiver who optimizes discounted rewards.

That patience difference is essential. The authors prove that, for a fully patient receiver, some instances force regret to grow linearly with T regardless of the learning algorithm. Learning also becomes more costly as receiver patience increases.

Changing beliefs, imperfect commitment and privacy-preserving learning remain outside the established guarantees. The paper thus offers a mathematical account of strategic belief learning, not a demonstration of practical effectiveness or user welfare improvements.