Does an adaptive policy create value—or just choose a good average mix?

A pricing rule, investment strategy or inventory policy may look effective because it responds intelligently to information. Alternatively, its average choices may simply be good. Separating those explanations requires a benchmark that removes responsiveness without changing the average mix of actions.

In the preprint Regret-based Characterization, Information Limits, and the Sample Complexity of Auditing Adaptive Policies, Irene Aldridge studies that comparison mathematically. The November 2026 manuscript, arXiv:2611.05227v1, presents theoretical results and small numerical illustrations, not a real-world validation of an auditing system.

A benchmark that ignores the state

The main setting is a finite-horizon Markov decision process with finite state and action sets. At each step, a policy chooses an action based on the current state; that action can affect both immediate costs and future states.

Aldridge constructs an “open-loop marginal” policy. At each step, it draws from the adaptive policy’s average action distribution, but ignores the state. It can change its action distribution over time: open-loop does not mean repeating one action forever.

The comparison uses expected total cost, so lower is better. A negative difference between the adaptive policy’s cost and its marginal benchmark’s cost means conditioning on the state adds value. A positive difference means it destroys value relative to that benchmark.

This is not necessarily a comparison with the best possible open-loop policy. The paper separately distinguishes a policy’s adaptive value from the overall advantage of the best adaptive policy over the best open-loop alternative.

Why immediate costs can mislead

The central identity expresses the cost difference as a sum of covariances. These measure how the policy’s state-dependent action probabilities align with action values evaluated using its open-loop benchmark.

The action values include immediate costs and expected downstream costs when subsequent decisions follow the benchmark. That detail matters because actions can change which states the system reaches. The benchmark need not experience the same future state distribution as the adaptive policy.

The paper gives a constructed example in which an immediate-cost audit reports value destruction even though the policy reduces total expected cost. An action can be expensive now but worthwhile because of what happens later. When actions do not affect transitions, the simpler immediate-cost covariance is sufficient; in the general setting, it is not.

Information limits the possible benefit

The manuscript also bounds the magnitude of adaptive value using mutual information between states and actions. Informally, this measures how much observing an action reveals about the state in which it was chosen.

The bound combines that dependence with the largest within-state difference between action values at each step. Information alone therefore does not determine value: the consequences of choosing different actions also matter.

Aldridge shows that the bound’s numerical constant cannot be improved in general. This is a worst-case statement, not a promise of a tight estimate for every application. Applied to an optimal policy, the result also bounds the problem’s overall adaptivity gap. Strong state–action dependence, however, does not itself establish useful adaptation.

An audit needs the missing alternatives

The statistical results make the counterfactual problem explicit. In the contextual-bandit setting, the auditor observes independent rounds containing the context, action probabilities, chosen action and its cost—not the costs of unchosen actions. Costs lie between zero and one, and the policy’s marginal action distribution is treated as known.

Under suitable overlap, an importance-weighted estimator can recover the adaptive cost difference. Its guarantee depends on an average chi-squared divergence measuring the mismatch between marginal and context-specific action probabilities. Rarely choosing an action that the benchmark uses makes evaluation harder.

The paper provides an upper bound and matching worst-case lower bounds, up to constants, for divergence values at least one-third. The required observations scale with that divergence, inverse squared estimation tolerance and a logarithmic confidence factor.

A sharper obstacle arises when an action used by the benchmark is never chosen in some contexts occurring with positive probability. Different counterfactual costs can then produce identical logs but different adaptive values. In this contextual setting, a deterministic policy whose action is not almost surely constant cannot be audited from its own logs alone.

Longer decisions, harder evaluation

For episodic decision processes, Aldridge proposes weighting entire trajectories, assuming logged action probabilities, known marginal distributions and overlap. The resulting bound can grow exponentially with the horizon; the paper does not establish that this growth is unavoidable for its particular benchmark.

The practical distinction is between defining adaptive value and being able to estimate it. A precise mathematical comparison does not supply observations of actions never taken. An audit must account for downstream consequences—and establish that its logs support the counterfactual comparison.