AI Safeguards Need to Be Judged by How They Fail Together

The problem with counting defenses

An AI agent can propose a tool call that is technically valid but unauthorized: changing a record when asked only to read it, for example. Safeguards try to intercept those actions, but they can also block legitimate requests. Designing a system therefore means balancing security misses, service interruptions and the expense of running its checks.

The preprint argues that individual safeguard scores are insufficient for this task. Checkers inspect the same action and context, and may share model characteristics or struggle with the same difficult cases. Their errors can overlap. Treating them as independent can substantially underestimate the chance that an unauthorized action passes every check.

The authors model joint safeguard decisions separately for authorized and unauthorized actions. They use copulas, statistical tools that connect individual outcome probabilities through a dependence structure. The resulting model estimates how often particular combinations of ALLOW and BLOCK decisions occur.

Two ways to combine checks

The framework considers two policy classes. In sequential layered screening, checks run in a chosen order and stop at the first BLOCK. An action proceeds only if every selected safeguard allows it. The selected safeguards determine error rates; their order determines how often later, potentially expensive checks run.

In selective verification, all chosen first-stage safeguards run before a routing decision. Each combination of their outputs triggers automatic allowance, automatic blocking or referral to another safeguard whose decision is final. This is more flexible than simply referring every flagged action for review: disagreement patterns can receive different treatment, and even unanimous allowance can trigger review.

The authors’ Dependence-Aware Policy Optimization Algorithms, or DAPOA, search these classes under specified error limits or operating budgets. Total expected cost combines invocation costs with assumed losses for security misses and false blocks. The optimization is exact within the admitted policy classes for the supplied probability distributions—not a guarantee that those distributions perfectly represent deployment.

The theoretical results also resist a simple claim that stronger dependence always makes safeguards worse. For layered screening, a specified ordering that increases simultaneous allowance raises misses and invocation costs while reducing false blocks. For selective verification, the direction of error changes depends on the routing rule. Under an additional assumption that the reviewer’s blocking rate within each authorization class does not vary across first-stage patterns, unconstrained cost minimization produces two evidence thresholds: allow low-risk patterns, review intermediate ones and block high-risk ones.

What the evaluation found

In eight numerical scenarios with fixed individual safeguard performance, the policy selected under independence exceeded the security-miss limit in six. Dependence-aware selection found a feasible policy in every scenario.

The empirical study used 2,524 benchmark-derived customer-service actions: 1,262 authorized actions paired with unauthorized variants. Four LLM safeguards assessed each action. Users were kept separate between calibration and held-out evaluation, which contained 598 actions of each authorization class.

Under the principal cost and risk assumptions, independence-based selection chose a policy that automatically allowed unanimous allowance from three local models. All 14 of its evaluation misses occurred in that shared-ALLOW pattern.

Dependence-aware selection instead chose Ministral followed by Sonnet whenever Ministral allowed an action, stopping immediately when Ministral blocked. This policy recorded zero misses in the held-out sample, compared with 14, but increased false blocks from nine to 30.

Operating cost rose from 0.3779 to 0.7300 normalized units per action. Once the assumed error losses were included, total cost fell from 1.6237 to 0.9809: the reported 39.6% reduction. This is a reduction in a weighted decision metric, not in actual spending or invocation cost.

A useful model, not a safety guarantee

Across 37 operating settings, dependence-aware selection produced lower evaluation total cost in 17, equal cost in 15 and higher cost in five. Both fitted approaches underestimated some observed error-event probabilities. Under the strictest false-block requirement examined, neither model’s selected policy satisfied that requirement on evaluation.

The authors identify further limits: the actions came from controlled benchmark environments rather than live deployments; invocation costs used pricing assumptions; error consequences were illustrative; and dependence was held fixed during optimization rather than adapting to changing attacks.

The practical conclusion is to evaluate which mistakes safeguards share and how the combination rule uses their decisions. More layers alone do not reveal the security, service or cost of the resulting system.