Teaching AI to judge the safety of a whole maneuver

A driver approaching a green light must consider more than whether accelerating is safe at this instant. The same decision could leave too little room to stop when the light changes. Reinforcement-learning systems face a similar problem: checking each immediate action does not necessarily protect against consequences that unfold later.

A preprint by Gian Luca Ascari and colleagues proposes Probabilistic Option Shielding, or POS, to address that gap. Rather than judging only individual actions, the method learns safety estimates for behaviors lasting several steps. Tests in two simulated urban grid environments suggest this can make training substantially safer without sacrificing successful navigation.

From individual steps to whole behaviors

Reinforcement learning trains an agent through interaction: it tries actions and learns to pursue rewards. Safety mechanisms called shields influence those choices using rules about what the agent should avoid.

Traditional shields can assess longer-term consequences using a model of the environment’s dynamics. But obtaining a complete, accurate model can be difficult. An alternative, probabilistic logic shielding, uses encoded safety rules and information about the current situation to estimate whether an immediate action is safe. It then adjusts the agent’s action probabilities accordingly.

The limitation is its short horizon. Extending its reasoning far into the future would require representing future transitions, bringing back some of the modeling burden.

POS instead works with options: predefined multi-step behaviors with their own stopping conditions. In the experiments, an option means traveling in a cardinal direction until reaching the next intersection. A higher-level policy chooses among these behaviors rather than selecting every movement separately.

Learning a safety estimate

The method learns an option’s safety from the one-step signals already supplied by the logic shield. A safety estimator, called a critic, combines the current action’s safety probability with its estimate for the remainder of the behavior. Each observed transition helps update that estimate.

Conceptually, POS asks: how likely is every step of this behavior to satisfy the safety rules before the behavior ends? Its mathematical definition averages the product of the individual safety probabilities over possible executions. Interpreting that product as the probability that all steps are safe requires an assumption: once the state-and-action sequence is specified, the individual safety events are conditionally independent.

The learned estimates reweight the agent’s proposed options, favoring those judged safer. This is probabilistic steering, not an unconditional barrier against every unsafe choice. It evaluates safety through an option’s completion, not automatically across the agent’s entire future.

The researchers prove that their exact mathematical update has a unique, stable solution when options terminate within a uniformly bounded number of steps. That result does not establish that a neural-network approximation will always learn perfectly or prevent every violation.

Testing detours and simulated inertia

The team evaluated POS with PPO, a standard reinforcement-learning algorithm, in the LogiCity simulator. One map was a 43-by-43 grid containing an always-forbidden military zone and pedestrians present during school time. Safe navigation could require a long detour.

A smaller, 23-by-23 map introduced simulated inertia: at each step, there was a 30% chance that the agent would repeat its previous movement instead of executing its newly selected action. This made last-moment changes of direction unreliable.

The researchers compared POS with five baselines, including unshielded learning, one-step shielding, penalties for violations and a method that balances rewards against safety costs. Each main experiment ran for 500,000 environment steps across ten random seeds, reporting average results with standard errors. Performance measured both cumulative safety violations during training and discounted rewards, which favor reaching the goal sooner.

On the larger map, POS with a pretrained critic produced near-zero violations during policy training. Learning the critic alongside the policy reduced violations by 87% relative to unshielded option learning and 83% relative to the safety-cost baseline. Those were the only baselines that consistently solved this task; several others had few violations but failed to learn successful navigation.

On the inertia map, all methods reached approximately optimal performance. Compared with one-step shielding, pretrained POS reduced cumulative violations by 98%, while the version learning its critic online reduced them by 75%.

Promising, but narrowly demonstrated

These are results from two simplified simulations. The experiments assumed correct safety rules and noise-free sensors, and the behaviors were supplied in advance rather than learned. Pretraining also requires a separate data-collection phase; near-zero violations during subsequent policy training should not be read as proof of risk-free learning from scratch.

The work was posted to arXiv on November 6, 2026, as a preprint and identifies a NeurIPS 2026 workshop venue. It offers evidence for a useful approach to safer learning, not validation for deployment in real vehicles. The authors identify more realistic driving benchmarks and safely learning the options themselves as future directions.