RACE: Calibrated Cumulative Scoring for Rollout-Free Robot Policy Checkpoint Selection

Why imitation error can mislead

Training produces checkpoints: saved versions of a policy at different stages of learning. Testing every version on a robot takes repeated trials and human effort. An alternative is to compare its predicted actions with recorded demonstrations, without actually letting it control the robot.

The preprint argues that averaging these prediction errors misses two important distinctions. A policy may correct some deviations, while others prevent completion. Meanwhile, a large early mistake could take the robot somewhere entirely different from the recorded path. Predictions scored against later observations on that path may then tell us little about what the robot would actually do.

The authors illustrate this with hypothetical policies: one closely imitates most actions but makes a consequential mistake; another deviates more often yet completes the task. Averaging errors can favor the wrong candidate. This example motivates the method, rather than constituting an experimental result.

Calibrating deviations, then carrying them forward

RACE uses a fixed collection of successful reference trajectories, either held-out demonstrations or previously collected successful policy runs. Each checkpoint predicts actions at the recorded observations. Scoring requires no new robot or simulated executions by the candidates.

First, the method finds similar reference observations using frozen visual features and information about the robot's own state. Differences among the actions recorded near those observations define local thresholds. These thresholds serve as a proxy for acceptable variation, becoming tighter where nearby successful actions are more consistent.

The method handles action groups separately, such as movement and rotation, rather than mixing different physical units into one distance. A deviation within its group's threshold receives the maximum group score; larger deviations reduce it. Closer imitation inside the accepted range earns no extra credit.

RACE combines group scores into a score for each step, then multiplies successive step scores together. A low score therefore reduces the contribution of later reference predictions. Earlier reductions affect more of the trajectory, while the preceding portion retains its contribution. Averaging these cumulative values over time and across trajectories produces the checkpoint score. Higher-scoring candidates rank first.

What the experiments show

In simulation, the authors evaluated eight DexMimicGen tasks with ten training seeds per task and twelve checkpoints per run. Each checkpoint's measured success rate came from 200 deployment rollouts. Offline scoring used ten successful reference trajectories per source, drawn from demonstrations, policy rollouts or noise-perturbed rollouts.

Their aggregate results favor RACE over validation loss, mean squared action error and Off-Manifold Norm, a baseline measuring action deviation outside a local expert-action subspace. With held-out demonstrations, RACE's top-three shortlist contained the empirically best checkpoint in 75% of runs, versus 59% for validation loss. That is evidence for shortlisting, not perfect identification: RACE's top-ranked candidate was the empirically best in 44% of runs.

Real-robot experiments used a Franka Research 3 arm with diffusion policies and a fine-tuned vision-language-action model. Across the two policy families, the paper reports 1,800 deployment evaluations. The authors report that RACE rankings aligned more closely with deployment success than the tested baselines overall.

A separate Box Cleanup experiment compared noise spread throughout execution with larger noise at a critical lid-alignment stage. Across 50 paired rollouts from identical starting states, success fell by four percentage points for uniform noise and forty for bottleneck noise. Despite similar changes in average prediction error, RACE's score fell much more for the bottleneck disturbance.

A shortlist, not a guarantee

The paper explicitly acknowledges that nearby successful action variation cannot establish whether a candidate will recover from a mistake. Nor can RACE predict behavior at unseen states caused by the policy's own actions. Its output is a relative ranking, not a calibrated probability of success.

Results also vary by task. The appendix reports lower selection regret for validation loss than RACE on held-out Drawer Cleanup demonstrations. Shared scoring settings weakened aggregate ranking quality compared with task-specific tuning. The real-world ranking statistics are point estimates, and each checkpoint's reported success rate rests on thirty trials per task.

The supported use is therefore narrower than replacing hardware evaluation: reuse successful recordings to prioritize candidates, then validate the selected policies in deployment.