Teaching robots without demanding an exact copy

When imitation becomes too exact

A robot learning to pick up an object faces a problem hidden inside its examples: there may be several acceptable ways to move. Demonstrators can approach from slightly different positions or pause for different lengths of time. A training rule that compares every predicted action with the demonstration at exactly the same instant can penalize differences that do not prevent success.

SURE, described in a November 2026 preprint by Jingye Zhang and colleagues at Tsinghua University, addresses both kinds of variation. Its name stands for Spatiotemporal Uncertainty Representation. The researchers report better manipulation results with fewer demonstrations or training steps in selected comparisons—not a robot that can learn arbitrary tasks from a handful of examples.

Giving neighboring actions a relationship

The system builds on OpenVLA-7B, a vision-language-action model that processes visual observations, robot state and language instructions to produce movement commands. SURE changes the component that turns the model’s internal representations into actions.

For translation and rotation, this decoder divides each normalized action range into 256 ordered bins. It predicts probabilities across those bins rather than directly producing one number. Transposed convolutions—neural-network operations that expand a representation—encourage neighboring bins to share information. Nearby possible movements are therefore represented as related alternatives rather than unrelated categories.

The decoder then converts each distribution into a continuous command using its probability-weighted average. It does not sample a different trajectory from that distribution on each execution. A separate small network predicts gripper commands. For RoboTwin’s two-arm joint controls, the researchers adapt the decoder to predict distributions over 12 joint commands instead.

Importantly, SURE does not abandon exact action supervision. Its main action loss still measures absolute differences from demonstrated commands. A weaker additional loss encourages distributions centered around those demonstrations. The change combines a structured output representation with existing supervision rather than simply declaring nearby actions correct.

Comparing movements at different speeds

SURE’s second component addresses timing. Soft dynamic time warping, or Soft-DTW, compares sequences while allowing their local progress to differ. Instead of insisting that the third predicted step match only the third demonstrated step, it permits a flexible alignment that preserves sequence order.

In the main formulation, the researchers compare cumulative translation and rotation actions, excluding the gripper. They combine the alignment objective with an endpoint penalty so that flexibility in timing does not remove pressure to reach the demonstrated chunk’s final cumulative position.

This alignment operates during training. It is not an extra planning procedure the robot must run whenever it moves.

What fewer examples achieved

Across four LIBERO simulation suites, SURE achieved 97.0% average success after 10,000 optimization steps. OpenVLA-OFT, using the same underlying OpenVLA backbone, reached 87.1% after 10,000 steps and 94.3% after 50,000. The latter comparison represents a 2.7-percentage-point advantage using one-fifth as many steps.

A separate data-reduction experiment covered only LIBERO-Spatial, with both methods trained for 10,000 steps. SURE reached 95.0% success using 25% of the demonstrations, compared with 94.4% for OpenVLA-OFT using all demonstrations. At 10% of the data, SURE reached 93.8%: close to, but not above, that full-data baseline.

Removing either uncertainty component reduced performance in a LIBERO-Spatial ablation. Success fell from 99.0% to 97.4% without Soft-DTW and to 96.8% without the probabilistic decoder; removing both yielded 94.4%.

Moving beyond one simulator

On five RoboTwin 2.0 tasks, SURE averaged 72.2% success, compared with 60.2% for the strongest listed baseline. These tests used clean, non-randomized environments and a separate trained checkpoint for each task, not the benchmark’s entire 50-task collection.

Physical tests used LeRobot SO-101 arms, 50 demonstrations per task and six tabletop tasks. Across 20 evaluation trials per task, SURE averaged 57.5% success versus OpenVLA-OFT’s 41.7%, with fewer training steps. It improved five tasks but performed worse at pouring water: 40% versus 55%.

Where the evidence stops

This is a preprint, not an established general result. The principal LIBERO comparisons and data-efficiency experiments use a single seed, while some other baseline results come from earlier publications with different evaluation protocols. Training-step counts also do not establish equivalent reductions in computing time, especially when batch sizes differ.

The reported fast decoding likewise concerns only the action head, excluding the large vision-language backbone; the deterministic OFT head remained faster. Finally, task completion is not a safety assessment. The results support further investigation of more tolerant imitation learning, not unsupervised deployment around people.