Teaching reinforcement-learning agents to predict across sensors

A reinforcement-learning agent learns by acting in an environment and receiving feedback. A robot’s observations might include camera images alongside depth measurements or position coordinates. Combining them could help the agent decide what to do, but learning useful representations from limited interaction data is difficult.

A paper by Sijie Xu and Francisco Cruz proposes a change to how such agents learn. Their method, Improved Predictive Representation (IPR), adds two training objectives to a standard control algorithm. The authors test it in simulated tasks, comparing it with a closely related method called Multi-Modal Joint Predictive Representation (MJPR).

Predicting from the same starting point

A representation is a compact numerical summary of an observation. MJPR learns by predicting future representations and aligning information across sensors. Its predictions are recursive: a prediction at one time step feeds into the next. If an early prediction is off, that error may carry forward. The authors also argue that MJPR’s pairwise alignment emphasizes information shared between modalities.

IPR makes two changes. First, instead of building each future prediction from the previous one, it starts from the same observed moment and uses the actions taken since then. It predicts a future representation for each action-prefix length—for example, after one action, two actions or several actions—using the original observation as its anchor. These predictions do not depend on one another, so they can be computed in parallel. They are representations, not rendered predictions of what a future image or scene will look like.

Second, IPR aligns sensor information at the observed anchor, rather than aligning predictions during the future rollout. Its contrastive learning objective compares complete, multi-sensor views with views that retain one modality. In plain language, it encourages the agent’s representation to make sense both when all inputs are available and when it considers one sensor’s contribution.

The researchers added these objectives to Soft Actor-Critic, a reinforcement-learning algorithm, and tested the system in five DeepMind Control Suite tasks: catching a ball, running, spinning a finger, swinging up a cartpole and walking. They considered two sensor combinations: visual images with depth images, and visual images with position coordinates. They also tested a simulated robot navigating among obstacles and targets in Webots, with and without LiDAR, a sensor that measures distance.

Higher averages, but not everywhere

Across the five control-suite tasks, the IPR version that retained at least half of each image crop (IPR-H) had an average return 10.0% above the MJPR baseline with visual and depth images, and 3.4% above it with visual images and position coordinates. Return is the reward accumulated by the agent; a higher score indicates better performance in these particular tasks. The other crop setting, which could retain as little as 10% of an image, also had a higher average than the baseline in both sensor combinations.

Those averages hide differences among tasks. The crop settings did not perform uniformly, and the more aggressive cropping sometimes did better on individual tasks. In an ablation—an experiment that separates parts of a method—the authors found that the effect of action-prefix prediction depended on which alignment objective was used. They describe the results as suggestive of an interaction, not conclusive evidence of one.

In Webots, the ranking changed with the sensors available. Without LiDAR, both IPR versions had higher mean final returns than the baseline. With LiDAR, the baseline had the highest mean. The authors suggest that direct distance measurements may leave less room for IPR’s extra learning objectives to help, but they explicitly treat this as a hypothesis, not an established explanation.

What the results can—and cannot—show

The main comparisons use averages across five training seeds. Some tasks had substantial variation across seeds, and the authors performed no formal hypothesis tests. The reported differences therefore describe observed results in these experiments; they do not establish statistical significance, a causal explanation, or general superiority. The authors call for more runs and tests in additional environments.

The evidence is limited to simulated control tasks, including a constructed Webots navigation setup. Whether the method helps on physical robots or in other settings remains unknown. The source is an arXiv manuscript that identifies itself as a paper in the Proceedings of the Australasian Conference on Robotics and Automation (ACRA) 2026; this article does not independently verify its publication or peer-review status. Its claims should be read as research findings that may be revised, not as settled conclusions.

For now, IPR offers a design idea: predict future sensor representations from a shared starting observation, and align modalities at that starting point. The simulations suggest this combination can help in some settings, while varying results across sensors and tasks underline that additional learning objectives do not guarantee better performance.