X-World 1.6: Turning Recorded Streets into Controllable Driving Simulations

A city that responds to driving actions

X-World 1.6 generates synchronized video from seven vehicle cameras. In its closed-loop workflow, a driving policy receives those observations and produces an action or planned trajectory. The simulator advances the vehicle state, then generates new observations using the updated controls and previous video. Those observations return to the policy, continuing the interaction.

The authors build on X-World 1.0 rather than introducing this interaction loop for the first time. Their main additions concern more precise scene controls, stronger consistency during extended generation and an evaluation framework that separates visual defects from geometric consistency and downstream behavior.

Keeping the location while changing the scenario

The system adds controls for traffic-light positions and colors, plus road-surface markings such as directional arrows and stop lines. These complement existing inputs for vehicle motion, surrounding traffic, road geometry, weather and illumination.

A broader capability, called absolute static scene control, uses reference evidence to constrain buildings, roadside structures and other persistent surroundings. References can come from point clouds, 3D Gaussian scene representations or calibrated images. Geometry-aware encodings help associate reference content with the requested camera viewpoints.

These routes use separately trained models, not one model that accepts every reference type. Their intended benefit is to let developers change supported traffic conditions or driving paths while retaining a particular location. Preservation is limited by reference coverage: the model fills unobserved areas using learned patterns rather than recorded evidence.

The model family also supports generating multi-camera future video from a single image. A related dashcam-to-surround capability completes additional views around an existing video, but follows its recorded evolution rather than offering independently editable future dynamics.

Training for consequences that appear later

Generated video becomes the context for subsequent generation, allowing early mistakes to influence later observations. The authors address this with Connected Self Forcing, a training method that carries feedback from later predictions back through the earlier context that helped produce them.

Its replay procedure avoids retaining the complete computation graph for an extended rollout. The team also lengthens both the generator’s training rollouts and the teacher model’s supervision window. These changes target sustained coherence without changing the causal inference procedure.

Additional reward-guided training combines visual-preference scores with a measure of agreement between prescribed and inferred vehicle motion. The authors report that optimizing the motion reward alone can lead to reward hacking, motivating the combined objective. This updates the video-generating world model; training a driving policy inside that simulator is a separate process.

What the reported results establish

The automated benchmark measures object appearance persistence, temporal predictability through video compression, and agreement with a shared multi-view reconstruction. Connected Self Forcing improves several tracking and compression measures over ordinary Self Forcing, but reconstruction results are mixed.

With long-horizon tuning, the connected variant achieves the best average rank across nine metrics among the five compared models. That does not mean it wins every measure: X-World 1.0 retains better reconstruction PSNR and mean absolute error. Internal consistency also cannot establish overall visual quality or compliance with requested controls.

For downstream evaluation, the authors report a 20.9% relative reduction in mean final displacement error against trajectory predictions obtained from corresponding real videos. In paired reviews of selected failure cases, version 1.6 reproduces the target behavior more faithfully in 19%, comparably in 77.4% and less faithfully in 3.6%. The combined 96.4% improved-or-comparable figure is not an absolute success rate.

Serving improvements raise aggregate throughput by 2.05 times on a four-accelerator node running 12 sessions. Mean per-session step latency remains 8.94 seconds, distinguishing throughput gains from low-latency interaction.

Remaining boundaries

The authors say multi-view rollouts can exceed 30 seconds, but prolonged occlusion and long-range movement can still prevent previously seen content from returning faithfully. Inconsistent controls can also cause discontinuities.

Their trained visual evaluator agrees with human preferences in 20 of 28 non-tied scene pairs, with uncertainty reflecting the small evaluation set. Reported policy-training improvements from internal road testers are qualitative; quantitative road-test results are not provided. The evidence therefore supports specific simulator improvements on evaluated cases, not a demonstrated general improvement in real-world driving safety.