OccSplat imagines the road hidden behind the foreground

A system that reconstructs a road scene from a single moment of surround-view images faces a basic information problem: parts of the world are hidden. A slightly later viewpoint can reveal surfaces that were never visible in the input, producing holes or stretched textures in a synthesized image. This is called disocclusion.

Dung Do Tien and colleagues propose OccSplat to address that problem in driving-scene reconstruction. Their paper, Quo Vadis, Driving Scene Reconstruction? Occlusion-Aware Gaussian Densification, is available as arXiv preprint 2611.06273v1. The landing page states that it was accepted to the proceedings track of the Computer Vision for Developing Countries workshop at ACCV 2026.

From visible pixels to a denser scene

OccSplat builds on DrivingForward, a feed-forward reconstruction system. Rather than optimizing a separate model for each scene, a trained network predicts a scene representation in a forward pass. The representation uses 3D Gaussian splatting: many small, adjustable 3D primitives carry position, shape, opacity and appearance information and can be rendered into an image from a chosen viewpoint.

The backbone estimates depth from the current surround-view images and uses it to place initial Gaussians in 3D. But primitives tied to observed pixels cannot directly capture a surface that none of those pixels shows. OccSplat adds two steps intended to find and fill this missing support for the next viewpoint.

First, Gaussian Occlusion Mapping renders the initial scene against both black and white backgrounds. Where the Gaussians are sufficiently opaque, the background has little effect; where support is weak, the background shows through. A threshold produces a mask of candidate gaps. This is a rendering-based measure of insufficient coverage, not a guarantee that every marked pixel corresponds to a truly occluded object.

Second, Gaussian Occlusion Completion predicts additional primitives in the masked areas. A small image-refinement network first repairs a preliminary next-view rendering, providing an intermediate cue rather than the final image. Separate attention branches use that cue along with source-view depth and image features to infer depth and appearance. The system converts these predictions into new Gaussians, reuses the backbone's Gaussian attribute predictor, and merges the additions with the original scene before rendering. The design uses image-space refinement, but its final additions live in the shared 3D representation.

Training is staged: the backbone and image refiner are trained first, then frozen while the attention branches learn. The latter use target-view images and pseudo-depth from Depth Anything V2 as supervision. At inference, the method remains feed-forward and does not require per-scene optimization.

What the experiments show

The authors evaluate next-frame view synthesis on nuScenes, using six cameras, 700 training scenes and 150 validation scenes. Images are evaluated at 114 by 228 pixels. Against DrivingForward, the reported overall peak signal-to-noise ratio (PSNR, higher is better) rises from 21.76 to 23.50 dB; perceptual distance LPIPS (lower is better) falls from 0.194 to 0.182. Structural similarity, or SSIM, is 0.767 for both. These are image-comparison metrics, not direct measurements of whether hidden geometry is correct.

In a separate evaluation restricted to disoccluded regions with identical masks across methods, OccSplat scores 18.602 dB PSNR, 0.763 SSIM and 0.465 LPIPS. MVSplat has a slightly higher PSNR of 18.606 dB, but lower SSIM of 0.509 and higher LPIPS of 0.620. DrivingForward scores 11.498 dB, 0.759 SSIM and 1.178 LPIPS on that regional comparison. Thus the regional result is not a clean win on every metric.

An ablation also makes the localization step consequential: adding completion across the whole image without the refiner or occlusion mapping lowers PSNR from the 21.76 dB baseline to 19.92 dB. The complete configuration reaches 23.50 dB. This supports the case for combining the components in the reported setup rather than assuming that adding Gaussians everywhere helps.

The limits of filling what cannot be seen

A plausible prediction is not an observation. The input cannot establish the true appearance or geometry behind an occluder, and better rendered-image scores do not prove that the generated scene is physically accurate or safe for driving decisions. The authors note that their image-space refiner cannot explicitly enforce 3D consistency across viewpoints and may oversmooth textures under severe disocclusion or large vehicle motion. Their proposed future work includes temporal information and explicit 3D priors.

The paper reports results on one benchmark at low resolution, so performance in other settings or at higher resolution remains unestablished by these experiments. There is also a speed trade-off: the reported inference time rises from 574.7 milliseconds for DrivingForward to 992.9 milliseconds for OccSplat, or about 1.01 frames per second, with reported VRAM rising from 2.96 to 3.10 GB. The work demonstrates a useful reconstruction strategy, not real-time driving readiness.