A 360-degree image can map a room—but not every surface

A panorama offers a tempting shortcut to 3D: instead of stitching together views taken while moving a camera, start with one image that already looks in every direction. But converting that image into a map is difficult. The image gives no direct distance measurements, and panoramic projections can distort the scene. A model trained on conventional photographs may not know how to handle the distortions or the boundaries where a panorama’s parts meet.

The authors of PaGeR—short for Panoramic Geometry Reconstruction—ask whether an existing 3D vision foundation model can be adapted to this setting without building a panorama system from scratch. Their approach starts from Depth Anything 3, a model designed for perspective views, and reformats a panorama as six perspective-like images arranged as a cubemap. Think of the views as the six faces of a box surrounding the camera. This avoids some of the severe stretching found in the common flat, equirectangular panorama format, while letting the model use knowledge learned from perspective images.

One image, several geometric estimates

From the six views, PaGeR predicts four outputs in a single forward pass: scale-invariant depth, which describes relative distances without fixing the scene’s overall scale; metric depth, which estimates distances in real units; surface normals, which indicate the direction a surface faces; and a sky mask, which marks regions where ordinary finite-depth estimates do not apply.

The system uses one shared transformer backbone with separate task-specific prediction heads. To help neighboring cubemap faces agree, the authors modify the decoder so it can draw information across face boundaries. They also train with both panoramic images and ordinary perspective images. The authors’ rationale is that panoramic training teaches the model about 360-degree geometry, while perspective images help preserve the model’s existing visual and geometric knowledge.

The training mix includes synthetic panoramas and real perspective images. For evaluation, the paper compares PaGeR with existing methods on indoor benchmarks, a synthetic outdoor benchmark and ZüriPano, a new real-world outdoor dataset collected in Zürich using LiDAR scans. The authors also report a separate surface-normal evaluation on Structured3D. Their tests include comparisons against other methods and ablations—experiments that remove or change parts of PaGeR to assess their contribution.

Strong benchmark results, with important qualifications

The paper reports that PaGeR outperforms the compared methods on several depth benchmarks. On ZüriPano, for example, its reported scale-invariant depth error is lower than that of the next-best method in the paper’s comparison. It also reports strong metric-depth results on that dataset and the best surface-normal results in its Structured3D comparison. Ablations support the design choices: removing camera conditioning, sky segmentation, cross-face padding or perspective training worsens performance on the tested tasks.

These are results on specified datasets and evaluation procedures, not proof that PaGeR can reliably map any place from any panorama. The authors identify several limits. Mirrors and other reflective or transparent surfaces can lead the system to reconstruct what is reflected or visible through the surface rather than the surface itself. Accuracy can fall in large textureless areas and under unfamiliar conditions, including complex outdoor lighting. Despite the boundary-handling design, some geometrical mismatches between cubemap faces remain in complex scenes.

Outdoor evidence is particularly limited. ZüriPano contains 100 panoramas from 11 urban locations in Zürich, and the paper’s outdoor evaluation also uses synthetic data. The authors say this does not establish how well the model generalizes to varied natural or non-urban landscapes. They also report a trade-off in the unified model: separately trained specialist models do somewhat better on some depth and normal measures, while the shared system offers several outputs together. In particular, the unified metric-depth head has a more pronounced performance gap against a dedicated metric-depth model.

PaGeR is presented in a paper listed as appearing at NeurIPS 2026. As with any research report, its benchmark claims should be read in the context of the chosen datasets, comparisons and methods; they do not by themselves establish performance in real-world applications. The work offers a practical route for adapting perspective-trained geometry models to panoramas, while leaving open how robust that route will be across scenes and surfaces beyond those tested.