A VAE-aware encoding aims to preserve dark-region color in HDR reconstruction

Where color information can be lost

HDR represents a wider range of brightness than SDR. When converting SDR to HDR, a clipped white region could correspond to many possible highlight intensities; in dark areas, crushing, banding and quantization artifacts can obscure subtle differences. A generative model may infer plausible missing detail, but the reconstruction remains underdetermined.

The paper examines the representation step before generation. A variational autoencoder, or VAE, maps image or video signals into a compact latent representation. The authors report that, in their tests, common log-like HDR encodings can contract chromatic differences at low luminance after signals pass through a frozen VAE. They interpret this as a potential bottleneck for later reconstruction; the experiments do not establish that every generative model loses shadow color in this way.

Encoding exposure and color separately

The proposed encoding, EquivFlow, is fitted against a frozen VAE rather than designed independently of it. It maps scene-linear color into the VAE’s input interface, where codes are clipped in the implementation. The mathematical encoding has an analytic inverse, but that does not make the entire clipped pipeline globally invertible.

EquivFlow is designed so that exposure changes shift the encoded signal along a predictable direction, while a separate component retains chromatic information independently of overall brightness. Exact exposure equivariance holds above the encoding’s low-luminance toe, when both exposure endpoints are above it; below the toe, the brightness mapping remains finite near black, but the exact relationship no longer applies. The authors fit the encoding first, then train a low-rank adapter for reconstruction while keeping the VAE and diffusion backbone frozen.

The study evaluates the representation separately from the full reconstruction system. In encoding-level tests, signals pass through frozen VAEs and researchers measure how well chromaticity remains recoverable as exposure falls. For cross-VAE tests, EquivFlow is fitted using LTX and then tested on four named hosts: LTX, Stable Diffusion 2.1, Wan 2.1 and Wan 2.2. The reported portability concerns exposure structure and chroma separability under specified readouts—not identical latent coordinates or general compatibility with all VAEs.

For reconstruction, the authors compare their complete EquivFlow system with a matched LogC3 system using the same LTX-2.3 backbone and adapter-training budget. The comparison is not an isolated test of the encoding: EquivFlow conditions on an inverse-tone-mapped scene-linear proxy, while LogC3 uses raw SDR. The main comparison uses a fixed 102-clip subset from ARRI sample footage, HDRTV4K and UPIQ. A separate chroma analysis covers HDRTV4K and UPIQ, totaling 58 parent scenes. Tests use the training Reinhard tone curve and an ACES-like curve not used for training.

What the results show

In the frozen-VAE probe, EquivFlow retained more linearly recoverable chroma at deep exposures than the tested log-like encodings. The round-trip results differed by metric: Log-Gamma scored higher than EquivFlow on the reported reconstruction measure, while the paper reports that the log-like alternatives retained less recoverable chroma at deep exposure. These representation tests are not themselves downstream video-reconstruction results.

Across the four tested VAE hosts, the authors report that exposure-linked structure and deep-dark chroma transport persisted. However, latent coordinates differed across hosts, and a readout fitted at ordinary exposure degraded when applied unchanged at deeper exposures. This supports a limited structural-portability claim, not broad transfer of a single shared representation.

In the video comparison, the complete EquivFlow system had lower chromatic error than the matched LogC3 system across the tested tone curves and reported chromatic measures. Aggregate reconstruction was less consistent: under the ACES-like curve, LogC3 led on aggregate PSNR in the main comparison. Statistical tests on the frozen subset found that ARRI did not separate under either curve; HDRTV4K did not separate under Reinhard; and UPIQ separated under both. The full HDRTV4K–UPIQ chroma panel favored EquivFlow on its reported primary chromatic measures. These findings apply to the tested systems and footage, not every HDR workflow.

Limits and context

There was one trained model per internal system, so the reported sampling-based uncertainty does not capture variation across training runs. Two of the 12 clips in the encoding’s out-of-distribution survival selection panel also appear in the frozen ARRI evaluation subset; no downstream prediction or metric was used to select the encoding. Qualitative examples were selected successes rather than typical cases.

The work is an arXiv preprint. The supplied source does not establish peer-review status, so readers should distinguish the authors’ reported experiments from independently validated conclusions. Broader testing across VAE families and reconstruction settings remains an open question.