MADSNet turns spike-camera signals into images

From spikes to a picture

In a conventional camera, each pixel collects light over an exposure interval, so fast motion can blur the result. A spike camera works differently: each pixel accumulates incoming light and emits a binary spike when it reaches a threshold, then resets and starts accumulating again. Brighter regions generally produce more frequent spikes. This captures rapid changes, but the output is not a conventional image.

Reconstructing intensity from these signals is difficult because a short binary spike sequence can be consistent with different underlying light patterns. The network must infer continuous pixel values from sparse observations affected by motion and photon-arrival noise.

MADSNet—short for Multi-scale Axial-attentive Dense Spiking Network—aims to solve this problem while retaining the event-driven computation associated with spiking neural networks (SNNs). Its spiking layers use accumulate-only operations, in contrast to the multiply–accumulate operations common in conventional artificial neural networks (ANNs).

A staged reconstruction

MADSNet uses an encoder–decoder: it first builds a representation of the input spike frames, then turns that representation into an intensity image. Its opening module processes input at several spatial scales to capture local detail and broader patterns. Axial spike attention gathers context along horizontal and vertical directions without building a full two-dimensional attention map. A dense bottleneck connects intermediate features so later stages can reuse earlier information. In the decoder, pixel-shuffle upsampling restores image resolution, and a fusion block bridges discrete spike-domain features and continuous-valued output pixels.

The authors evaluated grayscale reconstruction on the SREDS benchmark and tested transfer to color reconstruction on RBSS. They also report visual comparisons on real-captured grayscale and color spike streams. For grayscale evaluation, they followed the SSIR protocol and used the first 22 ground-truth frames in the validation split. These tests evaluate reconstruction performance; they do not demonstrate deployment on dedicated neuromorphic hardware.

What the comparisons show

On SREDS, MADSNet recorded a peak signal-to-noise ratio (PSNR) of 37.232 decibels. The paper reports this as 4.30 dB above the spiking baseline SSIR and 1.94 dB above the ANN-based Spk2ImgNet. PSNR measures agreement with a reference image, with higher values indicating closer pixel-level reconstruction under that metric. MADSNet also scored better than SSIR on the reported structural-similarity and learned perceptual metrics. This does not mean it led on every measure: the authors note that it did not achieve the best value on every no-reference quality metric.

In the RBSS color-transfer experiment, MADSNet scored 29.36 dB PSNR, compared with 24.31 dB for SSIR. It also exceeded SSIR on the other two reported metrics, while the dedicated color method SSL-CSC had slightly higher PSNR. The results suggest that the architecture transfers to this color setting, not that it is uniformly superior to specialized color methods.

The efficiency comparison is specific to the reported setup: an NVIDIA RTX 4090 and the tested input resolution. MADSNet’s inference latency was 20.4 milliseconds, compared with 305.4 milliseconds for Spk2ImgNet and 3.3 milliseconds for SSIR. The authors estimate energy using operation counts and per-layer spike firing rates, rather than reporting a direct hardware power measurement. By that estimate, MADSNet used about one-fortieth the energy of Spk2ImgNet. But it used more estimated energy, memory, and time than SSIR. Its efficiency advantage is therefore relative to that ANN baseline, not every spiking alternative.

Ablation tests, which remove or replace one component at a time, showed lower SREDS scores when key modules—including the multi-scale stem, attention, dense bottleneck, upsampling, or fusion—were removed or changed. Such tests indicate which design choices mattered within this model and setup; they do not establish that the same components will help across other datasets or hardware.

What remains uncertain

These are benchmark results, not a guarantee of performance with every camera, scene, or deployment. The grayscale evaluation follows a prior protocol, and the color experiment tests transfer on a separate benchmark. The paper’s table notes that results for some existing methods are taken from prior work, making consistent comparison conditions important. The energy advantage is estimated, and MADSNet is explicitly slower and more energy-intensive than the shallower SNN baseline SSIR. The authors identify hardware deployment and further model slimming as future directions.

The supplied evidence gives conflicting signals about publication status: its metadata identifies an arXiv item, while the paper text bears an IEEE Transactions on Image Processing heading. The available material does not resolve the work’s publication or peer-review status, so it should not be described as either a confirmed journal publication or an unreviewed preprint. Independent assessment and reproduction would help establish how well the results generalize.