Can modular spiking networks learn to choose actions?

A controller community receives information from the environment; each possible action has its own community; and additional “latent” communities can carry activity within the network. Rather than assigning each action to a single output neuron, the design reads out the action from activity reaching its action community. The authors call this a Community Recurrent Spiking Neural Network, or CoRe SNN.

The idea is to combine recurrence with modular structure: connections are dense within groups and relatively sparse between them. Since spikes travel along connections, activity can be viewed as a walk through neurons and, at a broader scale, communities. The authors use the map equation—a measure of how compactly a random walk can be described—to motivate a learning rule that encourages activity to remain within modules.

Three ways to form communities

The study compares three arrangements. In the dynamic version, a new rule called Map-STDP adjusts connections during learning to encourage modularity. In the partial version, the network starts with some modular structure, generated using a stochastic block model, while connections between communities remain possible. In the full version, communities are sealed off from one another, apart from designated input routing. Learning uses local changes based on spike timing together with broader signals related to reward or prediction error—a “three-factor” rule.

The researchers first tested standalone agents on CartPole, a task in which an agent must balance a pole. They compared four computational engines: three spiking-neuron models and a non-spiking flow model that represents activity as a walk. They then inserted recurrent action communities into a spiking actor–critic system and tested it on CartPole, Acrobot and LunarLander. The paper reports multiple random seeds, and all of these experiments were simulations rather than tests on a physical neuromorphic chip.

What the simulations showed

On CartPole, Map-STDP improved average training return over the same network without the modularity-learning term across the three spiking models. It matched the fully sealed design on two of those models. The dynamic and full arrangements also produced more compressible spike walks than the partial arrangement in these experiments. This suggests that the rule can shape activity into a modular pattern under the tested conditions; it does not show that every modular design will work equally well.

In the actor–critic tests, replacing the feed-forward actor with recurrent action communities improved results on CartPole and LunarLander, and matched the feed-forward version on Acrobot. The best degree of modularity depended on the task: full modularity helped most on Acrobot, whereas the different regimes performed similarly on LunarLander. In CartPole ablations, removing latent communities often reduced performance. However, the authors found that these communities mostly acted as sinks or routers for activity, rather than carrying out useful computations of their own.

The paper also explores whether compact activity might mean lower energy use. It estimates “synaptic operations” by counting spikes transmitted along connections that remain above a weight threshold. Across pooled settings, this proxy tended to increase with the map-equation description length. That is an association between model-based measures, not a direct measurement of electricity use or proof of an energy advantage on hardware. The relationship varied by task. For example, LunarLander showed exceptions, including a partial-modularity network that became less modular during training.

What remains uncertain

These results support CoRe as a candidate architecture in the tested benchmark simulations, not as a general replacement for feed-forward networks. Performance differed across tasks and neuron models, and the authors report that the effects of varying inter-community connections depend on both task and settings. The map equation is one possible principle for forming modules, not the only one. The authors also note that estimating random-walk statistics may become a bottleneck as networks grow. They leave open whether latent communities can be trained to store memories or perform distinct tasks.

The comparison with biology is also about design features, not evidence that brains learn this way. The authors identify parts of their learning rule that have no established biological counterpart. And although sparse, event-driven computation can be compatible with neuromorphic hardware, this study does not establish real-world energy savings in a deployed system.

This article is based on the version-one arXiv preprint identified in the source. The available evidence does not establish a peer-review outcome. Replication, broader benchmarks and tests on hardware would help clarify how robust and practically useful the approach is.