What Can the Shape of Neural Network Weights Tell Us About Learning?

A model can train on meaningful labels, on an arbitrary assignment, or on targets that keep changing. Might those situations leave different signatures in its weights—and would such a signature tell us anything useful? In an arXiv preprint, Mitra Varun Gogulapati tracks the geometry and topology of neural-network weights during training under these three conditions.

Here, “topology” does not mean the literal shape of the network. The analysis treats selected neuron weight vectors as points in a high-dimensional space. Persistent homology summarizes how structures in that point cloud appear and disappear as the distance scale changes. The analysis includes loop-like features, but these describe the point cloud—not literal cycles in the network or evidence of a particular learning mechanism.

Three kinds of supervision

The author trained the same multilayer perceptron on Fashion-MNIST, a dataset of clothing images. In the correct-label condition, images retained their actual labels. In the fixed-random condition, labels were shuffled among training examples once and then held constant. In the changing-random condition, labels were reshuffled every epoch. This last condition is an extreme test of shifting supervision, not a typical continual-learning benchmark.

Five paired seeds produced 15 models. For each seed, the three models shared their initialization and minibatch order, making the condition comparison more controlled. Each model trained for 100 epochs. At 22 checkpoints, the analysis examined point clouds made from incoming weight vectors in five hidden layers. It calculated H0 and H1 persistence—summaries of connected components and loop-like features—using both raw weights and vectors normalized row by row. Normalization removes each vector’s individual magnitude, but does not make the measurement invariant to every way network weights can be represented.

What the study measured

In this particular setup, correct-label models achieved a mean 89.39% accuracy on the original test task. Fixed-random models reached 67.94% accuracy against their assigned training targets, while averaging 10.29% on the true-label test task. Changing-random models recorded 10% accuracy on both assigned targets and the true-label test set. These are outcomes for the tested architecture, dataset and training schedule; they do not establish what must happen under other conditions.

One topological summary showed a descriptive difference: time-averaged H1 persistence in normalized weights from the third hidden layer was higher for correct-label models than for either random-label condition. The differences had the same direction across all five paired seeds. However, the configured exact comparisons did not reach conventional significance after Holm correction for multiple comparisons. The paper also reports unadjusted t intervals that exclude zero for these persistence contrasts; those intervals use a different statistical approach and are not multiplicity-corrected. With five seeds, the exact test’s smallest attainable two-sided p-value was 0.0625. The results suggest a pattern to investigate, but do not establish a statistically significant difference under the configured testing procedure. Nor does failure to reach significance show that the conditions are equivalent.

The analysis also tested whether measurements from the first five epochs could help classify a model’s supervision condition. Mean balanced accuracy was 60.0% with topology features, 66.7% with conventional diagnostics, and 60.0% with both. In this evaluation, combining topology with the baseline did not improve the observed score over topology alone and scored below the baseline. The result is limited: each held-out seed contributed only three test models, the outer training set contained 12, and the feature sets were large relative to the sample. It is not evidence that topology cannot help in other studies.

What remains uncertain

The topology summaries varied with layer and with whether weights were normalized. They could reflect ordinary geometric or scale changes; the study does not distinguish these explanations from a distinct signature of meaningful learning. It establishes no causal role for topology, topological phase transition, or general way to detect learning quality.

The scope is narrow: five seeds, one architecture, one dataset and one optimizer. Some H1 distance calculations were omitted when diagrams exceeded a feature cap. All 282 omissions were in normalized H1 comparisons and were treated as missing, not zeros. Bottleneck-distance analysis was deferred. The paper reports an archive of configured topology cases and describes associated analysis materials, but supplies no public repository identifier.

This work is an arXiv preprint, not a claim of peer-reviewed validation. Its contribution is a controlled, carefully qualified comparison. In this experiment, topological measurements showed descriptive differences, while the adjusted comparisons and small early-prediction evaluation leave their incremental value unresolved.