A clearer view inside protein AI—but steering useful proteins remains harder

The problem: reading and changing internal signals

Protein language models process amino-acid sequences through layers of numerical activity called activations. Researchers can examine these signals for associations with biological features, or modify them to influence the sequences a model generates. The difficulty is that an edited activation may no longer resemble anything the model normally produces.

One established interpretability approach, a sparse autoencoder, represents activations through a dictionary of features, with relatively few active at once. Generative Latent Priors, or GLPs, take a different route: they learn the activation distribution without requiring a sparse or linear decomposition. The study asks whether this offers a clearer biological readout and a useful way to repair edits.

Learning the landscape inside the model

The researchers kept two ESM-2 protein language models unchanged: versions with 8 million and 650 million parameters. They collected activations from one layer in each, using UniRef50 protein sequences, and trained GLPs on 100 million activations for both sizes and 500 million for the smaller model.

The GLP learns through flow matching, a generative training method connecting real activations with random noise. Learning to reverse that path allows it to generate activation-like signals. For correction, it partly noises an edited activation, then denoises it toward the learned distribution—the landscape the authors call the activation manifold.

Deeper GLPs generally produced activations closer to held-out examples, especially at 12–24 processing blocks. Training loss also improved with compute. However, the authors describe their compute fit as descriptive rather than a general scaling law: the tested range was too limited to establish one.

Biological information becomes easier to isolate

The interpretation tests examined 110 residue-level Swiss-Prot annotations and ten numerical properties calculated from protein sequences. The researchers selected individual computational units, called meta-neurons, inside the GLP and tested how well each predicted a target. Evaluation proteins were excluded from GLP training by accession.

Across the 110 annotations, mean test ROC-AUC was 0.903 for GLP units versus 0.845 for the released InterPLM sparse-autoencoder features on the smaller host. On the larger host, the corresponding values were 0.932 and 0.872. AUC measures how well a score separates positive from negative examples; these numbers are not percentages of correctly identified residues.

GLP also led the tested alternatives on average for numerical properties at the smaller scale. That advantage did not hold at the larger scale, where sparse-autoencoder features scored slightly higher, with overlapping uncertainty intervals.

Crucially, stronger individual units did not establish newly discovered biological information. Combining up to 20 sparse-autoencoder features matched the winning GLP unit for 94 of 104 concepts at the smaller scale and 101 of 105 at the larger scale. The evidence therefore points mainly to concentrating information that another representation distributes across several features.

Internal correction improves steering

The editing experiments used the smaller ESM-2 host. To steer hydrophobicity, charge and a sequence-derived instability index, the researchers calculated activation directions contrasting proteins with high and low property values. They repeatedly masked and regenerated 10% of sequence positions over 30 rounds, comparing edits with and without GLP correction.

Correction produced clearer separation between upward and downward steering across most tested strengths. In an isolated activation test, it also removed more than 90% of the excess diffusion loss caused by steering, according to the GLP’s own measure of typicality.

Correction was not neutral, however: even without steering, it shifted amino-acid composition toward greater hydrophobicity. The analysis accounted for this baseline drift rather than counting it as successful control.

The final sequence remains the bottleneck

Once the corrected activations were decoded into amino acids and the finished sequences passed through ESM-2 again, their improved activation typicality was lost. Predicted folding confidence also deteriorated. Even without steering or correction, repeated decoding reduced mean ESMFold pTM from 0.650 to 0.294 in one starting set.

The authors’ comparisons implicate extensive sequence rewriting rather than GLP correction as the main structural cost in this setup. Beam search better preserved predicted folding confidence at small edit budgets, but changed the target property less. Larger editing budgets strengthened the effect while reducing structural confidence.

Separate structural-steering tests consistently improved helix content, but did not produce complete zinc-finger motifs. Apparent coiled-coil gains reflected single helices being scored as hits rather than convincing paired structures.

What the evidence supports

These are computational results, not laboratory demonstrations of functional proteins. Folding confidence and sequence-derived property scores are proxies, and the interpretability tests establish associations rather than causal biological mechanisms. Evaluation was dominated by domains and motifs, used one layer per host, and offered GLP a larger candidate-unit pool than the alternatives. The ranking was also not established across different evaluation metrics.

The work is arXiv preprint version 1, submitted on 6 November 2026. Its landing record reports acceptance at the NeurIPS 2026 Interpretability for Discovery Workshop, not the main conference. Its central result is a distinction: making biological information easier to read, and internal edits more typical, does not yet ensure structurally plausible protein sequences.