When cell-predicting AI gets the baseline right but misses the change

Measuring the change, not just the resemblance

So-called cellular world models aim to predict how genetic interventions alter cells. Here, the predicted outcome is a transcriptome: a profile of gene activity measured through RNA. Such predictions could help researchers explore interventions computationally, but their usefulness depends on capturing the intervention’s effects.

The study examines Pearson correlation, a measure of how closely two profiles vary together. A global correlation compares predicted and observed expression across thousands of genes. If most genes barely respond, their familiar baseline expression can dominate the score, masking errors in the smaller response signal.

Gao instead emphasizes “delta Pearson”: subtract the average untreated control profile from both prediction and observation, then correlate the resulting changes. This asks whether the model predicts departures from the baseline. Neither correlation is a percentage of correct predictions, and dividing one by the other does not establish a corresponding multiplier in biological usefulness.

A simple baseline exposes the problem

The main experiment uses the Replogle K562 dataset, described in the paper as a genetic-disruption screen. It evaluates 509 held-out perturbations across 5,000 genes. Experiments downsampled the data to 30 cells per condition because of graphics-memory limits; the main GEARS model trained for 20 epochs.

GEARS achieved a reported global Pearson correlation of 0.977, but delta Pearson was just 0.076, with a standard deviation of 0.006 across three training seeds. That contrast is the paper’s “delta gap.”

A deliberately simple comparator predicted the same average training-set change for every intervention, ignoring which gene was targeted. Its delta correlation reached 0.267. On this evaluation, the elaborate model therefore lost to an intervention-independent average.

That does not mean the average was a good biological predictor. Both methods recovered only 1.19 of the true 20 most responsive genes per condition, on average. The result illustrates why several measures—not one headline correlation—are needed.

Can the predictions improve?

The study tests two corrections applied after training. One spreads predicted changes through a network connecting genes with correlated activity in control cells. It increased delta Pearson to 0.103. Another finds training interventions whose predicted responses resemble a test prediction, then averages their measured responses. This nearest-neighbor approach reached 0.211, still below the simple baseline.

Changing the training objective helped too. Alongside reconstructing expression, “delta-aligned” training explicitly rewarded agreement between predicted and observed changes. With the stronger tested alignment weight, delta Pearson rose to 0.179. Reported direction accuracy increased from 0.520 to 0.680, while recovery of the top 20 responsive genes increased from 1.19 to 1.52. Adding neighbor-based correction raised delta correlation to 0.217.

Because correlation does not measure whether response magnitudes are correct, the author also added a penalty for magnitude errors on strongly responding genes. The three-seed table reports delta correlation of 0.261 ± 0.038 and recovery of 2.73 ± 0.52 genes when this calibration was combined with neighbor correction. Despite improvement, the average correlation remained slightly below the simple baseline’s 0.267, and most top-response genes were still missed.

Transfer is promising but narrow

A separate experiment applied K562-trained models without retraining to the dataset the paper identifies as Norman A549. Evaluation covered only eight transferable conditions involving five perturbation genes and 741 shared measured genes. The aligned model reached a pooled delta correlation of 0.328, versus 0.158 for standard GEARS and 0.070 for the average-response baseline.

This is encouraging evidence within a small test, not proof of broad transfer across cell types or interventions. Most of the transferable perturbation genes had already appeared during source-model training.

Important limits in the evidence

An “oracle” neighbor experiment used actual test responses to select training neighbors, reaching 0.592. It demonstrates potential headroom for that retrieval procedure, but cannot operate prospectively: the answer helps choose the neighbors. It also does not establish that representation quality, rather than the output-generating component, is the only bottleneck.

The supplied text contains reporting inconsistencies. It alternates between pooled and per-condition metric descriptions, gives different Norman scores in different sections, and mixes single-seed calibration results with three-seed summaries. Some Replogle differences are attributed to data-preparation pipelines. These issues complicate precise comparisons.

The source is arXiv version 1, posted November 7, 2026; the supplied evidence does not establish peer review. It is extracted PDF text, with images omitted and possible equation or table distortions. The paper also says its code repository contains only a placeholder README, with full materials promised upon publication.

The practical lesson is stronger than any claim of a solved cellular simulator: evaluate intervention-induced changes, compare against simple controls, and report direction, magnitude and responsive-gene recovery alongside overall resemblance.