Heart failure prediction models need stronger validation, review finds

Predicting mortality risk in people with heart failure could support patient care and healthcare planning. But developing a model and establishing that it works beyond its original patient group are different tasks. A systematic review presented as an arXiv preprint highlights that distinction: the authors found increasing use of machine learning alongside substantial variation in how prediction models were developed, described and evaluated.

The review does not establish a best model for clinical use. Its central finding is that inconsistent study designs and limited external validation make the available tools difficult to compare.

What the researchers reviewed

The authors searched PubMed, IEEE and Scopus for studies published between January 1, 2009, and January 1, 2026. They focused on models predicting all-cause mortality—death from any cause—over periods of one to 60 months in people with heart failure.

Of 1,516 records initially identified, 56 studies met the review’s criteria. The criteria excluded, among other cases, predictions of death during the initial hospitalization, outcomes combining hospitalization with mortality, and cohorts defined by a surgical intervention. The findings therefore concern a particular subset of heart failure prediction research, rather than every prognostic tool.

One author performed study selection and data collection. Each paper’s risk of bias and applicability were independently assessed by two reviewers using PROBAST, a framework for evaluating prediction-model studies.

Patient definitions and validation varied

The development cohorts ranged from 117 to 35,732 patients. Their clinical descriptions also differed: the reviewers recorded 18 terms used to describe heart failure populations.

The authors classified 36 studies, or 64%, in an “Unknown/Undefined” heart failure group. This category included mixed, insufficiently described or nonstandard definitions. It should not be interpreted as meaning those patients did not have heart failure; it reflects difficulty assigning the reported cohorts to the review’s classification scheme.

External validation was reported in 13 studies, or 23%. The review defined this as evaluation using patients from an external hospital or patient group. Testing on previously unseen data from the same dataset, or a different time period within the same patient cohort, counted as internal validation.

That distinction matters to the authors’ interpretation. Without evaluation in external patient groups, confidence that reported performance will generalize beyond the original setting remains limited.

Machine learning grew, but comparisons remain uncertain

The review classified 21 studies as using conventional statistics and 35 as using machine learning or deep learning. The proportion using machine learning or deep learning increased over the period examined, while the proportion using conventional statistics declined.

These categories depended partly on how the original studies described their methods. Logistic regression, for example, appeared in both the conventional-statistics and machine-learning groups, depending on its use and presentation.

For short-range predictions, defined as one to six months, machine-learning and deep-learning models had higher average reported performance than conventional statistical models. However, the groups differed substantially in study numbers and development circumstances. Short-range prediction was the most common task among machine-learning and deep-learning studies, but the least common among conventional-statistics studies.

The authors describe broadly overlapping performance distributions for medium- and long-range predictions. Their comparisons do not establish that machine learning is intrinsically superior: the models were studied in different populations and under different conditions, rather than all being tested on a shared benchmark.

What the findings can support

PROBAST assessments showed no statistically significant differences between the modeling groups in overall concern about bias or applicability. Across all included studies, 25% received a high-concern rating for risk of bias, and 25% received a high-concern rating for applicability. These assessments depended on what the original publications reported.

The review itself also has limitations. Its protocol was not prospectively registered, which the authors acknowledge may increase reporting-bias risk. Assigning heart failure and computational-complexity categories required interpretation, while differences in cohorts and validation restricted comparability.

The authors’ practical recommendation is greater standardization of patient-group definitions, development reporting and external validation. As a preprint, the review should be read as a research assessment rather than settled clinical guidance. Its message is not that more sophisticated models lack value, but that reported performance needs a clear clinical and methodological context before it can guide model selection.