Bitcoin forensics benchmark loses much of its edge when time order is enforced

A strong benchmark score does not necessarily mean a system can identify suspicious Bitcoin addresses that appear after training. In a November 2026 preprint, University of Amsterdam researchers Runlin Liang, Simon Trimborn and Noud van Giersbergen argue that much of a prominent result on the Elliptic++ dataset reflects information leakage rather than reliable prediction across time.

Their central finding concerns a random forest, a classifier built from multiple decision trees. Its F1 score for illicit addresses falls from 0.846 under the original evaluation arrangement to 0.291 under their leakage-resistant pipeline. F1 combines precision—how many flagged addresses carry illicit labels—with recall—how many illicit-labelled addresses are found. It is not an overall accuracy percentage.

Two ways time enters the test

Elliptic++ extends the transaction-level Elliptic dataset to Bitcoin addresses, describing each address with 55 features summarising its activity. These include amounts sent and received, transaction fees, timing, transaction counts and interactions with counterparties.

The authors identify two problems. First, the released training and test split follows positions in the dataset rather than chronological order. Addresses from the same periods can therefore appear on both sides. A model may recognise patterns shared within those periods without learning patterns that transfer to later ones.

Second, the released features summarise each address’s activity across the sampled transaction history. For a training address, that summary can include transactions occurring after the training cutoff. Changing the split alone does not remove this future information.

Rebuilding the measurements

The team reconstructed the feature definitions using a public Bitcoin ledger dataset and a public mapping between anonymised Elliptic transaction identifiers and blockchain transaction hashes. The mapping covered 202,804 of Elliptic’s 203,769 transactions, and every mapped transaction was found in the ledger.

The resulting address sample contained 265,352 labelled addresses, including 14,266 labelled illicit. Two licit-labelled addresses were excluded because their released address strings were truncated and could not be matched.

Importantly, the reconstructed features describe activity within the recovered Elliptic transactions, not each address’s entire blockchain history. For each of the 55 features, the recovered lifetime values matched the released values to high precision for at least 99.9% of addresses. Using those recovered features also closely reproduced the original random forest result, supporting the comparison that followed.

The temporal evaluation trained on addresses first appearing in time steps 1–34 and tested on addresses first appearing in steps 35–49. Training-address histories were capped at 18 March 2017; test-address histories extended to the test period’s end on 2 October 2017. This evaluates classification at those observation endpoints, not necessarily at an address’s first transaction. Feature scaling used training addresses only under the temporal evaluation.

Most of the drop comes from the split

With released features retained but the temporal evaluation applied, the random forest’s F1 fell to 0.257. Replacing those features with leakage-resistant versions then raised it to 0.291. Thus, feature reconstruction did not produce another large performance collapse: the dominant decline accompanied the move away from the positional evaluation.

Only 1.45% of training addresses had features altered by the cutoff. The authors explain that most addresses were active for too short a period within the sample to accumulate post-cutoff information.

Logistic regression behaved differently. Its F1 rose from 0.110 under the positional evaluation to 0.285 with the temporal split and leakage-resistant features. The random forest’s final mean was therefore slightly higher, but its large original advantage had disappeared. Its F1 standard deviation across ten random seeds also increased from 0.002 to 0.058.

A benchmark correction, not a performance ceiling

The comparison preserves the earlier study’s modelling setup, using all 55 features, no class weighting and a default decision threshold of 0.5. The authors acknowledge that many features are redundant or exactly related, potentially destabilising logistic regression coefficients. They recommend feature selection or dimension reduction and more careful baseline tuning in future work.

They also deliberately call the pipeline “leakage-resistant,” not leakage-free: it closes the known split and feature channels without guaranteeing that none remain. The practical message is to re-evaluate Elliptic++ results chronologically before treating them as evidence of future detection ability. The paper further cautions that uncertain model flags should not, by themselves, justify freezing funds or action against an address owner.