Finding surprising shopping patterns without abandoning statistics

Finding products that appear together in shopping baskets is an established data-analysis task. Finding associations that are both trustworthy and worth investigating is harder. Strong statistical patterns can be obvious, while unexpected pairings can simply be noise.

In an arXiv preprint, Lijia Liu and Hisashi Kashima of Kyoto University introduce DiSARM, a system that divides these responsibilities. Conventional algorithms find candidate associations, statistical tests screen them, a language model scores their surprise, and a final selection step reduces repetition. The results support a compromise rather than a method that wins on every measure.

Let the data generate the candidates

An association rule describes a pattern such as shoppers buying one item also tending to buy another. It does not establish that purchasing the first causes purchasing the second.

Three quantities help describe these rules. Support measures how often the items occur together. Confidence measures how often the second item appears among baskets containing the first. Lift compares that conditional frequency with the second item's overall frequency; a lift above one indicates a positive association.

DiSARM starts with FP-Growth, an established algorithm for finding frequent item combinations. This avoids asking a language model to discover patterns by reading a potentially enormous transaction table.

The researchers split each dataset into 70% for mining, 10% for internal validation and 20% held aside for final evaluation. Candidate rules must survive checks on the internal validation portion before the language model evaluates them.

Those checks require positive lift and a statistically significant result from a one-sided Fisher's exact test, adjusted using the Benjamini–Hochberg procedure. The adjustment addresses the increased risk of false discoveries when testing many rules. Passing this screen is evidence of an association, not a guarantee that it will survive another test.

Ask what is unexpected, then avoid repetition

GPT-5.1 scores surviving rules from one to five according to how difficult they would have been to anticipate from ordinary knowledge of consumer behavior. This is a judgment about expectations, not another measurement of statistical strength.

Next, DiSARM represents each rule as a numerical vector encoding its meaning. A selection method called a determinantal point process favors a subset combining high-ranked rules with dissimilar meanings. The goal is to avoid filling the final list with variations on essentially the same relationship.

The experiments select ten rules per method. They cover roughly 3.35 million Instacart transactions, 7,501 Retail transactions and 9,835 Groceries transactions, with results averaged across five random data splits.

A measurable trade-off

On Instacart, all DiSARM-selected rules passed the final holdout significance criteria. Their average surprise score was 2.00, compared with 1.00 for each of three classical statistical baselines. However, some classical methods selected associations with substantially higher lift.

A language-model baseline explicitly prompted to find surprising rules scored higher for surprise: 3.38 on Instacart. But only 44% of its selected rules passed holdout validation. On Retail and Groceries, that baseline's passing rates fell to 10% and 2%, respectively.

DiSARM's corresponding rates were 74% and 84%. Those results were stronger than the surprise-focused language-model baseline, but weaker than the classical baselines, whose rates remained at least 92% across the datasets. Internal validation therefore improved the selection process without eliminating failures on unseen transactions.

The diversity component also mattered. Removing it reduced Instacart's item-level diversity score from 0.93 to about 0.67, while generally increasing surprise. Selecting varied rules carried its own trade-off.

Surprise is not ground truth

For final surprise evaluation, the researchers used Gemini-3.1-Pro rather than the model performing selection. Human scores for 170 distinct rules from one experimental split showed a Pearson correlation of 0.72 with that evaluator. This supports agreement, not an objective definition of surprise.

The authors acknowledge that surprise depends on culture and individual expectations. Statistical thresholds can also exclude rare but potentially informative associations. Moreover, all three tests concern shopping baskets, leaving performance in other domains unestablished.

As a preprint, the work should be treated as preliminary. Its central contribution is a division of labor: use observed transactions to establish candidate patterns, and language models to help prioritize which patterns might deserve a closer look.