When a Better AI Score Means an Easier Cheese-Recognition Task

A cleaner catalog, or a different question?

Product-image collections assembled through web searches inherit messy labels. Search results mix photographs of food with branded packaging, while folders may distinguish “cheddar,” “mature cheddar” and regional varieties that look similar in photographs. Combining such entries can make recognition scores rise—but does the model actually improve?

In the preprint “Scoring, Not Learning,” Kaden Leis and Xiang Ma of the University of Wisconsin–Eau Claire examine that distinction. Their finding is specific: after filtering and deduplicating one cheese-image archive, almost all the measured benefit of combining categories was available without retraining. The system was answering a broader question, not demonstrably becoming a better recognizer.

Building a less misleading test

The researchers started with 165,769 readable images across 1,824 cheese-named folders in the public Cheese Pics collection. They trained a simple packaging filter using 208 manually labeled examples and image features from CLIP, a pretrained image-and-text model. The filter retained 62,174 images, or 37.5% of the archive.

Duplicate removal was equally important. A conventional split based on filenames left 60.5% of test images with a pixel-identical counterpart in training. That creates an opportunity to recognize previously seen photographs rather than generalize to unfamiliar ones.

The team instead grouped identical and near-identical photographs using image hashes, keeping each detected group within one partition. The resulting recognition dataset contained 998 categories and 35,986 image entries, with 5,317 reserved for testing. Entries can share a photograph across category folders, so those totals do not represent entirely distinct pictures.

The researchers then fine-tuned three ConvNeXt image-recognition models, containing 28 million, 50 million and 88 million parameters, for 25 epochs. Each experiment used the same split and was repeated with six random training seeds.

The crucial comparison: change the scoring, not the model

Using a fixed external cheese catalog and rules based on name endings, the researchers combined 191 entries into 37 groups. This reduced the overall category count from 998 to 844. For example, names ending in “cheddar” could become one cheddar category. The mapping used no model predictions or test-image information.

They compared two routes to predicting those broader categories. One involved training again with the merged labels. The other left the original model untouched and added together its predicted probabilities for entries belonging to each broader group. The group with the largest combined probability became its answer.

For the 50-million-parameter model, average top-1 accuracy—the proportion of test entries receiving the correct first-choice answer—rose from 26.11% under the original labels to 30.36% under this rescoring. That is a gain of 4.25 percentage points without additional learning. Retraining reached 30.44%, just 0.08 points higher.

The extra benefit from retraining was similarly unresolved at the other sizes: −0.10 points for both the smallest and largest models. Across-seed standard deviations ranged from 0.53 to 0.65 points. These results do not prove that retraining has exactly zero effect: the middle model’s retraining benefit had a 95% confidence interval from −0.60 to +0.76 points.

The scoring method mattered. Simply taking the original winning label and translating it into a broader category missed probability spread across related entries. That weaker approach would have made retraining appear to contribute about one percentage point that probability pooling already delivered.

Broader categories drove the gain

Combining arbitrary categories did not reproduce the result. In 10,000 random regroupings of the same selected entries, preserving group sizes, the average rescoring gain was only 0.68 points, versus 4.27 points for the reference model run with the catalog-based mapping.

But meaningful grouping is not necessarily correction of duplicate names. The authors classified the merged groups as either aliases for one product or combinations of distinct products into broader types. The latter accounted for about 94% of the gain. The broad “blue” group alone, combining 43 entries, accounted for a 1.82-point reduction in the gain when dissolved.

Larger models still helped modestly: moving from 28 million to 88 million parameters added 1.50 points on the original task and 1.42 points on the consolidated task. The study detected no interaction between consolidation and model size, although uncertainty still allowed appreciable differences.

What the findings do—and do not—support

All runs shared one split, so six seeds measure training variability, not sensitivity to different dataset partitions. Training budgets were limited, and batch sizes differed between model sizes. The name-based merges were not verified image by image.

The packaging filter also remained imperfect, and approximately 45% of the archive’s named categories failed to enter the final recognition task. These were web photographs of unpackaged cheese, not a checkout evaluation; accuracy near 30% does not establish deployment readiness.

The practical lesson is narrower than “data cleaning does not work.” When cleaning changes the categories being judged, evaluate the existing model under those same categories before crediting retraining. A broader answer can be useful—but it is not the same as learning to make finer distinctions.