A compact way to search growing pathology image archives

Searching slides by what they show

A digital pathology slide is a high-resolution image of stained tissue. A whole-slide image (WSI) can contain many different tissue regions, so a computer cannot simply compare every pixel across a large archive. Searching by metadata can also be difficult when records are incomplete or inconsistent.

The study introduces CLEAR-WSI, a system for retrieving candidate slides related to a query. Its central design choice is to turn each slide into one fixed-length numerical representation, or embedding. This makes storage per indexed slide predictable, even when slides contain different amounts of tissue. The archive can also add each processed query as a new searchable case.

The goal is case retrieval, not automated diagnosis. Finding candidate cases could support review or exploration, but this study measures whether returned slides have matching dataset labels—not whether a pathologist’s decisions improve.

From tissue patches to a ranked list

Because a whole slide is too large to process directly, the researchers first identify tissue and divide it into smaller patches. A pathology image encoder turns each patch into a feature vector. An attention-based multiple-instance learning module, called Diagnostic-driven Aggregation (DX-Agg), combines the variable number of patch features into one slide-level embedding. During training, slide-level diagnostic labels guide the aggregation, allowing regions useful for predicting the slide’s label to receive greater weight.

For a query, CLEAR-WSI ranks archive entries by embedding similarity, then applies one of two label-based filters. In Self-Retrieval (SR), it uses its predicted diagnosis for the query and keeps candidates with matching predicted labels. This is the automated setting that does not require the query’s diagnosis. In Class-Informed (CI), it uses the query’s ground-truth diagnosis. CI is therefore an oracle comparison: it shows what happens when the correct query label is available, not the ordinary automated scenario.

After retrieval, the system adds the query’s embedding and predicted label to the memory bank. This grows the searchable archive but does not update the model’s parameters online. The researchers compared CLEAR-WSI with three existing retrieval approaches and tested variations in the image encoder, aggregation method, filtering, and archive growth.

What the tests found

The benchmark covered four tasks on three datasets: CAMELYON16, which distinguishes normal from tumour slides in lymph nodes; two BRACS breast-tissue classification tasks, one with three broad groups and one with seven more detailed categories; and X-HER2, a private clinical dataset used to retrieve slides by HER2 status. X-HER2 contained 77 slides. During evaluation, memory banks ranged from tens to hundreds of entries—not millions or hospital-archive scale.

In the automated SR setting, CLEAR-WSI had higher Precision@5 than the strongest evaluated baseline on CAMELYON16 and both BRACS tasks. Precision@5 is the share of the first five retrieved slides whose labels match the query’s label. The reported gains were 3.10 percentage points on CAMELYON16, 2.75 on BRACS-1, and 1.95 on BRACS-2. On X-HER2, SR matched the strongest baselines at rank one, was slightly lower at rank three, and exceeded them at rank five on the reported precision measure. Results varied by task and ranking measure; they were not a uniform win on every metric.

CI performed better than SR across the tasks, but this comparison assumes access to the correct query diagnosis. It illustrates the potential value of accurate diagnostic information for filtering; it is not the performance of a system that independently knows the answer. Ablation tests also found improvements from diagnosis-guided aggregation, filtering, and adding cases to the memory bank in the tested settings.

Latency tests ran on one NVIDIA H100 GPU. Feature loading and retrieval took a few seconds across the test sets, while feature extraction required a separate offline step. These measurements concern small evaluated memory banks; they do not show that the same speed or performance will hold across millions of slides or different hospital systems.

What remains uncertain

The authors identify important limits: the evaluation covers three datasets and four tasks, only one dataset is a private clinical cohort, and relevance is defined by matching dataset labels. Two slides with the same label may not be morphologically similar in ways a pathologist finds useful; meaningful similarities may also cross label categories. The study does not report reader trials showing that retrieved cases improve diagnostic practice.

The article is an arXiv preprint submitted to Elsevier, so it should be treated as preliminary rather than proof of clinical readiness. Its comparisons support further investigation of compact, diagnosis-informed slide retrieval—not a claim that CLEAR-WSI is validated for routine care or proven to scale to large clinical archives. Larger and more diverse evaluations, robustness testing, and studies involving pathologists would help establish its practical value.