A small sample can expose big problems in AI training data

An AI training dataset can contain billions of images and captions. Reading every caption and checking every picture is beyond the reach of a small research team. But does finding problems really require inspecting everything?

Giovana Marchetti and colleagues investigate a more manageable approach: draw random samples, then use structured documentation questions to guide close human inspection. Their November 2026 arXiv preprint reports privacy concerns, stereotypes and technical inconsistencies in samples from two image–text datasets. It demonstrates a way to uncover problems, not a method for declaring an entire dataset safe.

Questions that guide inspection

The researchers combined dataset documentation with direct examination of sampled examples. They used four existing frameworks: Datasheets for Datasets, Data Statements, Dataset Nutrition Labels and Data Cards. These ask questions about matters such as collection methods, representation, sensitive information and intended uses.

Some answers come from documentation rather than images. Others require inspecting captions, pictures and annotations together. Scripts can help count terms or labels, but people must interpret their meaning and context. The process is iterative: a discovery while answering one question can change an earlier answer.

The team first applied all four frameworks to a 1% sample of Framed Multi30K, a curated multilingual dataset. That sample contained 316 images, alongside English and Brazilian Portuguese captions and semantic annotations—labels describing situations, people and their roles.

Curation does not settle every question

The Framed Multi30K audit identified concerns about identifiable children, adult women described as “girls” or “meninas,” and limited representation of linguistic varieties. The original Portuguese descriptions reflected highly educated, urban speakers from Brazil’s Minas Gerais state, rather than the full diversity of Brazilian Portuguese.

Automatic annotation also treated related descriptions differently across languages. In one example, “white” in an English caption received a general descriptor label, while “branco” in its Portuguese translation received an ethnicity-related label. The researchers connected this difference to annotation patterns in the resources used to train the automatic labeling system.

Such distinctions matter because a dataset contains more than pictures. Captions and labels encode decisions about which characteristics deserve mention and how they should be classified. Human curation can introduce or preserve these choices, even when it improves other aspects of a dataset.

The four documentation tools produced broadly similar findings in this case. For the second dataset, the researchers therefore used only Datasheets for Datasets and examined two separate samples.

What remained after safety filtering

The second dataset, relaion2B-en-research-safe, is a safety-filtered collection of web-sourced image–text pairs. The team drew two non-overlapping random samples of 1,200 entries each, approximately 0.00006% of the dataset per sample.

Retrieval and image-validity filtering reduced these to 668 and 691 pairs. Even among those retained, some pictures were blank or access-restriction placeholders. These losses mean the inspection findings concern the retrieved material, not every originally selected entry.

The first retained sample contained 40 examples pairing personal imagery with information the researchers classified as sensitive, including names, locations and personal relationships. The second contained 43, with examples including phone numbers, health information and legal history. Recognizable children’s faces appeared in both samples.

The auditors also flagged 22 examples in the first sample and 24 in the second as potentially offensive, stereotyping or anxiety-inducing. Their categories included distressing imagery, body shaming and subjective descriptions of people. These findings illustrate a mismatch between an automated image-safety score and broader questions about privacy, language and context. They do not establish that every distressing image is inappropriate for every research use.

Finding a problem is not measuring its reach

The study’s strongest practical message is that scale need not prevent inspection from beginning. A small sample can expose concrete failures worth documenting or investigating further.

Its limits are substantial. Small samples can miss rare but serious problems, and the authors explicitly caution against using them to reliably estimate dataset-wide prevalence. Finding an example establishes its presence; it does not, by itself, establish how widespread it is or prove its cause.

The judgments also depended on the auditors. The team came from computer science and linguistics, acknowledged geographic and cultural limits, and did not formally assess the reliability of its classifications. The study did not test downstream model behavior or demonstrate that the proposed audits reduce harm.

This is a preprint, not an established safety standard. Its contribution is a feasible starting point: inspect manageable samples, record what they reveal, and keep investigating rather than treating dataset size as a reason to look away.