When a word stretches, looks matter—but examples matter too

Learning a word involves more than matching a label to a picture. If several dogs are called “feps,” should a cow count as a fep? A learner might be guided by how similar the cow looks to the examples. The examples themselves may also offer clues about the word’s scope: does it name a narrow kind of animal or a broader category? A preprint by Patricio M. Vera investigates both influences using existing adult data and artificial learners.

Measuring resemblance in pictures

For the human analysis, Vera revisited trial-level data from 12 online experiments reported by Lewis and Frank in 2018. Adults learned a made-up word from pictures of objects, then chose which of 24 pictured items the word could also describe. Teaching conditions varied the number and category of examples: participants might see one Dalmatian, three Dalmatians, several kinds of dog, or several kinds of animal.

The new analysis passed the original images through DINOv2, a self-supervised image model. The researchers used its numerical representations to calculate how far each test picture was from the examples shown in that trial. This is distance in the model’s representation space—not a direct measure of what a person sees or thinks.

Across 520 adults, greater distance was associated with a lower chance of selecting an item as another example of the word. In the confirmatory analysis, the researchers controlled for taxonomic level and teaching condition across all test items. However, that overall result was strongly influenced by the contrast between items inside and outside the taught domain. In an additional, exploratory post hoc analysis restricted to items within the taught domain, distance still contributed after accounting for taxonomic level and teaching condition. Because distance and taxonomic level are related in these stimuli, neither result establishes that visual resemblance independently causes people’s choices.

The teaching examples also mattered. Among the three-example conditions, adults extended a word to 1.1% of superordinate items after three subordinate examples, such as three Dalmatians, compared with 77.5% after three superordinate examples, such as different animals. This descriptive contrast compares those two three-example conditions; it is not a comparison across every teaching condition. It shows that the composition of examples can change extension dramatically, even when the number of examples is held at three. Exploratory analyses also suggest that the relationship between model-space distance and extension varied with teaching condition.

A controlled artificial learner

The second part examined a setting where the representation was known. The artificial learner used the same frozen DINOv2 encoder and represented each word with the normalized mean of the image representations for its training examples. It learned labels for images from CIFAR-10 and CIFAR-100, including words assigned to two categories, and was tested on new images.

Naming accuracy fell across category pairs grouped by near, middle, and far distances in the model’s representation space: 84.7%, 65.4%, and 53.0%, respectively. In this setup, pairs farther apart in that space were harder for the learner to join under one label. This is a result about the artificial learner, not a measure of human learning. Adult word extension and machine naming were different tasks, and their scores are not directly comparable.

The researchers also compared DINOv2 distances with measures from three human datasets. The model’s distances were associated with human similarity judgments and classification confusability, but those measures include more than visual appearance alone. And this alignment did not provide conclusive evidence that human-derived distances predict which category pairs the artificial learner would find difficult: the relevant tests involved only 61 and 32 pairs. A second encoder, trained on one child’s experience, reproduced the broad ordering of difficulty across tiers in one test, but did not show the pair-level relationship in another. The main artificial-learner tiers also contained just 3, 3, and 4 category pairs, limiting how far the tier results can be generalized across pairs.

What the findings can—and cannot—say

Taken together, the results suggest that visual similarity helps describe some patterns of generalization in adults and in an artificial learner. They do not show that the two systems share a learning mechanism. People may use expectations about how words are taught; the artificial learner used image representations and fixed labels. The adult results underline that a distance measure alone does not capture everything the teaching examples contribute.

The human analysis reused data collected for other research, so the authors could not change its design. Some follow-up analyses were exploratory, including analyses developed after the main results were known. The analysis plan was fixed before the confirmatory tests but was not externally preregistered. The artificial-learner findings depend on particular image datasets, encoders, preprocessing choices, and learning methods. The authors suggest future experiments with children and tests using other encoders.

This work is an arXiv preprint, not a peer-reviewed publication, so its conclusions should be considered provisional. Its takeaway is not that AI learns words like people. Rather, a particular image model’s measure of distance tracks some patterns of word extension and artificial learning, while the composition of teaching examples contributes information that measure does not capture.