Personalized AI struggles to connect the right memories to the right person

An assistant that knows your favorite foods still needs to recognize you at a buffet and distinguish available dishes from things mentioned only in your history. MPRBench turns that kind of challenge into a controlled test of personalized reasoning across images and text.

Dohyun Kim, Junyong Lhim and Kwanyong Park introduce the benchmark in an arXiv preprint submitted on November 4, 2026. Their central argument is that recognizing a familiar person or retrieving a stored preference does not establish that a model can use personal information appropriately in a more complicated situation.

Testing what information matters

MPRBench contains 1,667 multiple-choice questions across 12 tasks, using 119 personalized entities: 69 people and 50 objects. Each entity has reference images and a structured textual profile containing information such as preferences, birthdays, ownership or past events.

Five tasks require visual reasoning, including spatial relationships, counting and detecting appearance changes. Three require reasoning from text, including arithmetic involving dates and following relationships across multiple profiles. Four require both modalities—for example, counting objects that match a person’s recorded tastes or matching an object’s visible state to its history.

Models receive visual and textual information even when only one is needed. That arrangement tests not just whether they can use personal context, but whether they can leave it aside. Query images deliberately include clutter and distracting people or objects.

The personal profiles are synthetic, rather than factual biographies of the people pictured. The researchers combine rule-based attribute assignment with Gemini-2.5-Pro generation, sometimes conditioning histories and preferences on query images. Questions come from templates, human annotation or a generation pipeline that also produces images using Gemini 2.5 Flash Image.

Human review filters the resulting material. The team reports retaining 120 of approximately 4,000 inspected images from the VCR dataset and 579 of 3,117 generated image candidates. These are therefore carefully selected test situations, not an unfiltered sample of everyday interactions.

Strong models, uneven performance

The evaluation compares simple prompting with specialized personalization approaches that learn identity tokens, retrieve profiles or organize information in memory. Accuracy measures whether the model selects the correct option; reported overall scores weight performance by question counts.

In the main comparison, GPT-5 and Gemini-2.5 Pro, supplied with personal information through prompting, both score 83.1%. LLaVA with prompting scores 48.2%, while specialized approaches range from 33.4% for RAP to 52.6% for TAME. Random selection among four options would average 25%.

These results do not show that personalization methods are universally ineffective. They show that several tested methods fail to outperform simple prompting in this reasoning-focused setting. The authors suggest that training for elementary recognition tasks may not transfer well to more demanding reasoning, but present that explanation as a conjecture.

One targeted experiment helps separate recognition from subsequent inference. On a selected visual-commonsense subset, LLaVA with prompting scores 51.43% when it must locate the target and reason about it. Providing the target’s bounding box raises accuracy to 71.43%. That 20-percentage-point difference supports the authors’ identification of visual recognition as an important bottleneck.

Distractions and reasoning have different effects

Adding misleading textual histories to visual prediction problems reduces performance for most tested models, despite leaving the correct answer unchanged. Adding four other entities’ profiles and reference images also lowers most evaluated models’ overall scores, including GPT-5 by 3.2 points and Gemini-2.5 Pro by 4.6 points.

The reverse distraction is less consistent: misleading query images in text-specific tasks produce mixed changes, rather than a clear deterioration. The evidence therefore does not support treating every irrelevant modality as equally disruptive.

Additional Qwen3-VL-8B experiments show that stronger reasoning can help without resolving recognition. Overall accuracy rises from 56.7% for Instruct to 67.2% for Thinking, with much larger gains in multimodal tasks than in vision-specific tasks.

What the benchmark establishes

Wrong answer options are designed to represent failures such as confusing identities or using irrelevant personal facts. Their selection provides diagnostic evidence, but it is not a direct observation of a model’s internal reasoning.

The evaluation also remains bounded by its synthetic profiles, selected scenes and multiple-choice format. Memory-based systems originally designed for long-term interactions are adapted to a single-turn setting, limiting conclusions about their native use. The authors acknowledge that the benchmark cannot cover every personalized scenario and do not extensively investigate remedies.

The preprint’s practical message is narrower than a verdict on personalized assistants: supplying personal information does not ensure that a model will identify its owner, select the relevant details and combine them correctly.