AI finds overlooked themes in BPD interviews, but expert checks remain essential

Comparing interpretations of lived experience

A preprint by Marcin Moskalewicz and colleagues examines whether language models can support qualitative analysis of interviews with people diagnosed with borderline personality disorder, or BPD. The task was to identify meaningful patterns in personal narratives, not diagnose patients or recommend treatment.

The researchers used approximately 150,000 words of Polish interview transcripts from 24 hospitalized participants. All were white, Polish and aged 18–29. The original human analysis explored identity, relationships and the experience of time through a phenomenological approach: interpreting how people experience their lives, including meanings that may not be explicit in individual statements.

The study compared that analysis with outputs from GPT-4o, Gemini 2.5 Pro and Claude 4 Opus. Its three components examined expert judgments, computational similarities between texts, and public perceptions of individual themes.

How the models were guided

Each model received the same sequence of instructions. These described the professional background and perspective of the original lead investigator, supplied contextual research and explained the qualitative method. The models then analyzed the interviews in three rounds before critiquing and synthesizing their outputs.

This was therefore a test of a particular persona-based prompting procedure, not an assessment of what the models would produce without guidance. The authors acknowledge that different prompts could change the results.

The outputs also differed substantially in length. Human researchers produced 12 themes totaling 2,031 words. GPT produced five themes in 220 words, Gemini seven in 681 words, and Claude 11 in 412 words.

Agreement depended on the measure

Two researchers involved in the original study compared the models’ themes with the human interpretation. Under their measure of full semantic coverage, Claude captured seven of the 12 original themes, or 58%, while Gemini captured five, or 42%. GPT scored zero on that measure.

That zero did not mean GPT’s analysis contained no related material. A separate thematic measure, using a threshold of more than 50% shared meaning, gave GPT a Jaccard similarity score of 0.21, compared with 0.27 for Gemini and 0.28 for Claude. The measures assessed overlap differently and should not be treated as interchangeable accuracy scores.

Each model also supplied one previously omitted theme that the original researchers judged insightful after checking the transcripts. These concerned fragile moments of coherence and hope, feelings of otherness and a wish for normalcy, and bodily manifestations of emotional distress. These findings suggest a way to identify omissions; they do not establish that AI generally removes research bias.

Gemini received the strongest expert ratings

Four experts assessed the complete analyses using eight criteria, including grounding in the interviews, coherence, richness and credibility. Two were internal researchers who knew the sources. Two external experts read the original transcripts and initially assessed the analyses without knowing that some were AI-generated.

Across all four raters, the human analysis averaged 4.56 out of five, followed by Gemini at 4.21, Claude at 3.68 and GPT at 2.65. The reported comparison found no statistically significant difference between human and Gemini ratings. That result is not proof of equivalence or general clinical expertise, particularly with only two external experts.

A computational comparison converted theme descriptions into numerical representations of meaning and writing style. Semantic similarities were broadly comparable across sources, while stylistic differences were more pronounced. Gemini’s style was closest to the human analysis. The authors suggest that presentation might have influenced expert judgments, but this explanation remains untested.

Plausible themes are not necessarily faithful themes

A separate study recruited 115 English-fluent public evaluators. They rated individual themes for how well their titles matched their descriptions and how likely they seemed to be AI-generated. They did not see the interview transcripts.

Participants did not reliably distinguish human themes from model-generated themes. Longer descriptions were more likely to be judged AI-written, but length was not significantly associated with title–description match. These judgments concern presentation, not whether an interpretation faithfully represents participants’ experiences.

A supporting role with clear limits

The findings come from one narrowly defined interview dataset and one prompting approach. They do not demonstrate diagnostic accuracy, treatment benefits or generalization to other patient populations. Whether models can move beyond thematic lists to an integrated phenomenological account also remains uncertain.

The authors report that transcripts were anonymized before AI analysis and that outputs were critically reviewed. They also highlight the risks of sending sensitive medical material to external providers and recommend locally hosted models for optimal deployment.

The practical conclusion is limited but useful: models may offer alternative interpretations worth investigating. Human researchers still need to check those interpretations against the original accounts rather than accept fluent prose as evidence.