Can a chatbot’s reply reveal the gender cue in its prompt?

Would a chatbot answer the same request differently if the person asking introduced themselves as female rather than male? The difference might not be an obvious insult or an occupational stereotype. It could appear in how much a reply stresses protection, independence or capability.

In the November 2026 preprint GRAP: Gender Rubric-Aware Probing for Analyzing Gender Bias in LLMs, Seona Seo and colleagues at Ewha Womans University investigate those subtler patterns. Their framework turns open-ended responses into a small set of social scores, then asks whether a simple classifier can recover the gender condition behind each reply. This article draws on the extracted paper text; figures were unavailable in that extraction.

From replies to ten scores

GRAP stands for Gender Rubric-Aware Probing. The researchers construct 1,000 paired prompts. Within each pair, the instruction stays the same, while the introductory name and explicit female or male label change. Each prompt is submitted to a target language model.

Another model, gpt-4o-mini, acts as a judge. It scores each response from zero to one on ten rubric dimensions intended to capture how the answering model implicitly treats the questioner. The results discussion names dimensions involving communal care, warmth or respect, protection-oriented framing, agency, competence and capability attribution. The extracted text does not provide the complete rubric definitions.

A logistic probe then learns to distinguish responses generated under the female condition from those generated under the male condition. This is a small classifier operating on the ten scores, not on the full response text. It assigns each dimension a weight and combines the weighted scores into a predicted probability.

Positive weights point toward the female-labelled condition; negative weights point toward the male-labelled condition. These labels describe the experimental prompts—not the actual gender of a person inferred from their writing.

What the researchers report

The study evaluates gpt-5.4-mini, claude-sonnet-4.5, gemini-2.5-flash and llama-3.3-70b. The authors also say gpt-5.4-mini generated prompt-response pairs, although the short methods account leaves its precise role in that construction unclear.

The reported weights associate female-conditioned responses with communal care, warmth or respect, and protection. Male-conditioned responses are associated with agency, competence and capability. These are patterns in model-generated replies as interpreted by an automated judge, not findings about women’s or men’s characteristics.

The researchers compare two judging settings. In the full setting, the judge sees both the prompt and response. In the response-only setting, it sees just the response. They report that response-only classification matches or exceeds full-setting performance across the four models.

For gpt-5.4-mini, the reported AUC rises from 0.7946 to 0.8238; for claude-sonnet-4.5, it rises from 0.8428 to 0.8787. AUC measures how well a classifier ranks the two conditions: 0.5 corresponds to chance ranking and 1 to perfect separation. It is not the percentage of replies classified correctly, nor a measure of how harmful a response is.

Detectability is not a fairness verdict

The response-only result suggests that distinguishable information appears in the replies themselves, rather than requiring the judge to read the original introduction. It does not establish that every explicit gender cue has disappeared: the extracted text does not describe checks for replies repeating names or gender labels.

The authors present the probe as a way to identify discriminative dimensions when raw average-score differences are difficult to interpret because of variation. But a learned weight is not a causal explanation. Its size can depend on how rubric scores vary and overlap with one another; it should not automatically be read as the severity of a particular bias.

Other uncertainties remain. The extracted account does not specify train-test splitting, confidence intervals, human validation of the judge, or robustness across alternative judges. Because paired prompts change both names and gender labels, their effects are not separately isolated. The binary setup also does not examine nonbinary identities.

GRAP therefore offers a compact diagnostic: under these experimental conditions, a small set of automated social scores can distinguish replies to differently gender-labelled prompts. Establishing whether those differences are inappropriate, reliable across contexts or consequential for users requires evidence beyond classification performance.