Do language models consistently rank moral trade-offs?

What would a consistent moral ranking require?

When moral values conflict, an answer can sound reasonable without revealing a consistent way of weighing those values. Katelyn A. Wasson and colleagues distinguish moral performance, producing morally appropriate outputs, from moral competence, responding to the morally relevant considerations in a decision.

Their study, reported in an arXiv preprint, examines one proposed requirement for competence: whether a model’s choices can be represented by a ranking. It does not ask which moral ranking is correct.

The central concept is transitivity. If one violation is judged worse than a second, and the second worse than a third, a transitive ranking also places the first above the third. A cycle of contradictory comparisons cannot form that ranking. An ordinal ranking records relative positions, not how much worse one violation is than another.

How the models were tested

The researchers tested Llama 4 Maverick, GPT-5.2, Gemini 2.5 Flash, DeepSeek V3.2 and Grok 4.1 Fast. All were used without reasoning enabled.

The task used short scenarios associated with seven principles drawn from Moral Foundations Theory: authority, emotional care, physical care, fairness, liberty, loyalty and sanctity. Models received two scenarios at a time and were asked which situation was morally worse.

Each set contained all 21 pairwise comparisons among the seven principles. The researchers used six blocks of scenarios, grouped by decreasing severity according to earlier human ratings. They also reversed the presentation order of every pair. That produced 12 choice sets per model and 60 overall.

Rather than repeatedly generating answers and counting them, the researchers extracted the probabilities assigned to the first response tokens, “1” and “2.” They normalized these probabilities against each other and treated them as measures of choice.

Three mathematical models tested different forms of consistency. The weak utility model, or WUM, required every comparison to favor the same ranking with probability at least 0.5. The high certainty model, or HCM, raised that threshold to 0.9. The general random utility model, or G-RUM, tested whether the probabilities could arise from a mixture of transitive rankings.

WUM and G-RUM impose different requirements; passing one does not necessarily mean passing the other.

Weak rankings and sensitivity to order

Across the 60 choice sets, 36 satisfied WUM, eight satisfied HCM and 13 satisfied G-RUM. Thus, most sets supported a ranking under the weakest test, but relatively few supported consistently decisive choices or a mixture of transitive rankings.

GPT-5.2 satisfied WUM in 10 of its 12 sets and HCM in six. DeepSeek satisfied WUM in nine sets but HCM in none. No model satisfied any one of the three tests across all 12 sets.

Presentation order mattered. Across the 30 model-by-severity comparisons, reversing the same scenario pairs changed the resulting ranking or pattern of transitivity violations in 26 cases. Changes between different severity blocks could reflect meaningful differences in content. Reversing the same alternatives, however, changed no morally relevant content.

Averaging increased apparent consistency—with an important exception

The researchers then averaged each model’s probabilities across severity blocks and presentation orders. Four of the five models now satisfied both WUM and G-RUM. Their aggregate rankings placed physical-care violations first and loyalty violations last. None satisfied HCM after aggregation.

Gemini illustrated how pooling could conceal differences between conditions: its averaged probabilities satisfied G-RUM, although none of its individual sets did. The authors describe the apparent consistency emerging through averaging as an aggregation artifact.

However, the paper’s abstract and narrative overstate one aspect of this result. They say aggregate rankings matched no individual choice set, but Table 1 and Extended Data Table 2 show an exception. GPT-5.2’s aggregate ranking—physical care, sanctity, liberty, fairness, emotional care, authority, then loyalty—also appears in both presentation orders of severity block 4.

That agreement concerns the ordinal ranking, not identical choice probabilities or identical results on every test. Both block-4 sets satisfied WUM and HCM, but only the original presentation order satisfied G-RUM; the aggregate satisfied WUM and G-RUM but not HCM.

The tables therefore contradict the blanket claim that aggregate rankings never matched individual rankings. The narrower finding remains: pooled results can look more consistently structured than results across separate conditions, and an aggregate ranking should not be treated as evidence that a model applies that ranking throughout.

What the findings cannot establish

First-token probabilities are not necessarily equivalent to considered answers. The study did not test whether allowing models to reason before answering would improve transitivity. Its selection also excluded systems that did not expose the required token probabilities.

The scenario labels and severity ratings came from human research. The authors did not establish that the models interpreted them in the same way. Severity also differed within blocks, so a principle’s apparent priority could reflect the particular scenario’s severity rather than the principle alone.

Finally, passing a transitivity test would not prove moral understanding: a process without moral competence could still produce ordered choices. Within these limits, the preprint supports examining the stability of judgments across conditions before treating their average as evidence of moral alignment.