The benchmark behind an ‘unexplained’ pay gap

A difference in average wages is observable. Deciding how much of that difference is ‘explained’ requires an additional choice: what should people with the same characteristics receive in the comparison world?

In an arXiv preprint, Emmanuel Flachaire and Bertille Picard examine how that choice shapes inequality decompositions—and how to estimate them using machine learning. Their contribution is theoretical, supported by simulations and constructed examples, rather than a new measurement of a real-world pay gap.

Choose the question before the estimator

Kitagawa–Oaxaca–Blinder decompositions separate a difference in group averages into a component associated with different characteristics and an unexplained remainder. In a wage analysis, those characteristics might include education. The calculation compares observed wages with a hypothetical ‘reference outcome’ that values the same characteristics equally across groups.

But equal valuation does not specify a unique benchmark. Should everyone face the pay structure of the higher-paid group, the lower-paid group, or some combination?

The authors put several familiar proposals into one framework: mixtures of the two groups’ potential outcomes, meaning the outcomes an individual would receive under either group’s outcome structure. Reimers’s reference gives each structure equal weight. Cotton’s uses the groups’ overall population shares. A generalized reference inspired by Neumark instead lets the weights vary with a person’s characteristics.

For that last reference, the key quantity is the propensity score: the probability of belonging to one group given those characteristics. A profile concentrated in one group receives a benchmark closer to that group’s outcome structure.

These are different statistical targets, not merely competing ways to estimate the same number.

Why accurate prediction is not enough

The paper derives estimators using outcome predictions, probability-based reweighting, and combinations of both. Flexible methods such as boosting or random forests can estimate the required relationships without imposing a simple straight-line model.

However, useful predictions do not automatically produce reliable confidence intervals for the unexplained component. Prediction errors can enter the final calculation in ways that become consequential as uncertainty intervals narrow.

Double machine learning addresses this through two ingredients. Cross-fitting trains the prediction models on observations separate from those used for the final calculation, rotating through sample subsets. An orthogonal score—a specially constructed estimating formula—makes the target estimate insensitive, to first order, to small errors in those predictions.

The authors’ central result is that the appropriate formula depends on the reference. For the standard constant-weight references, their orthogonal approach uses group-specific outcome regressions. For the propensity-weighted Neumark reference, it uses a pooled regression of outcomes on characteristics, without the group indicator.

In simulations using gradient boosting, matching the formula to the reference matters for confidence-interval coverage. An inappropriate pairing can approach the correct value as the sample grows yet do so too slowly relative to the narrowing intervals. These simulations illustrate an inference problem, not evidence about actual wage discrimination.

When comparable people are scarce

Another difficulty is limited overlap: some characteristic profiles occur almost exclusively in one group.

Group-specific regressions may then predict outcomes where that group supplies very few observations. Reweighting methods can instead assign enormous influence to rare profiles because their formulas divide by probabilities close to zero.

The generalized Neumark reference avoids those particular problems. Its relevant probability denominators cancel, and the benchmark approaches the locally dominant group’s outcome structure where counterparts are scarce. This stability comes from choosing a different comparison outcome, not from recovering otherwise unavailable information about the other group.

There is a corresponding interpretive trade-off. When the conditional gap between outcome structures is constant across profiles, the authors show that Neumark’s unexplained component is shrunk toward zero relative to the standard references. Shrinkage strengthens as characteristics more clearly separate the groups.

When gaps vary across profiles, that ranking no longer holds. Constructed counterexamples show that Neumark’s component can exceed both single-group benchmarks in magnitude. The paper therefore rejects a blanket claim that this reference always produces the smallest unexplained gap.

What the calculation cannot decide

The counterfactual identification relies on ignorability: conditional on the included characteristics, group membership supplies no additional information about potential outcomes. Conventional comparisons also require suitable overlap, while machine-learning inference needs adequate estimation rates and regularity conditions. The unexplained remainder is therefore not an assumption-free measure of discrimination.

The authors particularly warn against adding variables merely because they predict group membership when those variables are irrelevant to the reference outcome. Under Neumark’s reference, that choice can alter the target through shrinkage.

Their broader conclusion is that statistical convenience cannot settle the benchmark. Different references embody different economic assumptions about wage setting and market adjustment. Machine learning can help estimate a chosen comparison, but it cannot choose the appropriate comparison world.