Median-based causal forests offer selective protection against outliers

Causal forests aim to estimate how an intervention’s average effect varies across people with different characteristics. But unusual outcomes can influence how these models divide their data, potentially distorting the patterns they uncover. An arXiv preprint by Karolina Gliszczyńska-Schroeder and Jens Klenke, both at the University of Duisburg-Essen, examines whether median-based splitting rules offer a practical defense.

Their answer is conditional: some rules reduce sensitivity to extreme outcomes, but none consistently outperforms the standard approach. Greater stability also does not necessarily mean greater accuracy.

Changing the splits, not the target

A causal tree divides observations into groups, called leaves, using characteristics such as age or education. Within each leaf, it estimates a treatment effect by comparing outcomes among treated and untreated observations. A causal forest averages estimates from many such trees.

The study uses an “honest” construction: separate observations determine the splits and estimate the effects within the resulting leaves. That separation helps reduce bias from reusing data, but does not by itself prevent extreme outcomes from influencing the model.

The researchers compare a standard mean-based causal-tree splitting rule with three alternatives: Median Squared Deviation, or MSD; Median Absolute Deviation, or MAD; and Least Median of Squares, or LMS.

MSD and MAD use the Hodges–Lehmann estimator—the median of all pairwise treated-minus-control outcome differences within a leaf. MSD rewards larger effects according to this estimator while penalizing disagreement with the mean-based estimate. MAD favors partitions where the two estimates agree. LMS instead evaluates outcome fit using median squared residuals rather than their average.

Crucially, these changes affect split selection only. Final leaf estimates remain differences in mean outcomes, and the target remains the conditional average treatment effect, or CATE—not a median treatment effect. For MSD, the implementation uses a computationally simpler approximation rather than the full variance-corrected criterion.

Stress-testing extreme outcomes

The simulation study covers four designs: linear treatment effects, sharply nonlinear effects, no treatment effect, and nonlinear effects with interactions between characteristics. Each uses 10 covariates and 1,000 Monte Carlo repetitions, with 1,000 training observations and 1,000 independent test observations per repetition.

The authors test normal, Student-t, Laplace and Cauchy outcome noise. Separately, they contaminate up to 35% of training outcomes with either shifted noise or additional high-variance noise, while leaving test data uncontaminated.

Moderately heavy-tailed Student-t and Laplace errors do not produce systematic advantages for median-based splitting. The standard rule performs best in the linear design and in the interaction-rich design under those distributions.

Some clearer differences appear under Cauchy noise. In the sharply nonlinear design, MSD records a root mean squared error of 10.55, compared with 11.36 for the standard rule. However, variability across simulation runs is large, and the standard approach still performs best in the linear design.

There is an important mathematical qualification: under Cauchy errors, the conditional expectations defining the usual CATE do not exist. The researchers therefore evaluate estimates against the difference between the simulated outcome distributions’ location parameters. Those results concern recovery of a reference location difference, not an ordinary mean treatment effect.

Artificial contamination likewise produces mixed rankings. Some median-based rules deteriorate less relative to their own uncontaminated baselines. But a smaller percentage increase can reflect a worse starting point, rather than a lower final error.

Benchmarks test stability, not ground truth

The empirical analysis uses job-training data from the Lalonde benchmark, a breast-cancer dataset and a semi-synthetic education benchmark from ACIC. Results are averaged over 100 training/test splits.

The standard rule generally remains competitive or more stable in Lalonde and ACIC. MSD often shows lower sensitivity in the breast-cancer benchmark, especially at low or moderate contamination levels.

Because true conditional effects are unavailable in these applications, the comparisons use a noisy transformed-outcome proxy and changes from estimates fitted to the original data. Stability relative to an original estimate does not establish causal accuracy.

The breast-cancer exercise is explicitly not evidence about tamoxifen’s effectiveness. It retains only 299 patients with observed event times, excluding censored observations; that restriction does not resolve potentially informative censoring, and the analysis treats treatment assignment as observational.

Uncertainty remains a problem

The study also compares confidence intervals from the Bootstrap of Little Bags and infinitesimal jackknife. Neither is consistently well calibrated. In one linear design with normal errors, nominal 95% intervals for the standard rule cover the target only about 6% and 9% of the time, respectively. The unadjusted jackknife performs much better in the zero-effect design.

The preprint’s central limitation is structural: robust split selection leaves mean-based final estimates exposed to outliers. It also does not remove causal identification requirements, including adequate treatment overlap and no unmeasured confounding. The findings support context-specific testing, not a universal replacement for standard causal forests.