When Dropping Incomplete Records Can Preserve a Causal Estimate

Deleting every record with a missing value is straightforward. Knowing what the resulting regression estimates is harder. Wenxin Lin and Shiyao Liu argue that researchers should put missingness directly into their causal diagrams, then examine both the validity of the remaining comparisons and how those comparisons are averaged.

Missing data create a choice: fill in unavailable values using an imputation model, or restrict analysis to records containing everything the model needs. The second approach, called listwise deletion or complete-case analysis, avoids specifying an imputation model but discards information and can introduce selection bias.

In a November 2026 preprint, Wenxin Lin and Shiyao Liu develop a framework for assessing when regression after deletion still consistently estimates the population average treatment effect—the average difference treatment would make across the target population. Their argument is not that deletion generally beats imputation. It is that either choice needs explicit assumptions.

Start with a regression that would work

The authors take an important starting condition as given: the chosen regression would consistently estimate the population average treatment effect if every required value were observed. They call this hypothetical complete dataset the “oracle” sample.

Their question is narrower than whether the original research design establishes causality. It asks whether applying the same regression only to complete records preserves its causal target.

For constant treatment effects, they propose two checks. First, do treated and untreated complete cases remain causally comparable within groups sharing the same covariates? Second, does the estimator combine those comparisons appropriately?

Put missingness into the causal diagram

The first check adds a missingness indicator and its causes to the researcher’s causal diagram. Restricting analysis to complete cases amounts to conditioning on that indicator, which can create associations that were absent before selection.

For example, if treatment and an unobserved cause of the outcome both influence whether a record is complete, selecting complete records can make treatment associated with that hidden cause. Adjusting for the original observed covariates may not remove the resulting distortion.

The paper applies an established graphical adjustment criterion to assess whether the covariates together with the missingness indicator support valid treatment comparisons. Both treatment groups must also remain possible within the relevant covariate groups. These are sufficient conditions under the assumed causal model, not a guarantee obtainable from the observed data alone.

Valid comparisons can still be pooled badly

Even when comparisons within covariate groups remain valid, an ordinary regression can combine them incorrectly because selection changes the outcome relationship.

The authors illustrate this with simulations having a true treatment effect of one. All three main missingness mechanisms remove, on average, 25% of observations and are classified as not missing at random. Yet their consequences differ.

Under the first mechanism, ordinary complete-case regression remains centered near the true effect. Under the second, the average estimate rises to 1.165: selection introduces curvature in the baseline outcome relationship that a linear covariate term cannot capture. Allowing separate baseline means for each covariate category, or applying inverse treatment-propensity weights, brings estimates back near one.

Those weights use treatment probabilities among complete cases to balance the covariate distributions of treated and untreated records. They address the aggregation problem without requiring the baseline outcome mean to be linear. But under the third mechanism, selection invalidates the comparisons themselves; neither adjustment approach recovers the true effect.

An appendix produces a similar progression under missing-at-random mechanisms. The lesson is not that missingness classifications are useless: they remain relevant to imputation assumptions. Rather, the label alone does not establish whether complete-case regression is consistent.

Whose average is being estimated?

When treatment effects vary, recovering an effect among complete cases is not enough. The authors additionally require that average effects within covariate groups carry over to the population, and that complete cases cover the population’s covariate support. Conditional effects must then be averaged using the population’s covariate distribution.

A reanalysis of a voter-canvassing experiment illustrates the distinction. College-educated voters represented 53.16% of respondents retained as complete cases but 47.00% of all canvassed voters. Averaging estimated education-specific effects using respondent shares produced 9.56 hundredths of a standard deviation; using full-sample shares produced 9.31. The difference was not statistically significant, but the two averages target different quantities under the illustration’s assumptions.

Guidance with explicit limits

The paper’s review finds deletion stated or implied in roughly half of eligible observational causal articles sampled from two political science journals, versus about 7% describing an imputation model in their main text. These are reconstructed-sample findings: coding used language models, and independent human double-coding had not been completed.

The framework also leaves general estimation and inference for heterogeneous effects with continuous covariates undeveloped. Its practical recommendation is nonetheless concrete: state assumed causes of missingness, reassess causal comparisons after selection, check how the estimator combines them, and distinguish the retained sample’s average effect from the population target.