When Missing Data Make Regression Results Look Too Certain

A regression can look precise even when its inputs are not. Household surveys report values in brackets, administrative records cap large observations, and participants disappear from follow-up studies. Replacing an interval with its midpoint or discarding incomplete records can simplify estimation, but the resulting certainty may depend on assumptions that the data cannot establish.

In their preprint, Martin Fankhauser and Gil Peled develop a framework for determining which regression parameters remain compatible with incomplete observations. Their goal is not to reconstruct the missing values. It is to describe the range of conclusions that the observed data and a maintained regression model can support.

What the data actually identify

The framework treats each incompletely observed variable as lying within a reported interval. An exact observation is an interval with identical endpoints; a completely missing value can be represented by the variable’s full assumed range.

Crucially, the authors do not require missingness to be random. Whether an observation is incomplete, and the location or width of its interval, may depend on the underlying unobserved values.

That does not make the method assumption-free. The reported intervals must contain the true values, the regression’s conditional restrictions must hold, and the theoretical results require conditions including bounded variable domains and regularity of the model’s moment functions.

The central object is the sharp identified set: every parameter value consistent with at least one possible completion of the data that respects those intervals and satisfies the regression restrictions. “Sharp” means that no compatible parameter is excluded and no incompatible parameter is included. It does not mean that the set is necessarily narrow.

The distinction from looser approaches concerns consistency across restrictions. Checking each restriction separately may allow a different completion of the missing data for every check. The sharp set instead requires one completion to satisfy them all together.

Turning compatibility into computation

The authors translate the conditional regression restrictions into infinitely many unconditional restrictions, then use random-set geometry and support functions to characterize which parameters survive them.

Under their stated conditions, they establish consistency of a set estimator and asymptotically valid confidence regions for individual compatible parameter values, uniformly over the relevant class of data distributions. This is not a guarantee that a confidence region simultaneously covers the entire identified set.

Computation requires finite approximations. Retaining only some restrictions produces an outer set that contains the sharp population set. For scalar monotone-index models, including standard linear, probit and logit specifications, the finite unstudentized estimation criterion has an exact linear-programming representation. The confidence procedure involves a separate, variance-adjusted statistic; the linear program does not directly solve that inference problem.

Their practical implementations either use prespecified directions with the full sample or select informative directions on one subsample and test them on another, then reverse the roles and intersect the confidence sets with an adjustment for the two tests.

A graduation finding becomes less conclusive

The empirical illustration revisits research linking children’s discount rates to on-time high-school graduation. Participants chose between an earlier payment and larger later payments. Because the choices used a discrete grid, even consistent responses revealed a bracket for the discount rate rather than an exact value.

In the specification without additional controls, conventional midpoint-based probit estimation gives a marginal effect of −0.086, with a 95% confidence interval of [−0.152, −0.020]. That interval excludes zero.

Allowing discount rates to vary within their survey-implied intervals gives an adaptive 95% confidence set of [−0.226, 0.020], which includes zero. Importantly, the change is not attributable entirely to interval measurement: applying the authors’ projection-based inference procedure to exact midpoint values already produces a wider set, [−0.194, 0.000]. The interval-data set is approximately 27% wider than that comparable midpoint benchmark.

The result makes the association inconclusive under these assumptions; it does not establish that the association is absent, or identify a causal effect. Allowing inconsistent switching responses and missing graduation outcomes broadens the sets further.

Nevertheless, the basic interval-data result remains substantially tighter than the paper’s implementation of the Aucejo–Bugni–Hotz comparison procedure, extended to interval data. That procedure reports [−0.396, 0.403], but its confidence set reaches the parameter-grid boundary. The reported interval is therefore truncated, not a determination of the confidence set’s unrestricted endpoints.

Robustness has a price

The simulations illustrate the trade-off. Conventional methods can lose coverage when interval observation depends on outcomes, while the proposed procedures retain coverage in the reported designs. When complete-case analysis is valid, however, it can be more powerful.

The application with additional controls also has a specific limitation: estimated control coefficients and propensity weights are held fixed, so its reported confidence sets omit first-stage uncertainty and are not covered by the formal guarantee.

The preprint’s contribution is therefore a disciplined benchmark, not a universal replacement for imputation. Stronger assumptions can deliver tighter answers when credible. This framework makes explicit how much remains supportable before those assumptions are added.