Why two methods can disagree about an economic shock

An unexpected increase in government spending might have different consequences in a weak economy than in a strong one. But estimating that difference presents a problem: two widely used statistical approaches can disagree even with arbitrarily large datasets.

In a preprint, Columbia University economist Valentin Winkler explains why this happens and proposes a modified estimator. The central distinction is between predicting an outcome directly from a shock and building that prediction through successive steps that change how they condition on the economy’s state.

Direct estimates versus successive forecasts

A local projection, or LP, estimates the relationship between a shock today and an outcome at a particular future horizon. Researchers repeat the regression for different horizons to trace an impulse response: how the outcome changes over time following the shock.

A vector autoregression, or VAR, instead models several variables using their past values. Researchers iterate these relationships forward to construct a response.

Without state dependence, established results connect the quantities these methods estimate in large samples. State dependence complicates that connection. An LP can ask about the response given unemployment before the shock. A conventional state-dependent VAR uses relationships associated with successively later states as it forecasts forward.

Those are not generally interchangeable operations.

Accounting for transitions is not enough

One familiar problem is that some VAR calculations hold the economy in its starting state throughout the response. An economy experiencing high unemployment today may not remain there for years.

Winkler shows that allowing future state changes does not necessarily resolve the disagreement. In a simplified model, the shock’s initial impact and its subsequent persistence both depend on the state. Averaging those components separately and then multiplying them can give a different answer from combining them along each possible state path and averaging afterwards.

The missing ingredient is their co-movement. A state associated with a large initial impact might also be associated with rapid decay. Separating impact from propagation loses that connection.

In the paper’s illustrative example, conventional VAR responses can even fall outside the range of effects possible along any state path. This is a counterexample to a general causal interpretation, not a claim that every state-dependent VAR fails. The broader analysis also identifies another source of disagreement: prediction errors that are unpredictable under one conditioning state can become predictable under another.

Keep the starting information fixed

The proposed solution is a “backshifted VAR.” It estimates a sequence of VARs with the state variable shifted progressively further into the past. When those models are used for successive forecast steps, each step conditions on the same pre-shock state.

This does not assume that the economy physically remains in that state. It keeps the conditioning information fixed while allowing subsequent economic developments to enter the estimated relationships.

Under the paper’s assumptions, the resulting estimator converges to the same target as the state-dependent LP at horizons up to the VAR’s lag order. With four lags, for example, that guarantee extends through four periods. Beyond that, truncating the history can introduce bias. The method does not require a model of state-transition probabilities.

What the causal guarantee means

A separate theoretical result clarifies the LP’s target. Under strong independence and regularity assumptions, it estimates a projection of a positively weighted average of marginal causal effects onto the chosen functions of the state. Marginal effects describe responses to small changes in the shock.

For a binary state, the method recovers the corresponding weighted average effect in each group. For a continuous state, a simple linear interaction instead summarizes how that effect co-varies with the state. If the true relationship bends, a fitted line need not recover the effect at a particular state value—especially an extreme one.

The assumptions matter. The shock must be independent of the pre-shock state and the other determinants of the outcome, not merely uncorrelated with them. State-dependent shock volatility can violate this requirement. A marginal-effect average also need not answer a policymaker’s question about a large intervention.

Precision comes with a cost

Winkler compares estimators in simulations calibrated to macroeconomic data. The backshifted VAR has lower variability than LPs, but can acquire greater bias at longer horizons. LP confidence intervals provide more reliable coverage in these experiments; the new method is not an across-the-board improvement.

An application revisits US military-spending news shocks using quarterly data from 1890 to 2015. The backshifted estimates closely follow LP responses initially and subsequently resemble smoother versions of them. Conventional VAR alternatives can diverge substantially.

The evidence supports a methodological distinction rather than a new verdict on fiscal stimulus. Broader simulations and improved inference remain open questions. The practical lesson is to establish what an estimator targets before interpreting differences between its response curves as differences in economic mechanisms.