Separating job training’s earnings gains from changes in time spent working

What the method separates

Difference-in-differences compares changes over time between treated and untreated groups. Its central idea is that, without treatment, the groups would have followed comparable outcome trends, after accounting for relevant baseline characteristics. This can accommodate some persistent, unmeasured differences between groups, rather than requiring their outcome levels to match.

The authors extend this approach to a mediator: something treatment may change that, in turn, affects the outcome. In their application, training is the treatment, the proportion of weeks employed is the mediator, and earnings are the outcome.

They target effects among people who actually received treatment, not the entire population. Their “natural direct effect” compares training with no training while holding employment intensity at the level each treated person would have under training. Their “natural indirect effect” changes employment intensity from its no-training level to its training level while keeping treatment itself absent. These two quantities add to the total effect.

“Direct” therefore means outside the specified employment-intensity pathway. It does not identify a particular alternative mechanism, such as better skills or higher hourly wages.

How they estimate it

The framework models changes in outcomes, treatment probabilities and mediator-related quantities, combining prediction with statistical weighting. The authors derive efficient influence functions: mathematical tools used to construct estimators and quantify uncertainty.

Their estimators are “multiply robust,” meaning that particular combinations of correctly estimated supporting models can suffice even when other models are wrong. Under additional conditions, the estimators reach a theoretical precision benchmark. Neither property protects against failure of the underlying causal assumptions.

Simulations examine continuous and binary mediators, including settings with incorrectly specified models. The proposed approach generally shows smaller bias than the paper’s simple additive-regression benchmark, which omits relevant interactions. Performance is less reassuring in small samples: confidence intervals sometimes cover the true effects less often than intended. These constructed examples demonstrate performance in selected settings, not universal reliability.

What the Job Corps analysis finds

Although the original Job Corps assignment was randomized, actual enrollment was not: participants could choose whether to undertake education or vocational training. The analysis treats first-year enrollment as treatment and examines earnings and employment intensity in years two and three, adjusting for baseline characteristics.

Earnings are transformed using log(1 + earnings), allowing zero earnings to remain in the analysis. The reported effects are on this transformed scale—not dollar gains or percentage increases.

For year two, the estimated total effect is 0.070, with a 95% confidence interval from −0.054 to 0.193. The direct effect is positive at 0.105, with an interval from 0.015 to 0.194. The indirect estimate is −0.035 and is not statistically significant. Thus, the estimates suggest offsetting pathways, but do not establish a negative employment-mediated effect.

For year three, the total estimate is 0.283, with an interval from 0.163 to 0.402. The indirect estimate is 0.191 and the direct estimate 0.091; both reported intervals exclude zero. The authors attribute roughly two-thirds of the estimated total effect to employment intensity. Alternative specifications broadly preserve the main pattern, although the longer-run direct effect is not statistically significant with the alternative earnings-per-employed-week outcome.

Why the interpretation remains conditional

The key trend assumption compares people according to employment intensity they would have experienced without training—a quantity unavailable for treated participants. Other assumptions connect mutually exclusive hypothetical scenarios and require no unmeasured confounding between treatment and potential employment intensity after baseline adjustment. Treatment-induced confounders are excluded.

These requirements cannot be fully checked from observed data. The application has only one baseline period, so baseline balance cannot establish parallel pre-treatment trends.

Employment intensity is also mathematically embedded in average earnings: people who never work generally have zero earnings. Some apparent mediation therefore reflects outcome construction rather than a separately demonstrated economic mechanism.

There is a further caution in the controlled-effect analysis. The paper states that fixing employment at zero makes the direct effect zero by construction, yet reports a nonzero estimate there. That discrepancy warrants caution rather than a substantive interpretation.

This is an arXiv preprint; the supplied material does not establish peer-review status. Its contribution is a proposed method and a conditional empirical interpretation, not definitive proof of how training changes earnings.