How combining outcomes can change an economic study’s conclusion

Economists often want to measure concepts they cannot observe directly, such as wealth, entrepreneurial confidence or involvement in crime. They collect several related indicators and combine them into an index. But how much weight should each indicator receive?

In the preprint Aggregating treatment effects across multiple outcomes, posted on November 3, 2026, Chun Pong Lau of the University of South Carolina examines three widely used answers. His central finding is that a method designed to summarize outcomes need not produce a precise or easily interpreted estimate of a program’s effect.

When a negative weight changes the story

Lau studies principal component analysis, or PCA; inverse variance matrix weighting, or IVM; and equally weighted standardized averaging, or SA. His survey identifies 30 applications of PCA, 23 of IVM and 33 of SA to aggregate outcomes in five leading economics journals during 2020–2024. These are method-use counts, not necessarily distinct articles.

PCA finds a combination that captures the greatest variation in the observed indicators. That is different from minimizing uncertainty about an estimated treatment effect, Lau explains. In a particular theoretical case with no treatment effects and equal error variances across treatment groups, PCA actually selects the least precise combination within its specified class of weights.

PCA can also assign negative weights. Consider a wealth index containing cows and goats. If households substitute one animal for the other, ownership can be negatively correlated. PCA may then give one animal a negative weight: holding everything else constant, owning more of it would lower measured wealth.

The overall sign of a PCA component also requires a normalization convention. Lau shows that different conventions can reverse the reported effect when weights have mixed signs. Near cases where the sign is weakly determined, conventional statistical inference can fail.

IVM likewise permits negative weights. It uses the covariance of outcomes, which need not match the covariance of their estimated treatment effects. Equal standardized averaging avoids negative weights, but does not use correlations to distinguish additional information from closely overlapping indicators. Neither method necessarily delivers maximum precision.

A different balance of precision and meaning

Lau’s alternative requires weights to be nonnegative and sum to one. The combined estimated effect therefore stays between the smallest and largest individual estimated effects, avoiding sign reversals caused by negative weights.

He chooses weights using mean-squared error, which combines sampling variability with squared bias. Here, bias includes the gap between an indicator’s treatment effect and the effect on the underlying concept. A precisely estimated effect on goat ownership, for example, need not perfectly represent an effect on wealth.

One option, the minimax approach, minimizes the worst-case error within a researcher-specified set of possible discrepancies. Another, the adaptive approach, chooses weights to limit the proportional cost of not knowing which discrepancy bound is appropriate. It works across a chosen range rather than requiring one bound.

Both approaches can incorporate assumptions about which indicators better capture the concept. Lau also develops confidence intervals that allow for potential bias, rather than treating the combined estimate as automatically unbiased.

Revisiting cash transfers in Ghana

Lau illustrates the stakes by revisiting a randomized cash-transfer study by Dean Karlan and coauthors involving low-income households in Ghana during the COVID-19 pandemic. His reanalysis focuses on two follow-up surveys and six indicators used to construct a social-distancing index.

The original PCA-based result was positive and statistically significant at the 1% level. Lau finds that the outcome with the largest negative treatment effect—mask wearing—also received the most negative PCA weight, helping push the combined estimate upward.

Without restrictions on relative indicator quality, his proposed methods produce smaller positive estimates that are not statistically significant at the 10% level. This does not establish that the transfers had no effect; it means those specifications do not clearly distinguish the estimated effect from zero.

A positive result significant at 5% can emerge under the minimax approach when mask wearing and days with visitors are heavily downweighted and assumed to be less faithful measures than the other four outcomes. The adaptive specifications shown still have confidence intervals containing zero.

Assumptions remain essential

The proposal does not reveal an unobserved concept without assumptions. Outcomes must be consistently oriented and suitably standardized. The statistical framework relies on normal approximations and a well-behaved covariance matrix; its error guarantees depend on the specified bias restrictions.

As a preprint, the paper presents a methodological proposal, not a universal verdict on cash transfers or index construction. Its practical contribution is to show which conclusions survive alternative weights—and which require stronger beliefs about what the indicators measure.