What Economic Theories Predict Before They See the Data

Beyond a good fit

A theory with adjustable parameters can accommodate many patterns of behavior. Testing its predictions on held-out observations helps guard against overfitting, but still evaluates a version selected using training data. It does not, by itself, establish how well the theory would predict using only its previously stated commitments.

In the preprint Comparing the Empirical Content of Economic Theories, Johanna Einsiedler proposes evaluating those commitments explicitly. Instead of choosing the best-fitting parameter values, the framework also scores predictions drawn from a distribution specified before evaluating the sample. That distribution represents which versions of a theory researchers consider plausible.

The distinction is between asking what a mathematical family can fit and asking what a particular, previously specified version of that theory predicts.

Two dimensions of empirical content

The framework represents a theory through three ingredients: qualitative restrictions on behavior, a functional form connecting situations to predictions, and a distribution over its parameters. For example, a theory of risky choice might require that a better lottery receive a higher valuation, while assigning particular plausibility to different degrees of risk sensitivity.

Probabilistic completeness measures the share of achievable predictive improvement captured by a rule drawn from the theory. Zero corresponds to a baseline prediction requiring no behavioral theory; one corresponds to the best achievable predictor. A negative score means the theory performs worse than the baseline.

Probabilistic restrictiveness measures how strongly the theory limits conceivable behavior. One component concerns the predictions its functional form can reach. Another concerns how tightly its parameter distribution concentrates predictions within that reachable set. Restrictiveness is not accuracy: a theory can make narrow predictions that are wrong.

The paper keeps these dimensions separate rather than collapsing them into one overall score. Its combined restrictiveness measure uses a particular aggregation convention, and some comparisons depend on that choice.

A test using lottery valuations

The main application uses an existing experiment involving 179 subjects in Zurich in 2003. Participants reported certainty equivalents for 25 binary lotteries over gains: the guaranteed sums they would accept instead of taking each gamble.

The comparison includes cumulative prospect theory, disappointment aversion, expected utility and the parameter-free expected-value rule. The baseline is each lottery’s prize-range midpoint, not its expected payout. A lookup table of average responses estimates the predictive ceiling. Fitted models are evaluated using ten-fold cross-validation with subjects held out.

For an average-person interpretation, fitted cumulative prospect theory reaches completeness of 0.972. Drawing parameters from its literature-informed prior yields 0.641—about 66% of that fitted score. Expected utility retains about 85%, scoring 0.521 compared with 0.616 when fitted. Disappointment aversion retains about 3%, scoring 0.020 compared with 0.626; its prior-based estimate is imprecise and close to the baseline.

Uniform parameter priors perform worse than the midpoint baseline for all three families. These results illustrate why specifying a functional form alone is different from specifying plausible parameter values. Under the paper’s combined measure, literature-informed cumulative prospect theory is approximately as restrictive as disappointment aversion but predicts substantially better on these gain lotteries.

Predicting diversity, not just an average

A parameter distribution can instead describe differences between people. In that interpretation, predictive spread is useful when it matches variation among respondents, rather than automatically counting against the theory.

The paper therefore uses a distributional scoring rule. Cumulative prospect theory achieves completeness around 0.70 with either a uniform parameter prior or a mixture based on participant types from two other samples. Its fitted parameter distribution reaches 0.890.

Expected utility and disappointment aversion perform less well. In the main specification, their predicted populations cannot reproduce the observed dispersion sufficiently well. However, that specification adds no response noise. The paper reports that adding noise improves scores, while preserving cumulative prospect theory’s lead. The population-level scoring also does not evaluate dependencies between one person’s responses across lotteries.

Scope changes the verdict

Extending the comparison to 25 loss lotteries from the same subjects exposes an important boundary. Under the average-person interpretation, every loss-specific literature prior performs worse than the midpoint baseline. With gains and losses weighted equally, expected value outperforms literature-informed cumulative prospect theory on both aggregated coordinates. Yet the heterogeneous-population version of prospect theory’s prior remains positively predictive on losses.

These are predictive comparisons, not evidence about what causes risk-taking. Their interpretation depends on the baselines, parameter distributions, admissible outcomes and domain weights. The prospect-theory meta-analysis used for calibration also includes the evaluated study, so the prior is not wholly independent. Moreover, the supplied preprint omits appendices containing calibration and robustness details. The contribution is a proposed framework with an illustrative application—not a universal ranking of economic theories.