On the statistical differences between binary forecasts and real-world payoffs

We map the difference between (univariate) binary predictions, bets and “beliefs” (expressed as a specific “event” will happen/will not happen) and real-world continuous payoffs (numerical benefits/harm from an event) and show the effect of their conflation and mischaracterization in the decision-science literature. We also examine the differences under thin and fat tails. The effects: [A] Spuriousness of many psychological results, particularly those documenting that humans overestimate tail probabilities. We quantify such conflations. [B] Being a “good forecaster” in binary space doesn’t lead to having a good actual performance, and vice versa, especially under nonlinearities. A binary forecasting record is likely to be a reverse indicator under some classes of distributions or deeper uncertainty. [C] Machine Learning: Some nonlinear payoff functions, while not lending themselves to verbalistic expressions, are well captured by ML or expressed in option contracts. Fattailedness: The difference is exacerbated in the power law classes of probability distributions.