From Predictive Algorithms to Automatic Generation of Anomalies

How can we extract theoretical insights from machine learning algorithms? We take a familiar lesson: researchers often turn their intuitions into theoretical insights by constructing "anomalies" -- specific examples highlighting hypothesized flaws in a theory, such as the Allais paradox and the Kahneman-Tversky choice experiments for expected utility. We develop procedures that replace researchers' intuitions with predictive algorithms: given a predictive algorithm and a theory, our procedures automatically generate anomalies for that theory. We illustrate our procedures with a concrete application: generating anomalies for expected utility theory. Based on a neural network that accurately predicts lottery choices, our procedures recover known anomalies for expected utility theory and discover new ones absent from existing work. In incentivized experiments, subjects violate expected utility theory on these algorithmically generated anomalies at rates similar to the Allais paradox and common ratio effect.