IDEAS home Printed from https://ideas.repec.org/p/pen/papers/18-010.html

The Theory is Predictive, but is it Complete? An Application to Human Perception of Randomness

Author

Listed:
  • Jon Kleinberg

    (Department of Computer Science, Cornell University)

  • Annie Liang

    (Department of Economics, University of Pennsylvania)

  • Sendhil Mullainathan

    (Department of Economics, Harvard University)

Abstract

When testing a theory, we should ask not just whether its predictions match what we see in the data, but also about its “completeness†: how much of the predictable variation in the data does the theory capture? Deï¬ ning completeness is conceptually challenging, but we show how methods based on machine learning can provide tractable measures of completeness. We also identify a model domain—the human perception and generation of randomness—where measures of completeness can be feasibly analyzed; from these measures we discover there is signiï¬ cant structure in the problem that existing theories have yet to capture.

Suggested Citation

  • Jon Kleinberg & Annie Liang & Sendhil Mullainathan, 2017. "The Theory is Predictive, but is it Complete? An Application to Human Perception of Randomness," PIER Working Paper Archive 18-010, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 09 Aug 2017.
    • Handle: RePEc:pen:papers:18-010
    as

    Download full text from publisher

    File URL: https://economics.sas.upenn.edu/system/files/working-papers/18-010%20PIER%20Paper%20Submission.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Peysakhovich, Alexander & Naecker, Jeffrey, 2017. "Using methods from machine learning to evaluate behavioral models of choice under risk and ambiguity," Journal of Economic Behavior & Organization, Elsevier, vol. 133(C), pages 373-384.
    2. Matthew Rabin, 2002. "Inference by Believers in the Law of Small Numbers," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(3), pages 775-816.
    3. Barberis, Nicholas & Shleifer, Andrei & Vishny, Robert, 1998. "A model of investor sentiment," Journal of Financial Economics, Elsevier, vol. 49(3), pages 307-343, September.
    4. Daniel L. Chen & Tobias J. Moskowitz & Kelly Shue, 2016. "Decision Making Under the Gambler’s Fallacy: Evidence from Asylum Judges, Loan Officers, and Baseball Umpires," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(3), pages 1181-1242.
    5. Dimitris Batzilis & Sonia Jaffe & Steven Levitt & John A. List & Jeffrey Picel, 2019. "Behavior in Strategic Settings: Evidence from a Million Rock-Paper-Scissors Games," Games, MDPI, vol. 10(2), pages 1-34, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shoshan, Vered & Hazan, Tamir & Plonsky, Ori, 2023. "BEAST-Net: Learning novel behavioral insights using a neural network adaptation of a behavioral model," OSF Preprints kaeny, Center for Open Science.
    2. Daoud, Adel & Kim, Rockli & Subramanian, S.V., 2019. "Predicting women's height from their socioeconomic status: A machine learning approach," Social Science & Medicine, Elsevier, vol. 238(C), pages 1-1.
    3. Gisches, Eyran J. & Qi, Hang & Becker, William J. & Rapoport, Amnon, 2021. "Strategic retailers and myopic consumers: Competitive pricing of perishable goods," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 92(C).
    4. Daniel J. Benjamin, 2018. "Errors in Probabilistic Reasoning and Judgment Biases," GRU Working Paper Series GRU_2018_023, City University of Hong Kong, Department of Economics and Finance, Global Research Unit.
    5. Pedro Bordalo & John J. Conlon & Nicola Gennaioli & Spencer Yongwook Kwon & Andrei Shleifer, 2023. "How People Use Statistics," NBER Working Papers 31631, National Bureau of Economic Research, Inc.
      • Pedro Bordalo & John Conlon & Nicola Gennaioli & Spencer Kwon & Andrei Shleifer, 2023. "How People Use Statistics," Working Papers 699, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    6. Isil Erel & Léa H Stern & Chenhao Tan & Michael S Weisbach, 2021. "Selecting Directors Using Machine Learning," NBER Chapters, in: Big Data: Long-Term Implications for Financial Markets and Firms, pages 3226-3264, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniel J. Benjamin, 2018. "Errors in Probabilistic Reasoning and Judgment Biases," GRU Working Paper Series GRU_2018_023, City University of Hong Kong, Department of Economics and Finance, Global Research Unit.
    2. Drew Fudenberg & Jon Kleinberg & Annie Liang & Sendhil Mullainathan, 2019. "Measuring the Completeness of Theories," Papers 1910.07022, arXiv.org.
    3. Daniel J. Benjamin & Don A. Moore & Matthew Rabin, 2017. "Biased Beliefs About Random Samples: Evidence from Two Integrated Experiments," NBER Working Papers 23927, National Bureau of Economic Research, Inc.
    4. Tobias J. Moskowitz, 2021. "Asset Pricing and Sports Betting," Journal of Finance, American Finance Association, vol. 76(6), pages 3153-3209, December.
    5. Stefano DellaVigna, 2009. "Psychology and Economics: Evidence from the Field," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 315-372, June.
    6. Wu, Chen-Hui & Wu, Chin-Shun & Liu, Victor W., 2009. "The conservatism bias in an emerging stock market: Evidence from Taiwan," Pacific-Basin Finance Journal, Elsevier, vol. 17(4), pages 494-505, September.
    7. Tiziana Assenza & Te Bao & Cars Hommes & Domenico Massaro, 2014. "Experiments on Expectations in Macroeconomics and Finance," Research in Experimental Economics, in: Experiments in Macroeconomics, volume 17, pages 11-70, Emerald Group Publishing Limited.
    8. Emara, Noha & Owens, David & Smith, John & Wilmer, Lisa, 2017. "Serial correlation in National Football League play calling and its effects on outcomes," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 69(C), pages 125-132.
    9. Elena Asparouhova & Michael Hertzel & Michael Lemmon, 2009. "Inference from Streaks in Random Outcomes: Experimental Evidence on Beliefs in Regime Shifting and the Law of Small Numbers," Management Science, INFORMS, vol. 55(11), pages 1766-1782, November.
    10. Roger K. Loh & Mitch Warachka, 2012. "Streaks in Earnings Surprises and the Cross-Section of Stock Returns," Management Science, INFORMS, vol. 58(7), pages 1305-1321, July.
    11. Si Chen, 2022. "Information and dynamic trading with the Gambler’s fallacy," Mathematics and Financial Economics, Springer, volume 16, number 1, December.
    12. Paola Criscuolo & Linus Dahlander & Thorsten Grohsjean & Ammon Salter, 2021. "The Sequence Effect in Panel Decisions: Evidence from the Evaluation of Research and Development Projects," Organization Science, INFORMS, vol. 32(4), pages 987-1008, July.
    13. Bloomfield, Robert & Hales, Jeffrey, 2002. "Predicting the next step of a random walk: experimental evidence of regime-shifting beliefs," Journal of Financial Economics, Elsevier, vol. 65(3), pages 397-414, September.
    14. He, Kevin, 2022. "Mislearning from censored data: The gambler's fallacy and other correlational mistakes in optimal-stopping problems," Theoretical Economics, Econometric Society, vol. 17(3), July.
    15. Sean Duffy & J. J. Naddeo & David Owens & John Smith, 2024. "Cognitive Load and Mixed Strategies: On Brains and Minimax," International Game Theory Review (IGTR), World Scientific Publishing Co. Pte. Ltd., vol. 26(03), pages 1-34, September.
    16. Zimper, Alexander, 2023. "Unrealized arbitrage opportunities in naive equilibria with non-Bayesian belief processes," Mathematical Social Sciences, Elsevier, vol. 125(C), pages 27-41.
    17. Matthew Rabin & Dimitri Vayanos, 2005. "The Gambler's and Hot-Hand Fallacies In a Dynamic-Inference Model," Levine's Bibliography 122247000000000972, UCLA Department of Economics.
    18. Jonas Radbruch & Amelie Schiprowski, 2025. "Interview Sequences and the Formation of Subjective Assessments," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 92(2), pages 1226-1256.
    19. Qingxia Kong & Georg D. Granic & Nicolas S. Lambert & Chung Piaw Teo, 2020. "Judgment Error in Lottery Play: When the Hot Hand Meets the Gambler’s Fallacy," Management Science, INFORMS, vol. 66(2), pages 844-862, February.
    20. Sanjit Dhami & Ali al-Nowaihi & Cass R. Sunstein, 2019. "Heuristics and Public Policy: Decision-making Under Bounded Rationality," Studies in Microeconomics, , vol. 7(1), pages 7-58, June.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pen:papers:18-010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Administrator (email available below). General contact details of provider: https://edirc.repec.org/data/deupaus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.