IDEAS home Printed from https://ideas.repec.org/p/pen/papers/18-010.html

The Theory is Predictive, but is it Complete? An Application to Human Perception of Randomness

Author

Listed:
  • Jon Kleinberg

    (Department of Computer Science, Cornell University)

  • Annie Liang

    (Department of Economics, University of Pennsylvania)

  • Sendhil Mullainathan

    (Department of Economics, Harvard University)

Abstract

When testing a theory, we should ask not just whether its predictions match what we see in the data, but also about its “completeness†: how much of the predictable variation in the data does the theory capture? Deï¬ ning completeness is conceptually challenging, but we show how methods based on machine learning can provide tractable measures of completeness. We also identify a model domain—the human perception and generation of randomness—where measures of completeness can be feasibly analyzed; from these measures we discover there is signiï¬ cant structure in the problem that existing theories have yet to capture.

Suggested Citation

  • Jon Kleinberg & Annie Liang & Sendhil Mullainathan, 2017. "The Theory is Predictive, but is it Complete? An Application to Human Perception of Randomness," PIER Working Paper Archive 18-010, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 09 Aug 2017.
    • Handle: RePEc:pen:papers:18-010
    as

    Download full text from publisher

    File URL: https://economics.sas.upenn.edu/system/files/working-papers/18-010%20PIER%20Paper%20Submission.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Barberis, Nicholas & Shleifer, Andrei & Vishny, Robert, 1998. "A model of investor sentiment," Journal of Financial Economics, Elsevier, vol. 49(3), pages 307-343, September.
    2. Peysakhovich, Alexander & Naecker, Jeffrey, 2017. "Using methods from machine learning to evaluate behavioral models of choice under risk and ambiguity," Journal of Economic Behavior & Organization, Elsevier, vol. 133(C), pages 373-384.
    3. Matthew Rabin, 2002. "Inference by Believers in the Law of Small Numbers," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 117(3), pages 775-816.
    4. Daniel L. Chen & Tobias J. Moskowitz & Kelly Shue, 2016. "Decision Making Under the Gambler’s Fallacy: Evidence from Asylum Judges, Loan Officers, and Baseball Umpires," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(3), pages 1181-1242.
    5. Dimitris Batzilis & Sonia Jaffe & Steven Levitt & John A. List & Jeffrey Picel, 2019. "Behavior in Strategic Settings: Evidence from a Million Rock-Paper-Scissors Games," Games, MDPI, vol. 10(2), pages 1-34, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Daniel J. Benjamin, 2018. "Errors in Probabilistic Reasoning and Judgment Biases," GRU Working Paper Series GRU_2018_023, City University of Hong Kong, Department of Economics and Finance, Global Research Unit.
    2. Shoshan, Vered & Hazan, Tamir & Plonsky, Ori, 2023. "BEAST-Net: Learning novel behavioral insights using a neural network adaptation of a behavioral model," OSF Preprints kaeny, Center for Open Science.
    3. Daoud, Adel & Kim, Rockli & Subramanian, S.V., 2019. "Predicting women's height from their socioeconomic status: A machine learning approach," Social Science & Medicine, Elsevier, vol. 238(C), pages 1-1.
    4. Isil Erel & Léa H Stern & Chenhao Tan & Michael S Weisbach, 2021. "Selecting Directors Using Machine Learning," NBER Chapters, in: Big Data: Long-Term Implications for Financial Markets and Firms, pages 3226-3264, National Bureau of Economic Research, Inc.
    5. Pedro Bordalo & John J. Conlon & Nicola Gennaioli & Spencer Yongwook Kwon & Andrei Shleifer, 2023. "How People Use Statistics," NBER Working Papers 31631, National Bureau of Economic Research, Inc.
      • Pedro Bordalo & John Conlon & Nicola Gennaioli & Spencer Kwon & Andrei Shleifer, 2023. "How People Use Statistics," Working Papers 699, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    6. Gisches, Eyran J. & Qi, Hang & Becker, William J. & Rapoport, Amnon, 2021. "Strategic retailers and myopic consumers: Competitive pricing of perishable goods," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 92(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniel J. Benjamin, 2018. "Errors in Probabilistic Reasoning and Judgment Biases," GRU Working Paper Series GRU_2018_023, City University of Hong Kong, Department of Economics and Finance, Global Research Unit.
    2. Drew Fudenberg & Jon Kleinberg & Annie Liang & Sendhil Mullainathan, 2019. "Measuring the Completeness of Theories," Papers 1910.07022, arXiv.org.
    3. Daniel J. Benjamin & Don A. Moore & Matthew Rabin, 2017. "Biased Beliefs About Random Samples: Evidence from Two Integrated Experiments," NBER Working Papers 23927, National Bureau of Economic Research, Inc.
    4. Tobias J. Moskowitz, 2021. "Asset Pricing and Sports Betting," Journal of Finance, American Finance Association, vol. 76(6), pages 3153-3209, December.
    5. Cassella, Stefano & Chen, Te-Feng & Gulen, Huseyin & Liu, Yan, 2025. "Extracting extrapolative beliefs from market prices: An augmented present-value approach," Journal of Financial Economics, Elsevier, vol. 164(C).
    6. Cass R. Sunstein & Lucia A. Reisch, 2025. "In Praise of Computation," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 88(10), pages 2707-2727, October.
    7. Stefano DellaVigna, 2009. "Psychology and Economics: Evidence from the Field," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 315-372, June.
    8. Massenot, Baptiste & Pettinicchi, Yuri, 2018. "Can firms see into the future? Survey evidence from Germany," Journal of Economic Behavior & Organization, Elsevier, vol. 145(C), pages 66-79.
    9. Wu, Chen-Hui & Wu, Chin-Shun & Liu, Victor W., 2009. "The conservatism bias in an emerging stock market: Evidence from Taiwan," Pacific-Basin Finance Journal, Elsevier, vol. 17(4), pages 494-505, September.
    10. Neszveda, G., 2019. "Essays on behavioral finance," Other publications TiSEM 05059039-5236-42a3-be1b-3, Tilburg University, School of Economics and Management.
    11. Bao, Te & Corgnet, Brice & Hanaki, Nobuyuki & Riyanto, Yohanes E. & Zhu, Jiahua, 2023. "Predicting the unpredictable: New experimental evidence on forecasting random walks," Journal of Economic Dynamics and Control, Elsevier, vol. 146(C).
    12. Tiziana Assenza & Te Bao & Cars Hommes & Domenico Massaro, 2014. "Experiments on Expectations in Macroeconomics and Finance," Research in Experimental Economics, in: Experiments in Macroeconomics, volume 17, pages 11-70, Emerald Group Publishing Limited.
    13. Emara, Noha & Owens, David & Smith, John & Wilmer, Lisa, 2017. "Serial correlation in National Football League play calling and its effects on outcomes," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 69(C), pages 125-132.
    14. Xinde Ji & Kelly M. Cobourn, 2021. "Weather Fluctuations, Expectation Formation, and Short-Run Behavioral Responses to Climate Change," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 78(1), pages 77-119, January.
    15. Jonas Radbruch & Amelie Schiprowski, 2025. "Interview Sequences and the Formation of Subjective Assessments," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 92(2), pages 1226-1256.
    16. Cary Frydman & Gideon Nave, 2017. "Extrapolative Beliefs in Perceptual and Economic Decisions: Evidence of a Common Mechanism," Management Science, INFORMS, vol. 63(7), pages 2340-2352, July.
    17. Corgnet, Brice, 2005. "Team formation and biased self-attribution," DEE - Working Papers. Business Economics. WB wb055214, Universidad Carlos III de Madrid. Departamento de Economía de la Empresa.
    18. Adem Atmaz & Huseyin Gulen & Stefano Cassella & Fangcheng Ruan, 2024. "Contrarians, Extrapolators, and Stock Market Momentum and Reversal," Management Science, INFORMS, vol. 70(9), pages 5949-5984, September.
    19. Bianchi, Milo & Jehiel, Philippe, 2020. "Bundlers' dilemmas in financial markets with sampling investors," Theoretical Economics, Econometric Society, vol. 15(2), May.
    20. Brett Green & Jeffrey Zwiebel, 2018. "The Hot-Hand Fallacy: Cognitive Mistakes or Equilibrium Adjustments? Evidence from Major League Baseball," Management Science, INFORMS, vol. 64(11), pages 5315-5348, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pen:papers:18-010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Administrator (email available below). General contact details of provider: https://edirc.repec.org/data/deupaus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.