IDEAS home Printed from https://ideas.repec.org/p/rco/dpaper/146.html
   My bibliography  Save this paper

Incentives, Search Engines, and the Elicitation of Subjective Beliefs: Evidence From Representative Online Survey Experiments

Author

Listed:
  • Grewenig, Elisabeth

    (ifo Institute)

  • Lergetporer, Philipp

    (ifo Institute)

  • Werner, Katharina

    (ifo Institute)

  • Woessmann, Ludger

    (ifo Institute and LMU Munich)

Abstract

A large literature studies subjective beliefs about economic facts using unincentivized survey questions. We devise randomized experiments in a representative online survey to investigate whether incentivizing belief accuracy affects stated beliefs about average earnings by professional degree and average public school spending. Incentive provision does not impact earnings beliefs, but improves school-spending beliefs. Response patterns suggest that the latter effect likely reflects increased online-search activity. Consistently, an experiment that just encourages search-engine usage produces very similar results. Another experiment provides no evidence of experimenter-demand effects. Overall, results suggest that incentive provision does not reduce bias in our survey-based belief measures.

Suggested Citation

  • Grewenig, Elisabeth & Lergetporer, Philipp & Werner, Katharina & Woessmann, Ludger, 2019. "Incentives, Search Engines, and the Elicitation of Subjective Beliefs: Evidence From Representative Online Survey Experiments," Rationality and Competition Discussion Paper Series 146, CRC TRR 190 Rationality and Competition.
  • Handle: RePEc:rco:dpaper:146
    as

    Download full text from publisher

    File URL: https://rationality-and-competition.de/wp-content/uploads/discussion_paper/146.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Adeline Delavande & Xavier Giné & David McKenzie, 2011. "Eliciting probabilistic expectations with visual aids in developing countries: how sensitive are answers to variations in elicitation design?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(3), pages 479-497, April.
    2. Grigorieff, Alexis & Roth, Christopher & Ubfal, Diego, 2016. "Does Information Change Attitudes Towards Immigrants? Representative Evidence from Survey Experiments," IZA Discussion Papers 10419, Institute of Labor Economics (IZA).
    3. Olivier Armantier & Scott Nelson & Giorgio Topa & Wilbert van der Klaauw & Basit Zafar, 2016. "The Price Is Right: Updating Inflation Expectations in a Randomized Price Information Experiment," The Review of Economics and Statistics, MIT Press, vol. 98(3), pages 503-523, July.
    4. Alesina, Alberto & La Ferrara, Eliana, 2005. "Preferences for redistribution in the land of opportunities," Journal of Public Economics, Elsevier, vol. 89(5-6), pages 897-931, June.
    5. Olivier Coibion & Yuriy Gorodnichenko & Saten Kumar, 2018. "How Do Firms Form Their Expectations? New Survey Evidence," American Economic Review, American Economic Association, vol. 108(9), pages 2671-2713, September.
    6. Ansolabehere, Stephen & Meredith, Marc & Snowberg, Erik, 2013. "Asking About Numbers: Why and How," Political Analysis, Cambridge University Press, vol. 21(1), pages 48-69, January.
    7. Mariana Blanco & Dirk Engelmann & Alexander Koch & Hans-Theo Normann, 2010. "Belief elicitation in experiments: is there a hedging problem?," Experimental Economics, Springer;Economic Science Association, vol. 13(4), pages 412-438, December.
    8. Ilyana Kuziemko & Michael I. Norton & Emmanuel Saez & Stefanie Stantcheva, 2015. "How Elastic Are Preferences for Redistribution? Evidence from Randomized Survey Experiments," American Economic Review, American Economic Association, vol. 105(4), pages 1478-1508, April.
    9. Armin Falk & Anke Becker & Thomas Dohmen & David Huffman & Uwe Sunde, 2016. "The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences," Working Papers 2016-003, Human Capital and Economic Opportunity Working Group.
    10. Bleemer, Zachary & Zafar, Basit, 2018. "Intended college attendance: Evidence from an experiment on college returns and costs," Journal of Public Economics, Elsevier, vol. 157(C), pages 184-211.
    11. Olivier Coibion & Yuriy Gorodnichenko, 2012. "What Can Survey Forecasts Tell Us about Information Rigidities?," Journal of Political Economy, University of Chicago Press, vol. 120(1), pages 116-159.
    12. Lergetporer, Philipp & Schwerdt, Guido & Werner, Katharina & West, Martin R. & Woessmann, Ludger, 2018. "How information affects support for education spending: Evidence from survey experiments in Germany and the United States," Journal of Public Economics, Elsevier, vol. 167(C), pages 138-157.
    13. Arcidiacono, Peter & Hotz, V. Joseph & Kang, Songman, 2012. "Modeling college major choices using elicited measures of expectations and counterfactuals," Journal of Econometrics, Elsevier, vol. 166(1), pages 3-16.
    14. Andreas Fuster & Ricardo Perez-Truglia & Mirko Wiederholt & Basit Zafar, 2018. "Expectations with Endogenous Information Acquisition: An Experimental Investigation," NBER Working Papers 24767, National Bureau of Economic Research, Inc.
    15. Ralph Stinebrickner & Todd R. Stinebrickner, 2014. "A Major in Science? Initial Beliefs and Final Outcomes for College Major and Dropout," Review of Economic Studies, Oxford University Press, vol. 81(1), pages 426-472.
    16. Katja Maria Kaufmann, 2014. "Understanding the income gradient in college attendance in Mexico: The role of heterogeneity in expected returns," Quantitative Economics, Econometric Society, vol. 5(3), pages 583-630, November.
    17. Orazio Attanasio & Teodora Boneva & Christopher Rauh, 2018. "Parental Beliefs about Returns to Different Types of Investments in School Children," Working Papers 2018-032, Human Capital and Economic Opportunity Working Group.
    18. Mounir Karadja & Johanna Mollerstrom & David Seim, 2017. "Richer (and Holier) Than Thou? The Effect of Relative Income Improvements on Demand for Redistribution," The Review of Economics and Statistics, MIT Press, vol. 99(2), pages 201-212, May.
    19. Teodora Boneva & Christopher Rauh, 2017. "Socio-Economic Gaps in University Enrollment: The Role of Perceived Pecuniary and Non-Pecuniary Returns," Working Papers 2017-080, Human Capital and Economic Opportunity Working Group.
    20. Zachary Bleemer & Basit Zafar, 2014. "Information heterogeneity and intended college enrollment," Staff Reports 685, Federal Reserve Bank of New York.
    21. Matthew Wiswall & Basit Zafar, 2015. "Determinants of College Major Choice: Identification using an Information Experiment," Review of Economic Studies, Oxford University Press, vol. 82(2), pages 791-824.
    22. Hugo Benítez-Silva & Moshe Buchinsky & Hiu Man Chan & Sofia Cheidvasser & John Rust, 2004. "How large is the bias in self-reported disability?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 19(6), pages 649-670.
    23. Basit Zafar, 2011. "How Do College Students Form Expectations?," Journal of Labor Economics, University of Chicago Press, vol. 29(2), pages 301-348.
    24. Christopher Roth & Johannes Wohlfart, 2020. "How Do Expectations about the Macroeconomy Affect Personal Expectations and Behavior?," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 731-748, October.
    25. Pietro Biroli & Teodora Boneva & Akash Raja & Christopher Rauh, 2018. "Parental Beliefs about Returns to Child Health Investments," Working Papers 2018-008, Human Capital and Economic Opportunity Working Group.
    26. Teodora Boneva & Christopher Rauh, 2018. "Parental Beliefs about Returns to Educational Investments—The Later the Better?," Journal of the European Economic Association, European Economic Association, vol. 16(6), pages 1669-1711.
    27. Karl Schlag & James Tremewan & Joël Weele, 2015. "A penny for your thoughts: a survey of methods for eliciting beliefs," Experimental Economics, Springer;Economic Science Association, vol. 18(3), pages 457-490, September.
    28. Arcidiacono, Peter, 2004. "Ability sorting and the returns to college major," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 343-375.
    29. Sendhil Mullainathan & Marianne Bertrand, 2001. "Do People Mean What They Say? Implications for Subjective Survey Data," American Economic Review, American Economic Association, vol. 91(2), pages 67-72, May.
    30. Wang, Stephanie W., 2011. "Incentive effects: The case of belief elicitation from individuals in groups," Economics Letters, Elsevier, vol. 111(1), pages 30-33, April.
    31. Alberto Cavallo & Guillermo Cruces & Ricardo Perez-Truglia, 2017. "Inflation Expectations, Learning, and Supermarket Prices: Evidence from Survey Experiments," American Economic Journal: Macroeconomics, American Economic Association, vol. 9(3), pages 1-35, July.
    32. Daniel Zizzo, 2010. "Experimenter demand effects in economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(1), pages 75-98, March.
    33. Prior, Markus & Sood, Gaurav & Khanna, Kabir, 2015. "You Cannot be Serious: The Impact of Accuracy Incentives on Partisan Bias in Reports of Economic Perceptions," Quarterly Journal of Political Science, now publishers, vol. 10(4), pages 489-518, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dietmar Fehr & Johanna Mollerstrom & Ricardo Perez-Truglia, 2019. "Your Place in the World: Relative Income and Global Inequality," NBER Working Papers 26555, National Bureau of Economic Research, Inc.
    2. Haaland, Ingar & Roth, Christopher & Wohlfart. Johannes, 2020. "Designing Information Provision Experiments," The Warwick Economics Research Paper Series (TWERPS) 1275, University of Warwick, Department of Economics.
    3. Patrick Bareinz & Fabian Koenings, 2021. "Framing of Economic News and Policy Support During a Pandemic: Evidence from an Information Experiment," Jena Economic Research Papers 2021-004, Friedrich-Schiller-University Jena.
    4. Christopher Roth & Sonja Settele & Johannes Wohlfart, 2020. "Beliefs About Public Debt and the Demand for Government Spending," CEBI working paper series 20-05, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).
    5. Egor Starkov, 2020. "Only Time Will Tell: Credible Dynamic Signaling," Discussion Papers 20-05, University of Copenhagen. Department of Economics.
    6. Juerg Schweri, 2021. "Predicting polytomous career choices in healthcare using probabilistic expectations data," Health Economics, John Wiley & Sons, Ltd., vol. 30(3), pages 544-563, March.
    7. Brad R. Taylor, 2020. "The psychological foundations of rational ignorance: biased heuristics and decision costs," Constitutional Political Economy, Springer, vol. 31(1), pages 70-88, March.
    8. Valeria Burdea & Jonathan Woon, 2021. "Online Belief Elicitation Methods," CESifo Working Paper Series 8823, CESifo.
    9. Lergetporer, Philipp & Woessmann, Ludger, 2021. "Earnings Information and Public Preferences for University Tuition: Evidence from Representative Experiments," IZA Discussion Papers 14386, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ingar Haaland & Christopher Roth & Johannes Wohlfart, 2020. "Designing Information Provision Experiments," CEBI working paper series 20-20, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).
    2. Katharina Werner, 2019. "Der Einfluss von Informationen auf die öffentliche Meinung zur Bildung - Erkenntnisse aus repräsentativen Befragungsexperimenten," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 82, August.
    3. Warn N. Lekfuangfu & Reto Odermatt, 2020. "All I have to do is dream? The role of aspirations in intergenerational mobility and well-being," PIER Discussion Papers 142, Puey Ungphakorn Institute for Economic Research, revised Jul 2020.
    4. Philipp Lergetporer & Marc Piopiunik & Lisa Simon, 2017. "Does the Education Level of Refugees Affect Natives' Attitudes?," CESifo Working Paper Series 6832, CESifo.
    5. Bobba, Matteo & Frisancho, Veronica, 2016. "Learning about Oneself: The Effects of Performance Feedback on School Choice," IZA Discussion Papers 10360, Institute of Labor Economics (IZA).
    6. Ralph Stinebrickner & Todd R. Stinebrickner, 2014. "A Major in Science? Initial Beliefs and Final Outcomes for College Major and Dropout," Review of Economic Studies, Oxford University Press, vol. 81(1), pages 426-472.
    7. Lergetporer, Philipp & Werner, Katharina & Woessmann, Ludger, 2018. "Does Ignorance of Economic Returns and Costs Explain the Educational Aspiration Gap? Evidence from Representative Survey Experiments," IZA Discussion Papers 11453, Institute of Labor Economics (IZA).
    8. Elisabeth Grewenig & Philipp Lergetporer & Katharina Werner, 2020. "Gender Norms and Labor-Supply Expectations: Experimental Evidence from Adolescents," CESifo Working Paper Series 8611, CESifo.
    9. Lergetporer, Philipp & Piopiunik, Marc & Simon, Lisa, 2021. "Does the education level of refugees affect natives’ attitudes?," European Economic Review, Elsevier, vol. 134(C).
    10. Lukas Kiessling, 2020. "How Do Parents Perceive the Returns to Parenting Styles and Neighborhoods?," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2020_14, Max Planck Institute for Research on Collective Goods.
    11. Bhalotra, Sonia & Delavande, Adeline & Font-Gilabert, Paulino & Maselko, Joanna, 2020. "Maternal investments in children: the role of expected effort and returns," ISER Working Paper Series 2020-03, Institute for Social and Economic Research.
    12. D'Acunto, Francesco & Hoang, Daniel & Paloviita, Maritta & Weber, Michael, 2019. "IQ, Expectations, and Choice," Research Discussion Papers 2/2019, Bank of Finland.
    13. Orazio Attanasio & Teodora Boneva & Christopher Rauh, 2018. "Parental Beliefs about Returns to Different Types of Investments in School Children," Working Papers 2018-032, Human Capital and Economic Opportunity Working Group.
    14. Kunz, Johannes S. & Staub, Kevin E., 2020. "Early subjective completion beliefs and the demand for post-secondary education," Journal of Economic Behavior & Organization, Elsevier, vol. 177(C), pages 34-55.
    15. Christopher Roth & Johannes Wohlfart, 2020. "How Do Expectations about the Macroeconomy Affect Personal Expectations and Behavior?," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 731-748, October.
    16. Ranveig Falch, 2021. "How Do People Trade Off Resources Between Quick and Slow Learners?," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2021_04, Max Planck Institute for Research on Collective Goods.
    17. Ralph Stinebrickner & Todd Stinebrickner, 2014. "Academic Performance and College Dropout: Using Longitudinal Expectations Data to Estimate a Learning Model," Journal of Labor Economics, University of Chicago Press, vol. 32(3), pages 601-644.
    18. Patrick Bareinz & Fabian Koenings, 2021. "Framing of Economic News and Policy Support During a Pandemic: Evidence from an Information Experiment," Jena Economic Research Papers 2021-004, Friedrich-Schiller-University Jena.
    19. Lergetporer, Philipp & Werner, Katharina & Woessmann, Ludger, 2020. "Educational inequality and public policy preferences: Evidence from representative survey experiments," Journal of Public Economics, Elsevier, vol. 188(C).
    20. Fricke, Hans & Grogger, Jeff & Steinmayr, Andreas, 2015. "Does Exposure to Economics Bring New Majors to the Field? Evidence from a Natural Experiment," IZA Discussion Papers 9003, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    beliefs; incentives; online search; survey experiment;
    All these keywords.

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:rco:dpaper:146. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://rationality-and-competition.de .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Viviana Lalli (email available below). General contact details of provider: https://rationality-and-competition.de .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.