IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_8406.html
   My bibliography  Save this paper

Designing Information Provision Experiments

Author

Listed:
  • Ingar K. Haaland
  • Christopher Roth
  • Johannes Wohlfart

Abstract

We review methodological questions relevant for the design of information provision experiments. We first provide a literature review of major areas in which information provision experiments are applied. We then outline key measurement challenges and design recommendations that may be of help for practitioners planning to conduct an information experiment. We discuss the measurement of subjective beliefs, including the role of incentives and ways to reduce measurement error. We also discuss the design of the information intervention, as well as the measurement of belief updating. Moreover, we describe ways to mitigate potential experimenter demand effects and numerical anchoring arising from the information treatment. Finally, we discuss typical effect sizes in information experiments.

Suggested Citation

  • Ingar K. Haaland & Christopher Roth & Johannes Wohlfart, 2020. "Designing Information Provision Experiments," CESifo Working Paper Series 8406, CESifo.
  • Handle: RePEc:ces:ceswps:_8406
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp8406.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Luis Armona & Andreas Fuster & Basit Zafar, 2019. "Home Price Expectations and Behaviour: Evidence from a Randomized Information Experiment," Review of Economic Studies, Oxford University Press, vol. 86(4), pages 1371-1410.
    2. Jonas Hjort & Diana Moreira & Gautam Rao & Juan Francisco Santini, 2019. "How Research Affects Policy: Experimental Evidence from 2,150 Brazilian Municipalities," NBER Working Papers 25941, National Bureau of Economic Research, Inc.
    3. Olivier Armantier & Scott Nelson & Giorgio Topa & Wilbert van der Klaauw & Basit Zafar, 2016. "The Price Is Right: Updating Inflation Expectations in a Randomized Price Information Experiment," The Review of Economics and Statistics, MIT Press, vol. 98(3), pages 503-523, July.
    4. Kate Orkin & Eliana Carranza & Robert Garlick & Neil Rankin, 2020. "Job Search and Hiring with Two-sided Limited Information about Workseekers’ Skills," CSAE Working Paper Series 2020-10, Centre for the Study of African Economies, University of Oxford.
    5. Daron Acemoglu & Ali Cheema & Asim I. Khwaja & James A. Robinson, 2020. "Trust in State and Nonstate Actors: Evidence from Dispute Resolution in Pakistan," Journal of Political Economy, University of Chicago Press, vol. 128(8), pages 3090-3147.
    6. Doerrenberg, Philipp & Peichl, Andreas, 2017. "Tax morale and the role of social norms and reciprocity: Evidence from a randomized survey experiment," ZEW Discussion Papers 17-045, ZEW - Leibniz Centre for European Economic Research.
    7. Benjamin Enke & Thomas Graeber, 2019. "Cognitive Uncertainty," NBER Working Papers 26518, National Bureau of Economic Research, Inc.
    8. Ansolabehere, Stephen & Meredith, Marc & Snowberg, Erik, 2013. "Asking About Numbers: Why and How," Political Analysis, Cambridge University Press, vol. 21(1), pages 48-69, January.
    9. Elisabeth Grewenig & Philipp Lergetporer & Katharina Werner & Ludger Woessmann, 2019. "Incentives, search engines, and the elicitation of subjective beliefs: evidence from representative online survey experiments," CESifo Working Paper Series 7556, CESifo.
    10. Fitzsimons, Emla & Malde, Bansi & Mesnard, Alice & Vera-Hernández, Marcos, 2016. "Nutrition, information and household behavior: Experimental evidence from Malawi," Journal of Development Economics, Elsevier, vol. 122(C), pages 113-126.
    11. Abebe, Girum & Caria, Stefano & Fafchamps, Marcel & Falco, Paolo & Franklin, Simon & Quinn, Simon, 2017. "Anonymity of distance? Job search and labour market exclusion in a growing African city," LSE Research Online Documents on Economics 86573, London School of Economics and Political Science, LSE Library.
    12. Philipp Lergetporer & Katharina Werner & Ludger Woessmann, 2018. "Does Ignorance of Economic Returns and Costs Explain the Educational Aspiration Gap? Evidence from Representative Survey Experiments," CESifo Working Paper Series 7000, CESifo.
    13. Bleemer, Zachary & Zafar, Basit, 2018. "Intended college attendance: Evidence from an experiment on college returns and costs," Journal of Public Economics, Elsevier, vol. 157(C), pages 184-211.
    14. Zoë Cullen & Ricardo Perez-Truglia, 2018. "How Much Does Your Boss Make? The Effects of Salary Comparisons," NBER Working Papers 24841, National Bureau of Economic Research, Inc.
    15. Michèle Belot & Philipp Kircher & Paul Muller, 2018. "How Wage Announcements Affect Job Search - A Field Experiment," CESifo Working Paper Series 7302, CESifo.
    16. Coibion, Olivier & Gorodnichenko, Yuriy & Kumar, Saten & Pedemonte, Mathieu, 2020. "Inflation expectations as a policy tool?," Journal of International Economics, Elsevier, vol. 124(C).
    17. Facchini, Giovanni & Margalit, Yotam & Nakata, Hiroyuki, 2016. "Countering Public Opposition to Immigration: The Impact of Information Campaigns," IZA Discussion Papers 10420, Institute of Labor Economics (IZA).
    18. Lergetporer, Philipp & Schwerdt, Guido & Werner, Katharina & West, Martin R. & Woessmann, Ludger, 2018. "How information affects support for education spending: Evidence from survey experiments in Germany and the United States," Journal of Public Economics, Elsevier, vol. 167(C), pages 138-157.
    19. John Horton & David Rand & Richard Zeckhauser, 2011. "The online laboratory: conducting experiments in a real labor market," Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 399-425, September.
    20. Olivier Coibion & Yuriy Gorodnichenko & Saten Kumar & Jane Ryngaert, 2018. "Do You Know That I Know That You Know...? Higher-Order Beliefs in Survey Data," NBER Working Papers 24987, National Bureau of Economic Research, Inc.
    21. Leonardo Bursztyn & Stefano Fiorin & Daniel Gottlieb & Martin Kanz, 2019. "Moral Incentives in Credit Card Debt Repayment: Evidence from a Field Experiment," Journal of Political Economy, University of Chicago Press, vol. 127(4), pages 1641-1683.
    22. Marcelo L. Bérgolo & Rodrigo Ceni & Guillermo Cruces & Matias Giaccobasso & Ricardo Perez-Truglia, 2017. "Tax Audits as Scarecrows: Evidence from a Large-Scale Field Experiment," NBER Working Papers 23631, National Bureau of Economic Research, Inc.
    23. Alberto Alesina & Michela Carlana & Eliana La Ferrara & Paolo Pinotti, 2018. "Revealing Stereotypes: Evidence from Immigrants in Schools," NBER Working Papers 25333, National Bureau of Economic Research, Inc.
    24. Mounir Karadja & Johanna Mollerstrom & David Seim, 2017. "Richer (and Holier) Than Thou? The Effect of Relative Income Improvements on Demand for Redistribution," The Review of Economics and Statistics, MIT Press, vol. 99(2), pages 201-212, May.
    25. Teodora Boneva & Christopher Rauh, 2017. "Socio-Economic Gaps in University Enrollment: The Role of Perceived Pecuniary and Non-Pecuniary Returns," Working Papers 2017-080, Human Capital and Economic Opportunity Working Group.
    26. Bursztyn, Leonardo & Cantoni, Davide & Funk, Patricia & Yuchtman, Noam, 2017. "Polls, the Press, and Political Participation: The Effects of Anticipated Election Closeness on Voter Turnout," CEPR Discussion Papers 12088, C.E.P.R. Discussion Papers.
    27. Dietmar Fehr & Johanna Mollerstrom & Ricardo Perez-Truglia, 2019. "Your Place in the World: The Demand for National and Global Redistribution," NBER Working Papers 26555, National Bureau of Economic Research, Inc.
    28. Alberto Alesina & Stefanie Stantcheva & Edoardo Teso, 2018. "Intergenerational Mobility and Preferences for Redistribution," American Economic Review, American Economic Association, vol. 108(2), pages 521-554, February.
    29. Cesi Cruz & Philip Keefer & Julien Labonne & Francesco Trebbi, 2018. "Making Policies Matter: Voter Responses to Campaign Promises," NBER Working Papers 24785, National Bureau of Economic Research, Inc.
    30. Leonardo Bursztyn & Davide Cantoni & Patricia Funk & Felix Schönenberger & Noam Yuchtman, 2017. "Identifying the Effect of Election Closeness on Voter Turnout: Evidence from Swiss Referenda," NBER Working Papers 23490, National Bureau of Economic Research, Inc.
    31. Zachary Bleemer & Basit Zafar, 2014. "Information heterogeneity and intended college enrollment," Staff Reports 685, Federal Reserve Bank of New York.
    32. Dolls, Mathias & Doerrenberg, Philipp & Peichl, Andreas & Stichnoth, Holger, 2018. "Do retirement savings increase in response to information about retirement and expected pensions?," Journal of Public Economics, Elsevier, vol. 158(C), pages 168-179.
    33. Abhijit Banerjee & Eliana La Ferrara & Victor H. Orozco-Olvera, 2019. "The Entertaining Way to Behavioral Change: Fighting HIV with MTV," NBER Working Papers 26096, National Bureau of Economic Research, Inc.
    34. Benabou, Roland & Falk, Armin & Tirole, Jean, 2018. "Narratives, Imperatives, and Moral Reasoning," IZA Discussion Papers 11665, Institute of Labor Economics (IZA).
    35. Bernard, Tanguy & Dercon, Stefan & Orkin, Kate & Taffesse, Alemayehu, 2014. "The Future in Mind: Aspirations and Forward-Looking Behaviour in Rural Ethiopia," CEPR Discussion Papers 10224, C.E.P.R. Discussion Papers.
    36. Bursztyn, Leonardo & Rao, Aakaash & Roth, Christopher & Yanagizawa-Drott, David, 2020. "Misinformation During a Pandemic," CAGE Online Working Paper Series 481, Competitive Advantage in the Global Economy (CAGE).
    37. Leonardo Bursztyn & Florian Ederer & Bruno Ferman & Noam Yuchtman, 2012. "Understanding Peer Effects in Financial Decisions: Evidence from a Field Experiment," NBER Working Papers 18241, National Bureau of Economic Research, Inc.
    38. Altmann, Steffen & Falk, Armin & Jäger, Simon & Zimmermann, Florian, 2018. "Learning about job search: A field experiment with job seekers in Germany," Journal of Public Economics, Elsevier, vol. 164(C), pages 33-49.
    39. Emma Riley, 2017. "Role models in movies: the impact of Queen of Katwe on students’ educational attainment," CSAE Working Paper Series 2017-13, Centre for the Study of African Economies, University of Oxford.
    40. Olivier Coibion & Dimitris Georgarakos & Yuriy Gorodnichenko & Michael Weber, 2020. "Forward Guidance and Household Expectations," NBER Working Papers 26778, National Bureau of Economic Research, Inc.
    41. Daniel J. Benjamin & Don A. Moore & Matthew Rabin, 2017. "Biased Beliefs About Random Samples: Evidence from Two Integrated Experiments," NBER Working Papers 23927, National Bureau of Economic Research, Inc.
    42. Dietmar Fehr & Daniel Müller & Marcel Preuss, 2020. "Social Mobility Perceptions and Inequality Acceptance," Working Papers 2020-02, Faculty of Economics and Statistics, University of Innsbruck.
    43. Leonardo Bursztyn & Georgy Egorov & Stefano Fiorin, 2017. "From Extreme to Mainstream: How Social Norms Unravel," NBER Working Papers 23415, National Bureau of Economic Research, Inc.
    44. Leiser, David & Drori, Shelly, 2005. "NaIve understanding of inflation," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 34(2), pages 179-198, March.
    45. Cruces, Guillermo & Perez-Truglia, Ricardo & Tetaz, Martin, 2013. "Biased perceptions of income distribution and preferences for redistribution: Evidence from a survey experiment," Journal of Public Economics, Elsevier, vol. 98(C), pages 100-112.
    46. Hanspal, Tobin & Weber, Annika & Wohlfart, Johannes, 2020. "Exposure to the COVID-19 stock market crash and its effect on household expectations," SAFE Working Paper Series 279, Leibniz Institute for Financial Research SAFE.
    47. Olivier Coibion & Yuriy Gorodnichenko & Michael Weber, 2019. "Monetary Policy Communications and their Effects on Household Inflation Expectations," NBER Working Papers 25482, National Bureau of Economic Research, Inc.
    48. Kimmo Eriksson & Brent Simpson, 2012. "What do Americans know about inequality? It depends on how you ask them," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 7(6), pages 741-745, November.
    49. Pedro Bordalo & Nicola Gennaioli & Andrei Shleifer, 2015. "Memory, Attention and Choice," Working Paper 237951, Harvard University OpenScholar.
    50. Daniel Zizzo, 2010. "Experimenter demand effects in economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 13(1), pages 75-98, March.
    51. David Danz & Lise Vesterlund & Alistair J. Wilson, 2020. "Belief Elicitation: Limiting Truth Telling with Information on Incentives," CESifo Working Paper Series 8048, CESifo.
    52. Tobin Hanspal & Annika Weber & Johannes Wohlfart, 2020. "Exposure to the Covid-19 Stock Market Crash and its Effect on Household Expectations," CESifo Working Paper Series 8244, CESifo.
    53. Peter Andre & Carlo Pizzinelli & Christopher Roth & Johannes Wohlfart, 2019. "Subjective Models of the Macroeconomy: Evidence from Experts and a Representative Sample," CESifo Working Paper Series 7850, CESifo.
    54. Grewenig, Elisabeth & Lergetporer, Philipp & Simon, Lisa & Werner, Katharina & Woessmann, Ludger, 2018. "Can Online Surveys Represent the Entire Population?," IZA Discussion Papers 11799, Institute of Labor Economics (IZA).
    55. Davide Cantoni & David Y Yang & Noam Yuchtman & Y Jane Zhang, 2019. "Protests as Strategic Games: Experimental Evidence from Hong Kong's Antiauthoritarian Movement," The Quarterly Journal of Economics, Oxford University Press, vol. 134(2), pages 1021-1077.
    56. Jonathan de Quidt & Johannes Haushofer & Christopher Roth, 2018. "Measuring and Bounding Experimenter Demand," American Economic Review, American Economic Association, vol. 108(11), pages 3266-3302, November.
    57. Leonardo Bursztyn & Alessandra L. González & David Yanagizawa-Drott, 2018. "Misperceived Social Norms: Female Labor Force Participation in Saudi Arabia," NBER Working Papers 24736, National Bureau of Economic Research, Inc.
    58. Andreas Fuster & Ricardo Perez-Truglia & Mirko Wiederholt & Basit Zafar, 2018. "Expectations with Endogenous Information Acquisition: An Experimental Investigation," NBER Working Papers 24767, National Bureau of Economic Research, Inc.
    59. Olivier Coibion & Yuriy Gorodnichenko & Tiziano Ropele, 2020. "Inflation Expectations and Firm Decisions: New Causal Evidence," The Quarterly Journal of Economics, Oxford University Press, vol. 135(1), pages 165-219.
    60. Bullock, John G. & Gerber, Alan S. & Hill, Seth J. & Huber, Gregory A., 2015. "Partisan Bias in Factual Beliefs about Politics," Quarterly Journal of Political Science, now publishers, vol. 10(4), pages 519-578, December.
    61. Orazio Attanasio & Teodora Boneva & Christopher Rauh, 2018. "Parental Beliefs about Returns to Different Types of Investments in School Children," Working Papers 2018-032, Human Capital and Economic Opportunity Working Group.
    62. Chad Kendall & Tommaso Nannicini & Francesco Trebbi, 2015. "How Do Voters Respond to Information? Evidence from a Randomized Campaign," American Economic Review, American Economic Association, vol. 105(1), pages 322-353, January.
    63. Blesse, Sebastian & Buhlmann, Florian & Doerrenberg, Philipp, 2019. "Do people really want a simple tax system? Evidence on preferences towards income tax simplification," ZEW Discussion Papers 19-058, ZEW - Leibniz Centre for European Economic Research.
    64. Alesina, Alberto F & Miano, Armando & Stantcheva, Stefanie, 2018. "Immigration and Redistribution," CEPR Discussion Papers 13035, C.E.P.R. Discussion Papers.
    65. Grant D. Jacobsen, 2019. "How do different sources of policy analysis affect policy preferences? Experimental evidence from the United States," Policy Sciences, Springer;Society of Policy Sciences, vol. 52(3), pages 315-342, September.
    66. Rema Hanna & Sendhil Mullainathan & Joshua Schwartzstein, 2014. "Learning Through Noticing: Theory and Evidence from a Field Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 129(3), pages 1311-1353.
    67. Teodora Boneva & Marta Golin & Christopher Rauh, 2019. "Can Perceived Returns Explain Enrollment Gaps in Postgraduate Education?," Working Papers 2019-045, Human Capital and Economic Opportunity Working Group.
    68. Tobin Hanspal & Annika Weber & Johannes Wohlfart, 2020. "Exposure to the COVID-19 Stock Market Crash and its Effect on Household Expectations," CEBI working paper series 20-13, University of Copenhagen. Department of Economics. The Center for Economic Behavior and Inequality (CEBI).
    69. Bursztyn, Leonardo & Rao, Akaash & Roth, Christopher & Yanagizawa-Drott, David, 2020. "Misinformation during a Pandemic," The Warwick Economics Research Paper Series (TWERPS) 1274, University of Warwick, Department of Economics.
    70. Felix Chopra & Ingar K. Haaland & Christopher Roth, 2019. "Do People Value More Informative News?," CESifo Working Paper Series 8026, CESifo.
    71. Leonardo Bursztyn & Stefano Fiorin & Daniel Gottlieb & Martin Kanz, 2019. "Moral Incentives in Credit Card Debt Repayment: Evidence from a Field Experiment," Journal of Political Economy, University of Chicago Press, vol. 127(4), pages 1641-1683.
    72. Haaland, Ingar & Roth, Christopher, 2019. "Beliefs About Racial Discrimination and Support for Pro-Black Policies," Discussion Paper Series in Economics 2/2019, Norwegian School of Economics, Department of Economics.
    73. Bansak, Kirk & Hainmueller, Jens & Hangartner, Dominik, 2016. "How economic, humanitarian, and religious concerns shape European attitudes toward asylum seekers," LSE Research Online Documents on Economics 67898, London School of Economics and Political Science, LSE Library.
    74. Raymond Fisman & Keith Gladstone & Ilyana Kuziemko & Suresh Naidu, 2017. "Do Americans Want to Tax Capital? Evidence from Online Surveys," NBER Working Papers 23907, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Abel, Martin & Byker, Tanya & Carpenter, Jeffrey P., 2020. "Socially Optimal Mistakes? Debiasing COVID-19 Mortality Risk Perceptions and Prosocial Behavior," IZA Discussion Papers 13560, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    experimental design; beliefs; information; obfuscation;

    JEL classification:

    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making
    • L82 - Industrial Organization - - Industry Studies: Services - - - Entertainment; Media

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_8406. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Klaus Wohlrabe). General contact details of provider: http://edirc.repec.org/data/cesifde.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.