IDEAS home Printed from https://ideas.repec.org/p/cpr/ceprdp/11343.html
   My bibliography  Save this paper

Persuasion Bias in Science: Can Economics Help?

Author

Listed:
  • Ottaviani, Marco
  • Di Tillio, Alfredo
  • Sørensen, Peter Norman

Abstract

The widespread adoption of randomized controlled experiments owes much to their ability to curtail researchers' conflicts of interest. This paper casts data collection and analysis in a game-theoretic framework. A researcher aims at persuading an evaluator that the causal effect of a treatment outweighs its cost, so as to justify acceptance. The researcher uses private information to (1) sample subjects based on their treatment effect (challenging external validity), (2) assign subjects to treatment based on their baseline outcome (challenging internal validity), or (3) selectively report experimental outcomes (challenging both external and internal validity). The resulting biases have different welfare implications: for sufficiently high acceptance cost, the evaluator loses in cases (1) and (3), but benefits from the researcher's information in case (2).

Suggested Citation

  • Ottaviani, Marco & Di Tillio, Alfredo & Sørensen, Peter Norman, 2016. "Persuasion Bias in Science: Can Economics Help?," CEPR Discussion Papers 11343, C.E.P.R. Discussion Papers.
    • Handle: RePEc:cpr:ceprdp:11343
    as

    Download full text from publisher

    File URL: https://cepr.org/publications/DP11343
    Download Restriction: CEPR Discussion Papers are free to download for our researchers, subscribers and members. If you fall into one of these categories but have trouble downloading our papers, please contact us at subscribers@cepr.org
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Charles F. Manski, 2013. "Response to the Review of ‘Public Policy in an Uncertain World’," Economic Journal, Royal Economic Society, vol. 0, pages 412-415, August.
    2. Dahm, Matthias & González, Paula & Porteiro, Nicolás, 2009. "Trials, tricks and transparency: How disclosure rules affect clinical knowledge," Journal of Health Economics, Elsevier, vol. 28(6), pages 1141-1153, December.
    3. Emeric Henry, 2009. "Strategic Disclosure of Research Results: The Cost of Proving Your Honesty," Economic Journal, Royal Economic Society, vol. 119(539), pages 1036-1064, July.
    4. Mario Forni & Luca Gambetti & Marco Lippi & Luca Sala, 2017. "Noise Bubbles," Economic Journal, Royal Economic Society, vol. 127(604), pages 1940-1976, September.
    5. Emir Kamenica & Matthew Gentzkow, 2011. "Bayesian Persuasion," American Economic Review, American Economic Association, vol. 101(6), pages 2590-2615, October.
    6. Anton Kolotilin & Tymofiy Mylovanov & Andriy Zapechelnyuk & Ming Li, 2017. "Persuasion of a Privately Informed Receiver," Econometrica, Econometric Society, vol. 85(6), pages 1949-1964, November.
    7. Hunt Allcott, 2015. "Site Selection Bias in Program Evaluation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(3), pages 1117-1165.
    8. Denton, Frank T, 1985. "Data Mining as an Industry," The Review of Economics and Statistics, MIT Press, vol. 67(1), pages 124-127, February.
    9. Benjamin A. Olken, 2015. "Promises and Perils of Pre-analysis Plans," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 61-80, Summer.
    10. Luis Rayo & Ilya Segal, 2010. "Optimal Information Disclosure," Journal of Political Economy, University of Chicago Press, vol. 118(5), pages 949-987.
    11. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    12. Isabelle Brocas & Juan D. Carrillo, 2007. "Influence through ignorance," RAND Journal of Economics, RAND Corporation, vol. 38(4), pages 931-947, December.
    13. Aleksey Tetenov, 2016. "An economic theory of statistical testing," CeMMAP working papers CWP50/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    14. Nicola Lacetera & Lorenzo Zirulia, 2011. "The Economics of Scientific Misconduct," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 27(3), pages 568-603.
    15. Manski, Charles F., 2013. "Public Policy in an Uncertain World: Analysis and Decisions," Economics Books, Harvard University Press, number 9780674066892, Spring.
    16. Luke Holman & Megan L Head & Robert Lanfear & Michael D Jennions, 2015. "Evidence of Experimental Bias in the Life Sciences: Why We Need Blind Data Recording," PLOS Biology, Public Library of Science, vol. 13(7), pages 1-12, July.
    17. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    18. Mike Felgenhauer & Elisabeth Schulte, 2014. "Strategic Private Experimentation," American Economic Journal: Microeconomics, American Economic Association, vol. 6(4), pages 74-105, November.
    19. Jacob Glazer & Ariel Rubinstein, 2004. "On Optimal Rules of Persuasion," Econometrica, Econometric Society, vol. 72(6), pages 1715-1736, November.
    20. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    21. Zacharias Maniadis & Fabio Tufano & John A. List, 2014. "One Swallow Doesn't Make a Summer: New Evidence on Anchoring Effects," American Economic Review, American Economic Association, vol. 104(1), pages 277-290, January.
    22. Michael J. Fishman & Kathleen M. Hagerty, 1990. "The Optimal Amount of Discretion to Allow in Disclosure," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 105(2), pages 427-444.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aleksey Tetenov, 2016. "An economic theory of statistical testing," CeMMAP working papers CWP50/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Eliaz, Kfir & Spiegler, Ran & Weiss, Yair, 2019. "Cheating with (recursive) models," CEPR Discussion Papers 14100, C.E.P.R. Discussion Papers.
    3. Maximilian Kasy & Jann Spiess, 2022. "Rationalizing Pre-Analysis Plans:Statistical Decisions Subject to Implementability," Economics Series Working Papers 975, University of Oxford, Department of Economics.
    4. Alonso, Ricardo & Câmara, Odilon, 2021. "Organizing Data Analytics," CEPR Discussion Papers 16768, C.E.P.R. Discussion Papers.
    5. Michel Abramowicz & Ariane Szafarz, 2019. "Ethics of Randomized Controlled Trials: Should Economists Care about Equipoise?," Working Papers CEB 19-017, ULB -- Universite Libre de Bruxelles.
    6. Herresthal, C., 2017. "Hidden Testing and Selective Disclosure of Evidence," Cambridge Working Papers in Economics 1712, Faculty of Economics, University of Cambridge.
    7. So, Tony, 2020. "Classroom experiments as a replication device," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 86(C).
    8. Dahm, Matthias & González, Paula & Porteiro, Nicolás, 2018. "The enforcement of mandatory disclosure rules," Journal of Public Economics, Elsevier, vol. 167(C), pages 21-32.
    9. Maximilian Kasy & Jann Spiess, 2022. "Optimal Pre-Analysis Plans: Statistical Decisions Subject to Implementability," Papers 2208.09638, arXiv.org, revised Oct 2023.
    10. Zacharias Maniadis & Fabio Tufano & John A. List, 2017. "To Replicate or Not To Replicate? Exploring Reproducibility in Economics through the Lens of a Model and a Pilot Study," Economic Journal, Royal Economic Society, vol. 127(605), pages 209-235, October.
    11. Jeremy Bertomeu & Davide Cianciaruso, 2018. "Verifiable disclosure," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 65(4), pages 1011-1044, June.
    12. Herresthal, Claudia, 2022. "Hidden testing and selective disclosure of evidence," Journal of Economic Theory, Elsevier, vol. 200(C).
    13. Felgenhauer, Mike, 2021. "Experimentation and manipulation with preregistration," Games and Economic Behavior, Elsevier, vol. 130(C), pages 400-408.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alfredo Di Tillio & Marco Ottaviani & Peter Norman Sørensen, 2021. "Strategic Sample Selection," Econometrica, Econometric Society, vol. 89(2), pages 911-953, March.
    2. Susan Athey & Guido Imbens, 2016. "The Econometrics of Randomized Experiments," Papers 1607.00698, arXiv.org.
    3. Tsakas, Elias & Tsakas, Nikolas, 2021. "Noisy persuasion," Games and Economic Behavior, Elsevier, vol. 130(C), pages 44-61.
    4. Felgenhauer, Mike, 2021. "Experimentation and manipulation with preregistration," Games and Economic Behavior, Elsevier, vol. 130(C), pages 400-408.
    5. Alex Eble & Peter Boone & Diana Elbourne, 2017. "On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics," The World Bank Economic Review, World Bank, vol. 31(3), pages 687-707.
    6. Gregor Martin, 2015. "To Invite or Not to Invite a Lobby, That Is the Question," The B.E. Journal of Theoretical Economics, De Gruyter, vol. 15(2), pages 143-166, July.
    7. Thomas Mariotti & Nikolaus Schweizer & Nora Szech & Jonas von Wangenheim, 2023. "Information Nudges and Self-Control," Management Science, INFORMS, vol. 69(4), pages 2182-2197, April.
    8. Susan Athey & Raj Chetty & Guido Imbens, 2020. "Combining Experimental and Observational Data to Estimate Treatment Effects on Long Term Outcomes," Papers 2006.09676, arXiv.org.
    9. Daniel Stone, 2011. "A signal-jamming model of persuasion: interest group funded policy research," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 37(3), pages 397-424, September.
    10. Chan, Jimmy & Gupta, Seher & Li, Fei & Wang, Yun, 2019. "Pivotal persuasion," Journal of Economic Theory, Elsevier, vol. 180(C), pages 178-202.
      • Jimmy Chan & Seher Gupta & Fei Li & Yun Wang, 2018. "Pivotal Persuasion," Working Papers 2018-11-03, Wang Yanan Institute for Studies in Economics (WISE), Xiamen University.
    11. Martin Gregor, 2014. "Receiver's access fee for a single sender," Working Papers IES 2014/17, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised May 2014.
    12. Kolotilin, Anton, 2015. "Experimental design to persuade," Games and Economic Behavior, Elsevier, vol. 90(C), pages 215-226.
    13. Florian Hoffmann & Roman Inderst & Marco Ottaviani, 2013. "Hypertargeting, Limited Attention, and Privacy: Implications for Marketing and Campaigning," Working Papers 479, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    14. Florian Hoffmann & Roman Inderst & Marco Ottaviani, 2020. "Persuasion Through Selective Disclosure: Implications for Marketing, Campaigning, and Privacy Regulation," Management Science, INFORMS, vol. 66(11), pages 4958-4979, November.
    15. Hedlund, Jonas, 2017. "Bayesian persuasion by a privately informed sender," Journal of Economic Theory, Elsevier, vol. 167(C), pages 229-268.
    16. Abel Brodeur & Mathias Lé & Marc Sangnier & Yanos Zylberberg, 2016. "Star Wars: The Empirics Strike Back," American Economic Journal: Applied Economics, American Economic Association, vol. 8(1), pages 1-32, January.
    17. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    18. Ozan Candogan & Philipp Strack, 2021. "Optimal Disclosure of Information to a Privately Informed Receiver," Papers 2101.10431, arXiv.org, revised Jan 2022.
    19. Raphael Boleslavsky & Christopher Cotton, 2018. "Limited capacity in project selection: competition through evidence production," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 65(2), pages 385-421, March.
    20. Mike Felgenhauer, 2019. "Endogenous Persuasion with Costly Verification," Scandinavian Journal of Economics, Wiley Blackwell, vol. 121(3), pages 1054-1087, July.

    More about this item

    JEL classification:

    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cpr:ceprdp:11343. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://www.cepr.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.