IDEAS home Printed from https://ideas.repec.org/p/gat/wpaper/2511.html
   My bibliography  Save this paper

Pre-Registration and Pre-Analysis Plans in Experimental Economics

Author

Listed:
  • Taisuke Imai

    (The University of Osaka)

  • Séverine Toussaert

    (University of Oxford)

  • Aurélien Baillon

    (emlyon business school, CNRS, Université Lumière Lyon 2, Université Jean-Monnet Saint-Etienne, GATE)

  • Anna Dreber

    (Stockholm School of Economics)

  • Seda Ertaç

    (Koç University)

  • Magnus Johannesson

    (Stockholm School of Economics)

  • Levent Neyse

    (WZB, DIW Berlin)

  • Marie Claire Villeval

    (CNRS, Université Lumière Lyon 2, Université Jean-Monnet Saint-Etienne, emlyon business school, GATE, IZA, Bonn, Germany)

Abstract

The open science movement has gained significant momentum over the past decade, with pre-registration and the use of pre-analysis plans being central to ongoing debates. Combining observational evidence on trends in adoption with survey data from 519 researchers, this study examines the adoption of pre-registration (potentially but not necessarily including pre-analysis plans) in experimental economics. Pooling statistics from 19 leading journals published between 2017 and 2023, we observe that the number of papers containing a pre-registration grew from seven per year to 190 per year. Our findings indicate that pre-registration has now become mainstream in experimental economics, with two-thirds of respondents expressing favorable views and 86% having pre-registered at least one study. However, opinions are divided on the scope and comprehensiveness of pre-registration, highlighting the need for clearer guidelines. Researchers assign a credibility premium to pre-registered tests, although the exact channels remain to be understood. Our results suggest growing support for open science practices among experimental economists, with demand for professional associations to guide researchers and reviewers on best practices for pre-registration and other open science initiatives.

Suggested Citation

  • Taisuke Imai & Séverine Toussaert & Aurélien Baillon & Anna Dreber & Seda Ertaç & Magnus Johannesson & Levent Neyse & Marie Claire Villeval, 2025. "Pre-Registration and Pre-Analysis Plans in Experimental Economics," Working Papers 2511, Groupe d'Analyse et de Théorie Economique Lyon St-Étienne (GATE Lyon St-Étienne), Université de Lyon.
    • Handle: RePEc:gat:wpaper:2511
    as

    Download full text from publisher

    File URL: https://www.gate.cnrs.fr/RePEc/2025/2511.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joel Ferguson & Rebecca Littman & Garret Christensen & Elizabeth Levy Paluck & Nicholas Swanson & Zenan Wang & Edward Miguel & David Birke & John-Henry Pezzuto, 2023. "Survey of open science practices and attitudes in the social sciences," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    2. Abhijit Banerjee & Esther Duflo & Amy Finkelstein & Lawrence F. Katz & Benjamin A. Olken & Anja Sautmann, 2020. "In Praise of Moderation: Suggestions for the Scope and Use of Pre-Analysis Plans for RCTs in Economics," NBER Working Papers 26993, National Bureau of Economic Research, Inc.
    3. Marjan Bakker & Coosje L S Veldkamp & Marcel A L M van Assen & Elise A V Crompvoets & How Hwee Ong & Brian A Nosek & Courtney K Soderberg & David Mellor & Jelte M Wicherts, 2020. "Ensuring the quality and specificity of preregistrations," PLOS Biology, Public Library of Science, vol. 18(12), pages 1-18, December.
    4. repec:plo:pone00:0059910 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Balafoutas, Loukas & Celse, Jeremy & Karakostas, Alexandros & Umashev, Nicholas, 2025. "Incentives and the replication crisis in social sciences: A critical review of open science practices," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 114(C).
    2. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, 2021. "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.
    3. Button, Patrick & Walker, Brigham, 2020. "Employment discrimination against Indigenous Peoples in the United States: Evidence from a field experiment," Labour Economics, Elsevier, vol. 65(C).
    4. Felix Holzmeister & Magnus Johannesson & Robert Böhm & Anna Dreber & Jürgen Huber & Michael Kirchler, 2023. "Heterogeneity in effect size estimates: Empirical evidence and practical implications," Working Papers 2023-17, Faculty of Economics and Statistics, Universität Innsbruck.
    5. repec:osf:socarx:rkyf7_v1 is not listed on IDEAS
    6. Blimpo, Moussa P. & Pugatch, Todd, 2023. "Unintended Consequences of Youth Entrepreneurship Programs: Experimental Evidence from Rwanda," GLO Discussion Paper Series 1332, Global Labor Organization (GLO).
    7. Fries, Tilman & Barron, Kai, 2023. "Narrative Persuasion," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277691, Verein für Socialpolitik / German Economic Association.
    8. Blesse, Sebastian, 2021. "Are your tax problems an opportunity not to pay taxes? Evidence from a randomized survey experiment," ZEW Discussion Papers 21-040, ZEW - Leibniz Centre for European Economic Research.
    9. Rose, Julian & Neubauer, Florian & Ankel-Peters, Jörg, 2024. "Long-Term Effects of the Targeting the Ultra-Poor Program - A Reproducibility and Replicability Assessment of Banerjee et al. (2021)," I4R Discussion Paper Series 142, The Institute for Replication (I4R).
    10. Lars Behlen & Oliver Himmler & Robert Jäckle, 2023. "Defaults and effortful tasks," Experimental Economics, Springer;Economic Science Association, vol. 26(5), pages 1022-1059, November.
    11. Jeffrey D. Michler & Dewan Abdullah Al Rafi & Jonathan Giezendanner & Anna Josephson & Valerien O. Pede & Elizabeth Tellman, 2024. "Impact Evaluations in Data Poor Settings: The Case of Stress-Tolerant Rice Varieties in Bangladesh," Papers 2409.02201, arXiv.org.
    12. Abel Brodeur & Nikolai M. Cook & Jonathan S. Hartley & Anthony Heyes, 2024. "Do Preregistration and Preanalysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(3), pages 527-561.
    13. Mattozzi, Andrea & Nocito, Samuel & Sobbrio, Francesco, 2022. "Fact-checking Politicians," CEPR Discussion Papers 17710, C.E.P.R. Discussion Papers.
    14. Maximilian Kasy & Jann Spiess, 2022. "Rationalizing Pre-Analysis Plans:Statistical Decisions Subject to Implementability," Economics Series Working Papers 975, University of Oxford, Department of Economics.
    15. Sarah A. Janzen & Jeffrey D. Michler, 2021. "Ulysses' pact or Ulysses' raft: Using pre‐analysis plans in experimental and nonexperimental research," Applied Economic Perspectives and Policy, John Wiley & Sons, vol. 43(4), pages 1286-1304, December.
    16. Abhijit Banerjee & Rema Hanna & Benjamin A Olken & Elan Satriawan & Sudarno Sumarto, "undated". "Pangan versus Kupon Makanan: Bukti dari Eksperimen Berskala Besar di Indonesia," Working Papers 3537, Communications Section.
    17. Thibaut Arpinon & Romain Espinosa, 2023. "A practical guide to Registered Reports for economists," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 9(1), pages 90-122, June.
    18. Dalton, Patricio & Rüschenpöhler, Julius & Uras, Burak & Zia, Bilal, 2020. "Curating Local Knowledge : Experimental Evidence from Small Retailers in Indonesia (Revision of DP 2019-015)," Other publications TiSEM 5c5fc2c6-7cfe-4fa9-b1d1-4, Tilburg University, School of Economics and Management.
    19. Eliot Abrams & Jonathan Libgober & John A. List, 2020. "Research Registries: Facts, Myths, and Possible Improvements," NBER Working Papers 27250, National Bureau of Economic Research, Inc.
    20. Laurel Wheeler & Robert Garlick & Eric Johnson & Patrick Shaw & Marissa Gargano, 2022. "LinkedIn(to) Job Opportunities: Experimental Evidence from Job Readiness Training," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 101-125, April.
    21. Diarmaid Ó Ceallaigh & Kirsten I.M. Rohde & Hans van Kippersluis, 2024. "Skipping your workout, again? Measuring and understanding time inconsistency in physical activity," Tinbergen Institute Discussion Papers 24-028/V, Tinbergen Institute.

    More about this item

    Keywords

    pre-registration; pre-analysis plans; experimental economics; open science;
    All these keywords.

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C80 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - General
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gat:wpaper:2511. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Nelly Wirth The email address of this maintainer does not seem to be valid anymore. Please ask Nelly Wirth to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/gateefr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.