IDEAS home Printed from https://ideas.repec.org/p/dpr/wpaper/1283.html

Pre-Registration and Pre-Analysis Plans in Experimental Economics

Author

Listed:
  • Taisuke Imai
  • Séverine Toussaert
  • Aurélien Baillon
  • Anna Dreber
  • Seda Ertaç
  • Magnus Johannesson
  • Levent Neyse
  • Marie Claire Villeval

Abstract

The open science movement has gained significant momentum over the past decade, with pre-registration and the use of pre-analysis plans being central to ongoing debates. Combining observational evidence on trends in adoption with survey data from 519 researchers, this study examines the adoption of pre-registration (potentially but not necessarily including pre-analysis plans) in experimental economics. Pooling statistics from 19 leading journals published between 2017 and 2023, we observe that the number of papers containing a pre-registration grew from seven per year to 190 per year. Our findings indicate that pre-registration has now become mainstream in experimental economics, with two-thirds of respondents expressing favorable views and 86% having pre-registered at least one study. However, opinions are divided on the scope and comprehensiveness of pre-registration, highlighting the need for clearer guidelines. Researchers assign a credibility premium to pre-registered tests, although the exact channels remain to be understood. Our results suggest growing support for open science practices among experimental economists, with demand for professional associations to guide researchers and reviewers on best practices for pre-registration and other open science initiatives.

Suggested Citation

  • Taisuke Imai & Séverine Toussaert & Aurélien Baillon & Anna Dreber & Seda Ertaç & Magnus Johannesson & Levent Neyse & Marie Claire Villeval, 2025. "Pre-Registration and Pre-Analysis Plans in Experimental Economics," ISER Discussion Paper 1283, Institute of Social and Economic Research, The University of Osaka.
    • Handle: RePEc:dpr:wpaper:1283
    as

    Download full text from publisher

    File URL: https://www.iser.osaka-u.ac.jp/static/resources/docs/dp/DP1283.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joel Ferguson & Rebecca Littman & Garret Christensen & Elizabeth Levy Paluck & Nicholas Swanson & Zenan Wang & Edward Miguel & David Birke & John-Henry Pezzuto, 2023. "Survey of open science practices and attitudes in the social sciences," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    2. Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018. "Redefine statistical significance," Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
      • Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.
    3. Abhijit Banerjee & Esther Duflo & Amy Finkelstein & Lawrence F. Katz & Benjamin A. Olken & Anja Sautmann, 2020. "In Praise of Moderation: Suggestions for the Scope and Use of Pre-Analysis Plans for RCTs in Economics," NBER Working Papers 26993, National Bureau of Economic Research, Inc.
    4. Marjan Bakker & Coosje L S Veldkamp & Marcel A L M van Assen & Elise A V Crompvoets & How Hwee Ong & Brian A Nosek & Courtney K Soderberg & David Mellor & Jelte M Wicherts, 2020. "Ensuring the quality and specificity of preregistrations," PLOS Biology, Public Library of Science, vol. 18(12), pages 1-18, December.
    5. repec:plo:pone00:0059910 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Maniadis, Zacharias, 2025. "Best practices for reproducibility, research assessment reforms, and implications for experimental economists," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 119(C).
    2. Reca Sarfati & Vod Vilfort, 2025. ""Post" Pre-Analysis Plans: Valid Inference for Non-Preregistered Specifications," Papers 2510.02507, arXiv.org.
    3. Rahal, Rima-Maria, 2025. "Advancing openness in economic research through the lens of behavioral and experimental economics," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 119(C).
    4. Marcus, Jan, 2025. "Replication code as a cornerstone of the credibility revolution 2.0," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 117(C).
    5. Klaus M. Schmidt & Levent Neyse & Marianne Saam & Doreen Siegfried & Lars Vilhuber & Joachim Winter, 2026. "Open Science in den Wirtschaftswissenschaften: Transparenz, Reproduzierbarkeit und Zugang," Rationality and Competition Discussion Paper Series 570, CRC TRR 190 Rationality and Competition.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Balafoutas, Loukas & Celse, Jeremy & Karakostas, Alexandros & Umashev, Nicholas, 2025. "Incentives and the replication crisis in social sciences: A critical review of open science practices," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 114(C).
    2. Heckelei, Thomas & Huettel, Silke & Odening, Martin & Rommel, Jens, "undated". "The replicability crisis and the p-value debate – what are the consequences for the agricultural and food economics community?," Discussion Papers 316369, University of Bonn, Institute for Food and Resource Economics.
    3. Kathryn N. Vasilaky & J. Michelle Brock, 2020. "Power(ful) guidelines for experimental economists," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 6(2), pages 189-212, December.
    4. Felix Holzmeister & Magnus Johannesson & Robert Böhm & Anna Dreber & Jürgen Huber & Michael Kirchler, 2023. "Heterogeneity in effect size estimates: Empirical evidence and practical implications," Working Papers 2023-17, Faculty of Economics and Statistics, Universität Innsbruck.
    5. Mayo, Deborah, 2024. "Error statistics, Bayes-factor Tests and the Fallacy of Non-exhaustive Alternatives," OSF Preprints tmgqd, Center for Open Science.
    6. Dreber, Anna & Heikensten, Emma & Säve-Söderbergh, Jenny, 2022. "Why do women ask for less?," Labour Economics, Elsevier, vol. 78(C).
    7. Ardila Brenøe, Anne & Eyibak, Zeynep & Heursen, Lea & Ranehill, Eva & Weber, Roberto A., 2024. "Gender Identity and Economic Decision Making," Working Papers 2024:6, Lund University, Department of Economics.
    8. Buchanan, Erin Michelle & Scofield, John E., 2017. "Bulletproof Bias? Considering the Type of Data in Common Proportion of Variance Effect Sizes," OSF Preprints cs4vy, Center for Open Science.
    9. John A. List, 2024. "Optimally generate policy-based evidence before scaling," Nature, Nature, vol. 626(7999), pages 491-499, February.
    10. Samahita, Margaret & Holm, Håkan J., 2020. "Mining for Mood Effect in the Field," Working Papers 2020:2, Lund University, Department of Economics.
    11. Fossen, Frank M. & Neyse, Levent & Johannesson, Magnus & Dreber Almenberg, Anna, 2020. "2D:4D and Self-Employment Using SOEP Data: A Replication Study," IZA Discussion Papers 13180, IZA Network @ LISER.
    12. Jonas Vandenbruaene & Marc De Ceuster & Jan Annaert, 2022. "Efficient Spread Betting Markets: A Literature Review," Journal of Sports Economics, , vol. 23(7), pages 907-949, October.
    13. Huber, Christoph & Huber, Jürgen & Kirchler, Michael, 2021. "Market shocks and professionals’ investment behavior – Evidence from the COVID-19 crash," Journal of Banking & Finance, Elsevier, vol. 133(C).
    14. Gonzalez-Jimenez, David & Capozza, Francesco & Dirkmaat, Thomas & van de Veer, Evelien & van Druten, Amber & Baillon, Aurélien, 2025. "Falling and failing (to learn): Evidence from a nation-wide cybersecurity field experiment with SMEs," Journal of Economic Behavior & Organization, Elsevier, vol. 230(C).
    15. Vigren, Andreas & Pyddoke, Roger, 2020. "The impact on bus ridership of passenger incentive contracts in public transport," Transportation Research Part A: Policy and Practice, Elsevier, vol. 135(C), pages 144-159.
    16. Khuram Mirza, 2023. "An analysis of early marriage in Pakhtun society: determinants, consequences, and interventions," Journal of International Relations and Social Dynamics, 50sea, vol. 2(2), pages 53-63, August.
    17. Dudek, Thomas & Brenøe, Anne Ardila & Feld, Jan & Rohrer, Julia, 2022. "No Evidence That Siblings' Gender Affects Personality across Nine Countries," IZA Discussion Papers 15137, IZA Network @ LISER.
    18. Douglas Kiarelly Godoy de Araujo, 2025. "Open-sourced central bank macroeconomic models," IFC Bulletins chapters, in: Bank for International Settlements (ed.), Data science in central banking: enhancing the access to and sharing of data, volume 64, Bank for International Settlements.
    19. Patrick Vu, 2022. "Can the Replication Rate Tell Us About Publication Bias?," Papers 2206.15023, arXiv.org, revised Jul 2022.
    20. Grüner Sven, 2020. "Sample Size Calculation in Economic Experiments," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 240(6), pages 791-823, December.

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C80 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - General
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:dpr:wpaper:1283. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Librarian (email available below). General contact details of provider: https://edirc.repec.org/data/isosujp.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.