IDEAS home Printed from https://ideas.repec.org/a/eee/soceco/v119y2025ics2214804325001363.html

Best practices for reproducibility, research assessment reforms, and implications for experimental economists

Author

Listed:
  • Maniadis, Zacharias

Abstract

Scientists are under pressure to adhere to best practices for enhancing reproducibility, such as preregistration and data sharing. This tendency will certainly increase with the unfolding reforms in researcher assessment, and it brings new challenges. Heterogeneity in the amenability of different domains to reproducibility-enhancing practices raises an issue of possible inequity: will different scientific domains bear disparate adjustment costs? Is this justified and efficient? To illustrate the problem, we consider recent concerns expressed by experimental economists, namely that they are unfairly burdened relative to other economics domains. Our analysis indicates that such fairness concerns may have merit, but only insofar as research assessment does not fully internalize the costs of adjusting to new practices.

Suggested Citation

  • Maniadis, Zacharias, 2025. "Best practices for reproducibility, research assessment reforms, and implications for experimental economists," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 119(C).
  • Handle: RePEc:eee:soceco:v:119:y:2025:i:c:s2214804325001363
    DOI: 10.1016/j.socec.2025.102472
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S2214804325001363
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.socec.2025.102472?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Brodeur, Abel & Mikola, Derek & Cook, Nikolai & Brailey, Thomas & Briggs, Ryan & de Gendre, Alexandra & Dupraz, Yannick & Fiala, Lenka & Gabani, Jacopo & Gauriot, Romain & Haddad, Joanne & Federice, A, 2024. "Mass Reproducibility and Replicability: A New Hope," I4R Discussion Paper Series 107, The Institute for Replication (I4R).
    2. Taisuke Imai & Séverine Toussaert & Aurélien Baillon & Anna Drever & Seda Ertaç & Magnus Johannesson & Levent Neyse & Marie Claire Villeval, 2025. "Pre-Registration and Pre-Analysis Plans in Experimental Economics," Rationality and Competition Discussion Paper Series 530, CRC TRR 190 Rationality and Competition.
    3. Daniele Fanelli, 2013. "Redefine misconduct as distorted reporting," Nature, Nature, vol. 494(7436), pages 149-149, February.
    4. Herbert Dawid & Philipp Harting & Sander Hoog & Michael Neugart, 2019. "Macroeconomics with heterogeneous agent models: fostering transparency, reproducibility and replication," Journal of Evolutionary Economics, Springer, vol. 29(1), pages 467-538, March.
    5. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    6. David Moher & Lex Bouter & Sabine Kleinert & Paul Glasziou & Mai Har Sham & Virginia Barbour & Anne-Marie Coriat & Nicole Foeger & Ulrich Dirnagl, 2020. "The Hong Kong Principles for assessing researchers: Fostering research integrity," PLOS Biology, Public Library of Science, vol. 18(7), pages 1-14, July.
    7. Thomas Gall & John P A Ioannidis & Zacharias Maniadis, 2017. "The credibility crisis in research: Can economics tools help?," PLOS Biology, Public Library of Science, vol. 15(4), pages 1-13, April.
    8. Christopher D. Chambers & Loukia Tzavella, 2022. "The past, present and future of Registered Reports," Nature Human Behaviour, Nature, vol. 6(1), pages 29-42, January.
    9. Abel Brodeur & Nikolai M. Cook & Jonathan S. Hartley & Anthony Heyes, 2024. "Do Preregistration and Preanalysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(3), pages 527-561.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Balafoutas, Loukas & Celse, Jeremy & Karakostas, Alexandros & Umashev, Nicholas, 2025. "Incentives and the replication crisis in social sciences: A critical review of open science practices," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 114(C).
    2. Ferman, Bruno & Finamor, Lucas, 2025. "There must be an error here! Experimental evidence on coding errors' biases," I4R Discussion Paper Series 266, The Institute for Replication (I4R).
    3. Costanza Naguib, 2025. "Does single-blind review encourage or discourage p-hacking?," Diskussionsschriften dp2504, Universitaet Bern, Departement Volkswirtschaft.
    4. Rahal, Rima-Maria, 2025. "Advancing openness in economic research through the lens of behavioral and experimental economics," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 119(C).
    5. Deer, Lachlan & Adler, Susanne J. & Datta, Hannes & Mizik, Natalie & Sarstedt, Marko, 2025. "Toward open science in marketing research," International Journal of Research in Marketing, Elsevier, vol. 42(1), pages 212-233.
    6. Marcus, Jan, 2025. "Replication code as a cornerstone of the credibility revolution 2.0," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 117(C).
    7. Reca Sarfati & Vod Vilfort, 2025. ""Post" Pre-Analysis Plans: Valid Inference for Non-Preregistered Specifications," Papers 2510.02507, arXiv.org.
    8. Gall, Thomas & Maniadis, Zacharias, 2019. "Evaluating solutions to the problem of false positives," Research Policy, Elsevier, vol. 48(2), pages 506-515.
    9. Costanza Naguib, 2024. "P-hacking and Significance Stars," Diskussionsschriften dp2409, Universitaet Bern, Departement Volkswirtschaft.
    10. Michaela Strinzel & Josh Brown & Wolfgang Kaltenbrunner & Sarah Rijcke & Michael Hill, 2021. "Ten ways to improve academic CVs for fairer research assessment," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 8(1), pages 1-4, December.
    11. Bergemann, Dirk & Ottaviani, Marco, 2021. "Information Markets and Nonmarkets," CEPR Discussion Papers 16459, C.E.P.R. Discussion Papers.
    12. repec:osf:metaar:jt5zf_v1 is not listed on IDEAS
    13. Rose, Julian & Neubauer, Florian & Ankel-Peters, Jörg, 2024. "Long-Term Effects of the Targeting the Ultra-Poor Program - A Reproducibility and Replicability Assessment of Banerjee et al. (2021)," I4R Discussion Paper Series 142, The Institute for Replication (I4R).
    14. Silvano Cincotti & Marco Raberto & Andrea Teglio, 2022. "Why do we need agent-based macroeconomics?," Review of Evolutionary Political Economy, Springer, vol. 3(1), pages 5-29, April.
    15. Chuang, Shih-Hsien & Holian, Matthew & Pattison, Nathaniel & Ramakrishnan, Prasanthi, 2024. "A Comment on "Populist Leaders and the Economy"," I4R Discussion Paper Series 157, The Institute for Replication (I4R).
    16. Orlando Gomes, 2021. "Hand-to-mouth consumers, rule-of-thumb savers, and optimal control," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 16(2), pages 229-263, April.
    17. Isidore Komla Zotoo & Guifeng Liu & Zhangping Lu & Frank Kofi Essien & Wencheng Su, 2023. "The Impact of Key Stakeholders and the Computer Skills of Librarians on Research Data Management Support Services (Id so-21-1893.r2)," SAGE Open, , vol. 13(3), pages 21582440231, September.
    18. Emma L Henderson & Christopher D Chambers, 2022. "Ten simple rules for writing a Registered Report," PLOS Computational Biology, Public Library of Science, vol. 18(10), pages 1-9, October.
    19. Terranova, Roberta & Turco, Enrico M., 2022. "Concentration, stagnation and inequality: An agent-based approach," Journal of Economic Behavior & Organization, Elsevier, vol. 193(C), pages 569-595.
    20. Fei Wang & Chaowen Zhu, 2023. "Statistical analysis of research integrity construction in 466 Chinese universities with medical programs," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 10(1), pages 1-8, December.
    21. Bonander, Carl & Hammar, Olle & Jakobsson, Niklas & Bensch, Gunther & Holzmeister, Felix & Brodeur, Abel, 2025. "“Try to Balance the Baseline”: A comment on “Parent–teacher meetings and student outcomes: Evidence from a developing country” by Islam (2019)," European Economic Review, Elsevier, vol. 175(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:soceco:v:119:y:2025:i:c:s2214804325001363. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/620175 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.