IDEAS home Printed from https://ideas.repec.org/a/nat/nature/v554y2018i7693d10.1038_d41586-018-02108-9.html
   My bibliography  Save this article

How to make replication the norm

Author

Listed:
  • Paul Gertler
  • Sebastian Galiani
  • Mauricio Romero

Abstract

The publishing system builds in resistance to replication. Paul Gertler, Sebastian Galiani and Mauricio Romero surveyed economics journals to find out how to fix it.

Suggested Citation

  • Paul Gertler & Sebastian Galiani & Mauricio Romero, 2018. "How to make replication the norm," Nature, Nature, vol. 554(7693), pages 417-419, February.
    • Handle: RePEc:nat:nature:v:554:y:2018:i:7693:d:10.1038_d41586-018-02108-9
    DOI: 10.1038/d41586-018-02108-9
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/d41586-018-02108-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1038/d41586-018-02108-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bistline, John E.T. & Merrick, James H., 2020. "Parameterizing open-source energy models: Statistical learning to estimate unknown power plant attributes," Applied Energy, Elsevier, vol. 269(C).
    2. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    3. Reed, W. Robert, 2019. "Takeaways from the special issue on The Practice of Replication," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 13, pages 1-11.
    4. Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021. "Non-Standard Errors," Documents de travail du Centre d'Economie de la Sorbonne 21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian Brownlees & Javier Gil-Bazo, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Díaz & Menachem Abudy & To, 2021. "Non-Standard Errors," Working Paper Series, Social and Economic Sciences 2021-11, Faculty of Social and Economic Sciences, Karl-Franzens-University Graz.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
      • Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
      • Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
      • Moinas, Sophie & Declerck, Fany & Menkveld, Albert J. & Dreber, Anna, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
      • Ferrara, Gerardo & Jurkatis, Simon, 2021. "Non-standard errors," Bank of England working papers 955, Bank of England.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Dí­az & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Janeway Institute Working Papers 2112, Faculty of Economics, University of Cambridge.
      • Wolff, Christian & Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, C.E.P.R. Discussion Papers.
    5. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," I4R Discussion Paper Series 38, The Institute for Replication (I4R).
    6. So, Tony, 2020. "Classroom experiments as a replication device," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 86(C).
    7. Fišar, Miloš & Greiner, Ben & Huber, Christoph & Katok, Elena & Ozkes, Ali & Management Science Reproducibility Collaboration, 2023. "Reproducibility in Management Science," Department for Strategy and Innovation Working Paper Series 03/2023, WU Vienna University of Economics and Business.
    8. Brodeur, Abel & Esterling, Kevin & Ankel-Peters, Jörg & Bueno, Natália S. & Desposato, Scott & Dreber, Anna & Genovese, Federica & Green, Donald P. & Hepplewhite, Matthew & Hoces de la Guardia, Fernan, 2024. "Promoting Reproducibility and Replicability in Political Science," I4R Discussion Paper Series 100, The Institute for Replication (I4R).
    9. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    10. Fernando Hoces de la Guardia & Sean Grant & Edward Miguel, 2021. "A framework for open policy analysis," Science and Public Policy, Oxford University Press, vol. 48(2), pages 154-163.
    11. Christophe Pérignon & Olivier Akmansoy & Christophe Hurlin & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johanneson & Michael Kirchler & Albert Menkveld & Michael Razen & Utz Weitzel, 2022. "Reproducibility of Empirical Results: Evidence from 1,000 Tests in Finance," Working Papers hal-03810013, HAL.
    12. Edward Miguel, 2021. "Evidence on Research Transparency in Economics," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 193-214, Summer.
    13. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
    14. Chan, Shu Wing & Schilizzi, Steven & Iftekhar, Md Sayed & Da Silva Rosa, Raymond, 2019. "Web-based experimental economics software: How do they compare to desirable features?," Journal of Behavioral and Experimental Finance, Elsevier, vol. 23(C), pages 138-160.
    15. Brinkerink, Jasper & De Massis, Alfredo & Kellermanns, Franz, 2022. "One finding is no finding: Toward a replication culture in family business research," Journal of Family Business Strategy, Elsevier, vol. 13(4).
    16. Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
      • Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Díaz & Menachem Abudy & To, 2021. "Non-Standard Errors," Working Paper Series, Social and Economic Sciences 2021-11, Faculty of Social and Economic Sciences, Karl-Franzens-University Graz.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Dí­az & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
    17. Gauri, Varun & Jamison, Julian C. & Mazar, Nina & Ozier, Owen, 2021. "Motivating bureaucrats through social recognition: External validity—A tale of two states," Organizational Behavior and Human Decision Processes, Elsevier, vol. 163(C), pages 117-131.
    18. Robert Kirkby, 2023. "Quantitative Macroeconomics: Lessons Learned from Fourteen Replications," Computational Economics, Springer;Society for Computational Economics, vol. 61(2), pages 875-896, February.
    19. Kai Hockerts & Cory Searcy, 2023. "How to Sharpen Our Discourse on Corporate Sustainability and Business Ethics—A View from the Section Editors," Journal of Business Ethics, Springer, vol. 187(2), pages 225-235, October.
    20. Moore, Don A. & Thau, Stefan & Zhong, Chenbo & Gino, Francesca, 2022. "Open Science at OBHDP," Organizational Behavior and Human Decision Processes, Elsevier, vol. 168(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nature:v:554:y:2018:i:7693:d:10.1038_d41586-018-02108-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.