IDEAS home Printed from https://ideas.repec.org/a/bla/ecinqu/v63y2025i2p357-386.html

Replication code availability over time and across fields: Evidence from the German Socio‐Economic Panel

Author

Listed:
  • Lukas Fink
  • Jan Marcus

Abstract

Providing replication code is an inexpensive way to facilitate reproducibility. However, little is known about the extent of replication code provision. Therefore, we examine the availability of replication code for over 2500 peer‐reviewed articles based on the German Socio‐Economic Panel (SOEP), one of the most widely used datasets in economics and other social sciences. We find that only 6% of SOEP‐based studies have code available, but that this proportion has increased sharply over time. We provide evidence that the increase in code provision is driven by technological advances, individual researcher initiatives, and journal policies.

Suggested Citation

  • Lukas Fink & Jan Marcus, 2025. "Replication code availability over time and across fields: Evidence from the German Socio‐Economic Panel," Economic Inquiry, Western Economic Association International, vol. 63(2), pages 357-386, April.
    • Handle: RePEc:bla:ecinqu:v:63:y:2025:i:2:p:357-386
    DOI: 10.1111/ecin.13267
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/ecin.13267
    Download Restriction: no
    File URL: https://libkey.io/10.1111/ecin.13267?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. repec:plo:pbio00:2006930 is not listed on IDEAS
    2. McCullough, B. D. & McGeary, Kerry Anne & Harrison, Teresa D., 2006. "Lessons from the JMCB Archive," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(4), pages 1093-1107, June.
    3. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    4. Thomas Herndon & Michael Ash & Robert Pollin, 2014. "Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 38(2), pages 257-279.
    5. Goebel, Jan & Grabka, Markus M. & Liebig, Stefan & Kroh, Martin & Richter, David & Schröder, Carsten & Schupp, Jürgen, 2019. "The German Socio-Economic Panel (SOEP)," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 239, pages 345-360.
    6. Andrew C. Chang & Phillip Li, 2017. "A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time," American Economic Review, American Economic Association, vol. 107(5), pages 60-64, May.
    7. Edward Miguel, 2021. "Evidence on Research Transparency in Economics," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 193-214, Summer.
    8. Garret Christensen & Allan Dafoe & Edward Miguel & Don A Moore & Andrew K Rose, 2019. "A study of the impact of data sharing on article citations using journal policies as a natural experiment," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-13, December.
    9. Frank Mueller‐Langer & Richard Watt, 2018. "How Many More Cites Is A $3,000 Open Access Fee Buying You? Empirical Evidence From A Natural Experiment," Economic Inquiry, Western Economic Association International, vol. 56(2), pages 931-954, April.
    10. John J. Donohue III & Steven D. Levitt, 2001. "The Impact of Legalized Abortion on Crime," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(2), pages 379-420.
    11. Vilhuber, Lars, 2023. "Reproducibility and transparency versus privacy and confidentiality: Reflections from a data editor," Journal of Econometrics, Elsevier, vol. 235(2), pages 2285-2294.
    12. Abel Brodeur & Nikolai M. Cook & Jonathan S. Hartley & Anthony Heyes, 2024. "Do Preregistration and Preanalysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(3), pages 527-561.
    13. Nicholas Swanson & Garret Christensen & Rebecca Littman & David Birke & Edward Miguel & Elizabeth Levy Paluck & Zenan Wang, 2020. "Research Transparency Is on the Rise in Economics," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 61-65, May.
    14. Miloš Fišar & Ben Greiner & Christoph Huber & Elena Katok & Ali I. Ozkes, 2024. "Reproducibility in Management Science," Management Science, INFORMS, vol. 70(3), pages 1343-1356, March.
    15. Tom E. Hardwicke & John P. A. Ioannidis, 2018. "Mapping the universe of registered reports," Nature Human Behaviour, Nature, vol. 2(11), pages 793-796, November.
    16. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    17. Alberto Alesina & Nichola Fuchs Schuendeln, 2005. "Good bye Lenin (or not?): The Effect of Communism on People's Preferences," NBER Working Papers 11700, National Bureau of Economic Research, Inc.
    18. MARK J. McCABE & CHRISTOPHER M. SNYDER, 2014. "Identifying The Effect Of Open Access On Citations Using A Panel Of Science Journals," Economic Inquiry, Western Economic Association International, vol. 52(4), pages 1284-1300, October.
    19. Carmen M. Reinhart & Kenneth S. Rogoff, 2010. "Growth in a Time of Debt," American Economic Review, American Economic Association, vol. 100(2), pages 573-578, May.
    20. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 48(1), pages 62-83.
    21. Nosek, BA & Alter, G & Banks, GC & Borsboom, D & Bowman, SD & Breckler, SJ & Buck, S & Chambers, CD & Chin, G & Christensen, G & Contestabile, M & Dafoe, A & Eich, E & Freese, J & Glennerster, R & Gor, 2015. "Promoting an open research culture," Department of Economics, Working Paper Series qt7wh1000s, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    22. Joel Ferguson & Rebecca Littman & Garret Christensen & Elizabeth Levy Paluck & Nicholas Swanson & Zenan Wang & Edward Miguel & David Birke & John-Henry Pezzuto, 2023. "Survey of open science practices and attitudes in the social sciences," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    23. Christopher L. Foote & Christopher F. Goetz, 2008. "The Impact of Legalized Abortion on Crime: Comment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(1), pages 407-423.
    24. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    25. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    26. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    27. Monya Baker, 2016. "1,500 scientists lift the lid on reproducibility," Nature, Nature, vol. 533(7604), pages 452-454, May.
    28. Cristina Blanco-Perez & Abel Brodeur, 2020. "Publication Bias and Editorial Statement on Negative Findings," The Economic Journal, Royal Economic Society, vol. 130(629), pages 1226-1247.
    29. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    30. Benedikt Fecher & Sascha Friesike & Marcel Hebing, 2015. "What Drives Academic Data Sharing?," PLOS ONE, Public Library of Science, vol. 10(2), pages 1-25, February.
    31. Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad‐Díaz & Menachem (Meni) Abudy , 2024. "Nonstandard Errors," Journal of Finance, American Finance Association, vol. 79(3), pages 2339-2390, June.
      • Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021. "Non-Standard Errors," Documents de travail du Centre d'Economie de la Sorbonne 21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
      • Christian Brownlees & Anna Dreber & Javier Gil-Bazo & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Albert J. Menkveld & Sebastian Neussüs & Michael Razen & Utz Weitzel, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
      • Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neus ss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Fr mmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
      • Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
      • Menkveld, Albert J. & Dreber, Anna & Declerck, Fany & Moinas, Sophie, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüß, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Menac, 2024. "Nonstandard errors," LSE Research Online Documents on Economics 123002, London School of Economics and Political Science, LSE Library.
      • Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
      • Albert Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad-Díaz & Tobias Adrian & Yacine Ai, 2024. "Nonstandard Errors," Post-Print hal-04676112, HAL.
      • Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
      • Gerardo Ferrara & Simon Jurkatis, 2021. "Non-standard errors," Bank of England Staff Working Paper series 955, Bank of England.
      • Albert Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad-Díaz & Tobias Adrian & Yacine Ai, 2024. "Nonstandard Errors," Post-Print hal-05077550, HAL.
      • Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Dí­az & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
      • Albert Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad-Díaz & Tobias Adrian & Yacine Ai, 2024. "Nonstandard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-05077550, HAL.
      • Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
      • Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
      • Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Menkveld, Albert J. & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz & Wolff, Christian, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, Centre for Economic Policy Research.
    32. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    33. Paul Gertler & Sebastian Galiani & Mauricio Romero, 2018. "How to make replication the norm," Nature, Nature, vol. 554(7693), pages 417-419, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bensch, Gunther & Rose, Julian & Brodeur, Abel & Ankel-Peters, Jörg, 2025. "The Robustness Dashboard," I4R Discussion Paper Series 234, The Institute for Replication (I4R).
      • Bensch, Gunther & Rose, Julian & Brodeur, Abel & Ankel-Peters, Jörg, 2025. "The robustness dashboard," Ruhr Economic Papers 1167, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    2. Marcus, Jan, 2025. "Replication code as a cornerstone of the credibility revolution 2.0," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 117(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Balafoutas, Loukas & Celse, Jeremy & Karakostas, Alexandros & Umashev, Nicholas, 2025. "Incentives and the replication crisis in social sciences: A critical review of open science practices," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 114(C).
    2. Christophe Hurlin & Jean-Edouard Colliard & Christophe Pérignon, 2024. "The Economics of Computational Reproducibility," Working Papers hal-04759240, HAL.
    3. Anna Dreber & Magnus Johannesson, 2025. "A framework for evaluating reproducibility and replicability in economics," Economic Inquiry, Western Economic Association International, vol. 63(2), pages 338-356, April.
    4. Christophe Pérignon & Olivier Akmansoy & Christophe Hurlin & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johanneson & Michael Kirchler & Albert J Menkveld & Michael Razen & Utz Weitzel, 2022. "Reproducibility of Empirical Results: Evidence from 1,000 Tests in Finance," Working Papers hal-03810013, HAL.
    5. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    6. Bruno Ferman & Lucas Finamor, 2025. "There must be an error here! Experimental evidence on coding errors' biases," Papers 2508.20069, arXiv.org, revised Sep 2025.
    7. Jörg Ankel‐Peters & Nathan Fiala & Florian Neubauer, 2025. "Is economics self‐correcting? Replications in the American Economic Review," Economic Inquiry, Western Economic Association International, vol. 63(2), pages 463-485, April.
    8. repec:osf:osfxxx:mydzv_v1 is not listed on IDEAS
    9. Miloš Fišar & Ben Greiner & Christoph Huber & Elena Katok & Ali I. Ozkes, 2024. "Reproducibility in Management Science," Management Science, INFORMS, vol. 70(3), pages 1343-1356, March.
    10. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 48(1), pages 62-83.
    11. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    12. Campbell, Douglas & Brodeur, Abel & Dreber, Anna & Johannesson, Magnus & Kopecky, Joseph & Lusher, Lester & Tsoy, Nikita, 2024. "The Robustness Reproducibility of the American Economic Review," I4R Discussion Paper Series 124, The Institute for Replication (I4R).
    13. Gabriel Bayle & Dimitri Dubois & Marc Willinger, 2025. "L' économie à l'ère de la science ouverte : un nouvel élan pour la reproductibilité," Post-Print hal-05173859, HAL.
    14. Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre.
    15. Tom L. Dudda & Lars Hornuf, 2025. "The Perks and Perils of Machine Learning in Business and Economic Research," CESifo Working Paper Series 11721, CESifo.
    16. Edward Miguel, 2021. "Evidence on Research Transparency in Economics," Journal of Economic Perspectives, American Economic Association, vol. 35(3), pages 193-214, Summer.
    17. Christophe Pérignon & Olivier Akmansoy & Christophe Hurlin & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Albert J Menkveld & Michael Razen & Utz Weitzel, 2024. "Computational Reproducibility in Finance: Evidence from 1,000 Tests," The Review of Financial Studies, Society for Financial Studies, vol. 37(11), pages 3558-3593.
    18. Joëts, Marc & Mignon, Valérie, 2026. "Slaying the undead: How long does it take to kill zombie papers?," Research Policy, Elsevier, vol. 55(2).
    19. Zohid Askarov & Anthony Doucouli & Hristos Doucouli & T D Stanley, 2023. "The Significance of Data-Sharing Policy," Journal of the European Economic Association, European Economic Association, vol. 21(3), pages 1191-1226.
    20. Valérie Orozco & Christophe Bontemps & Elise Maigné & Virginie Piguet & Annie Hofstetter & Anne Lacroix & Fabrice Levert & Jean‐Marc Rousselle, 2020. "How To Make A Pie: Reproducible Research For Empirical Economics And Econometrics," Journal of Economic Surveys, Wiley Blackwell, vol. 34(5), pages 1134-1169, December.
    21. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:ecinqu:v:63:y:2025:i:2:p:357-386. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/weaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.