IDEAS home Printed from https://ideas.repec.org/a/aea/aecrev/v107y2017i5p27-31.html
   My bibliography  Save this article

Assessing the Rate of Replication in Economics

Author

Listed:
  • James Berry
  • Lucas C. Coffman
  • Douglas Hanley
  • Rania Gihleb
  • Alistair J. Wilson

Abstract

We assess the rate of replication for empirical papers in the 2010 American Economic Review. Across 70 empirical papers, we find that 29 percent have 1 or more citation that partially replicates the original result. While only a minority of papers has a published replication, a majority (60 percent) have either a replication, robustness test, or an extension. Surveying authors within the literature, we find substantial uncertainty over the number of extant replications.

Suggested Citation

  • James Berry & Lucas C. Coffman & Douglas Hanley & Rania Gihleb & Alistair J. Wilson, 2017. "Assessing the Rate of Replication in Economics," American Economic Review, American Economic Association, vol. 107(5), pages 27-31, May.
    • Handle: RePEc:aea:aecrev:v:107:y:2017:i:5:p:27-31
    Note: DOI: 10.1257/aer.p20171119
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/articles?id=10.1257/aer.p20171119
    Download Restriction: no
    File URL: https://www.aeaweb.org/articles/attachments?retrieve=fvSE2uLbfTw2yfw1jwJfoUuZSmNU2hzP
    Download Restriction: no
    File URL: https://www.aeaweb.org/articles/attachments?retrieve=twd-hx8Cazs0DIquN7hW92jsOh3ASnAr
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    ---><---

    References listed on IDEAS

    as
    1. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Klege, Rebecca A. & Amuakwa-Mensah, Franklin & Visser, Martine, 2022. "Tenancy and energy choices in Rwanda. A replication and extension study," World Development Perspectives, Elsevier, vol. 26(C).
    2. Victor Gay, 2023. "Culture: An Empirical Investigation of Beliefs, Work, and Fertility. A Verification and Reproduction of Fernández and Fogli (2009)," Post-Print hal-04194417, HAL.
    3. Luca Gerotto & Paolo Pellizzari, 2021. "A replication of Pindyck’s willingness to pay: on the efforts required to obtain results," SN Business & Economics, Springer, vol. 1(5), pages 1-25, May.
    4. Grolleau, Gilles & Ibanez, Lisette & Mzoughi, Naoufel, 2020. "Moral judgment of environmental harm caused by a single versus multiple wrongdoers: A survey experiment," Ecological Economics, Elsevier, vol. 170(C).
    5. Bensch, Gunther & Gotz, Gunnar & Peters, Jörg, 2020. "Effects of rural electrification on employment: A comment on Dinkelman (2011)," Ruhr Economic Papers 840, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Chowdhury, Rosen & Cook, Steve & Watson, Duncan, 2023. "Reconsidering the relationship between health and income in the UK," Social Science & Medicine, Elsevier, vol. 332(C).
    7. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    8. Christian Dippel & Michael Poyker, 2019. "How Common are Electoral Cycles in Criminal Sentencing?," NBER Working Papers 25716, National Bureau of Economic Research, Inc.
    9. Roy Chen & Yan Chen & Yohanes E. Riyanto, 2021. "Best practices in replication: a case study of common information in coordination games," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 2-30, March.
    10. Bruns, Stephan B. & König, Johannes & Stern, David I., 2019. "Replication and robustness analysis of ‘energy and economic growth in the USA: A multivariate approach’," Energy Economics, Elsevier, vol. 82(C), pages 100-113.
    11. Dreber, Anna & Johannesson, Magnus, 2023. "A framework for evaluating reproducibility and replicability in economics," Ruhr Economic Papers 1055, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    12. Latré, Edwin & Perko, Tanja & Thijssen, Peter, 2017. "Public opinion change after the Fukushima nuclear accident: The role of national context revisited," Energy Policy, Elsevier, vol. 104(C), pages 124-133.
    13. Adesina, Adedoyin & Akogun, Oladele & Dillon, Andrew & Friedman, Jed & Njobdi, Sani & Serneels, Pieter, 2017. "Robustness and External Validity: What do we Learn from Repeated Study Designs over Time?," 2018 Allied Social Sciences Association (ASSA) Annual Meeting, January 5-7, 2018, Philadelphia, Pennsylvania 266292, Agricultural and Applied Economics Association.
    14. Emilie Caldeira & Alou Adessé Dama & Ali Compaoré & Mario Mansour & Grégoire Rota-Graziosi, 2020. "Tax effort in Sub-Saharan African countries : evidence from a new dataset," Working Papers hal-02543162, HAL.
    15. Isaiah Andrews & Maximilian Kasy, 2019. "Identification of and Correction for Publication Bias," American Economic Review, American Economic Association, vol. 109(8), pages 2766-2794, August.
    16. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    17. Alex Young, 2018. "Will the Real Specification Please Stand Up? A Comment on Andrew Bird and Stephen Karolyi," Econ Journal Watch, Econ Journal Watch, vol. 15(1), pages 1-35–48, January.
    18. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    19. Haller, Stefanie & Flaherty, Eoin T. & Balsvik, Ragnhild, 2023. "Knowledge Transfers from Multinational to Domestic Firms: Evidence from Worker Mobility. A Replication Study of Poole (Review of Economics and Statistics, 2013)," Journal of Comments and Replications in Economics (JCRE), ZBW - Leibniz Information Centre for Economics, vol. 2(2023-3), pages 1-30.
    20. Essi Eerola & Teemu Lyytikäinen, 2021. "Housing Allowance and Rents: Evidence from a Stepwise Subsidy Scheme," Scandinavian Journal of Economics, Wiley Blackwell, vol. 123(1), pages 84-109, January.

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:aecrev:v:107:y:2017:i:5:p:27-31. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.