IDEAS home Printed from https://ideas.repec.org/a/aea/aecrev/v107y2017i5p52-55.html
   My bibliography  Save this article

Replication and Economics Journal Policies

Author

Listed:
  • Jan H. Höffler

Abstract

Economics journals with reproducibility policies are cited more often than others. For the minority of journals with a mandatory and enforced policy, this is significant when controlling for time and journal effects. To cope with the large variety of software used and to develop standards for replicability, joint efforts of journals could ensure each empirical study is published with data, code, and instructions on how to use them together. Individual reviewers could take initiative by asking for replicable empirical results. The American Journal of Political Science sets an example by having all empirical studies externally check for replicability prior to publication.

Suggested Citation

  • Jan H. Höffler, 2017. "Replication and Economics Journal Policies," American Economic Review, American Economic Association, vol. 107(5), pages 52-55, May.
  • Handle: RePEc:aea:aecrev:v:107:y:2017:i:5:p:52-55
    Note: DOI: 10.1257/aer.p20171032
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/articles?id=10.1257/aer.p20171032
    Download Restriction: no

    File URL: https://www.aeaweb.org/articles/attachments?retrieve=w7gkUhN7lVHzsH6Ao4u7v9rVUkSqeLXD
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    ---><---

    References listed on IDEAS

    as
    1. Andrew C. Chang & Phillip Li, 2017. "A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time," American Economic Review, American Economic Association, vol. 107(5), pages 60-64, May.
    2. Jan H. Höffler, 2014. "Teaching Replication in Quantitative Empirical Economics," Replication Working Papers 2/2014, Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project.
    3. Vlaeminck, Sven, 2013. "Data Management in Scholarly Journals and Possible Roles for Libraries - Some Insights from EDaWaX," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 49-79.
    4. Heather A Piwowar & Roger S Day & Douglas B Fridsma, 2007. "Sharing Detailed Research Data Is Associated with Increased Citation Rate," PLOS ONE, Public Library of Science, vol. 2(3), pages 1-5, March.
    5. Vlaeminck, Sven & Herrmann, Lisa-Kristin, 2015. "Data Policies and Data Archives: A New Paradigm for Academic Publishing in Economic Sciences?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 145-155.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. A replication database for economics and social sciences: The ReplicationWiki
      by repecblogguest in RePEc blog on 2020-08-04 14:43:54

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 62-83.
    2. Valérie Orozco & Christophe Bontemps & Elise Maigne & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2017. "How to make a pie? Reproducible Research for Empirical Economics & Econometrics," Post-Print hal-01939942, HAL.
    3. Emma McManus & David Turner & Tracey Sach, 2019. "Can You Repeat That? Exploring the Definition of a Successful Model Replication in Health Economics," PharmacoEconomics, Springer, vol. 37(11), pages 1371-1381, November.
    4. Daniels, Gerald Eric & Kakar, Venoo, 2018. "Normalized CES supply-side system approach: How to replicate Klump, McAdam, and Willman (2007)," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW), vol. 12, pages 1-11.
    5. Kenneth Button, 2020. "Studying the empirical implications of the liberalization of airport markets," Competition and Regulation in Network Industries, , vol. 21(3), pages 223-243, September.
    6. Frank Mueller-Langer & Benedikt Fecher & Dietmar Harhoff & Gert G. Wagner, 2017. "The Economics of Replication," Discussion Papers of DIW Berlin 1640, DIW Berlin, German Institute for Economic Research.
    7. Josephson, Anna & Michler, Jeffrey D., 2018. "Viewpoint: Beasts of the field? Ethics in agricultural and applied economics," Food Policy, Elsevier, vol. 79(C), pages 1-11.
    8. Phil Armstrong, 2020. "Can Heterodox Economics Make a Difference?," Books, Edward Elgar Publishing, number 19964, November.
    9. Lucas C. Coffman & Muriel Niederle & Alistair J. Wilson, 2017. "A Proposal to Organize and Promote Replications," American Economic Review, American Economic Association, vol. 107(5), pages 41-45, May.
    10. Jan H. Höffler, 2020. "Making replicability the norm starting with oneself and depersonalizing research debates," Replication Working Papers 2/2020, Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project.
    11. Robert Webb & Duncan Watson & Steven Cook, 2021. "Price adjustment in the London housing market," Urban Studies, Urban Studies Journal Limited, vol. 58(1), pages 113-130, January.
    12. Cook, Steven & Fosten, Jack, 2019. "Replicating rockets and feathers," Energy Economics, Elsevier, vol. 82(C), pages 139-151.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Valérie Orozco & Christophe Bontemps & Elise Maigne & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2017. "How to make a pie? Reproducible Research for Empirical Economics & Econometrics," Post-Print hal-01939942, HAL.
    2. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 62-83.
    3. Jan H. Höffler, 2020. "Making replicability the norm starting with oneself and depersonalizing research debates," Replication Working Papers 2/2020, Institut für Statistik und Ökonometrie, Wirtschaftswissenschaftliche Fakultät, Georg-August-Universität Göttingen, Replication project.
    4. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    5. Nicola Milia & Alessandra Congiu & Paolo Anagnostou & Francesco Montinaro & Marco Capocasa & Emanuele Sanna & Giovanni Destro Bisol, 2012. "Mine, Yours, Ours? Sharing Data on Human Genetic Variation," PLOS ONE, Public Library of Science, vol. 7(6), pages 1-8, June.
    6. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    7. Vanessa V Sochat & Cameron J Prybol & Gregory M Kurtzer, 2017. "Enhancing reproducibility in scientific computing: Metrics and registry for Singularity containers," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-24, November.
    8. Garret Christensen & Allan Dafoe & Edward Miguel & Don A Moore & Andrew K Rose, 2019. "A study of the impact of data sharing on article citations using journal policies as a natural experiment," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-13, December.
    9. Sachverständigenrat zur Begutachtung der gesamtwirtschaftlichen Entwicklung, 2013. "Wachstum, Wohlstand, Lebensqualität - Auszug aus dem Jahresgutachten 2013/14," RatSWD Working Papers 227, German Data Forum (RatSWD).
    10. Sven Vlaeminck & Gert G. Wagner, 2013. "On the role of research data centres in the management of publication-related research data - Results of a survey among scientific infrastructure service providers," RatSWD Working Papers 226, German Data Forum (RatSWD).
    11. Andrew C. Chang & Phillip Li, 2018. "Measurement Error In Macroeconomic Data And Economics Research: Data Revisions, Gross Domestic Product, And Gross Domestic Income," Economic Inquiry, Western Economic Association International, vol. 56(3), pages 1846-1869, July.
    12. Andrew F Magee & Michael R May & Brian R Moore, 2014. "The Dawn of Open Access to Phylogenetic Data," PLOS ONE, Public Library of Science, vol. 9(10), pages 1-10, October.
    13. Thu-Mai Christian & Amanda Gooch & Todd Vision & Elizabeth Hull, 2020. "Journal data policies: Exploring how the understanding of editors and authors corresponds to the policies themselves," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-15, March.
    14. Maurizio Canavari & Andreas C. Drichoutis & Jayson L. Lusk & Rodolfo M. Nayga, Jr., 2018. "How to run an experimental auction: A review of recent advances," Working Papers 2018-5, Agricultural University of Athens, Department Of Agricultural Economics.
    15. Bull, Charles & Courty, Pascal & Doyon, Maurice & Rondeau, Daniel, 2019. "Failure of the Becker–DeGroot–Marschak mechanism in inexperienced subjects: New tests of the game form misconception hypothesis," Journal of Economic Behavior & Organization, Elsevier, vol. 159(C), pages 235-253.
    16. Robert Webb & Duncan Watson & Steven Cook, 2021. "Price adjustment in the London housing market," Urban Studies, Urban Studies Journal Limited, vol. 58(1), pages 113-130, January.
    17. Tierney, Warren & Hardy, Jay H. & Ebersole, Charles R. & Leavitt, Keith & Viganola, Domenico & Clemente, Elena Giulia & Gordon, Michael & Dreber, Anna & Johannesson, Magnus & Pfeiffer, Thomas & Uhlman, 2020. "Creative destruction in science," Organizational Behavior and Human Decision Processes, Elsevier, vol. 161(C), pages 291-309.
    18. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    19. Maren Duvendack & Richard W. Palmer-Jones & W. Robert Reed, 2015. "Replications in Economics: A Progress Report," Econ Journal Watch, Econ Journal Watch, vol. 12(2), pages 164–191-1, May.
    20. Hernández Alemán, Anastasia & León, Carmelo J., 2018. "La Réplica en el Análisis Económico Aplicado/Replication in Applied Economic Analysis," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 36, pages 317-332, Enero.

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:aecrev:v:107:y:2017:i:5:p:52-55. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.