IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/13026.html
   My bibliography  Save this paper

Replication in Economics

Author

Listed:
  • Daniel S. Hamermesh

Abstract

This examination of the role and potential for replication in economics points out the paucity of both pure replication -- checking on others' published papers using their data -- and scientific replication -- using data representing different populations in one's own work or in a Comment. Several controversies in empirical economics illustrate how and how not to behave when replicating others' work. The incentives for replication facing editors, authors and potential replicators are examined. Recognising these incentives, I advance proposals aimed at journal editors that will increase the supply of replication studies, and I propose a way of generating more scientific replication that will make empirical economic research more credible.

Suggested Citation

  • Daniel S. Hamermesh, 2007. "Replication in Economics," NBER Working Papers 13026, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:13026
    Note: LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w13026.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    2. Caroline M. Hoxby, 2000. "Does Competition among Public Schools Benefit Students and Taxpayers?," American Economic Review, American Economic Association, vol. 90(5), pages 1209-1238, December.
    3. Feldstein, Martin S, 1982. "Social Security and Private Saving: Reply," Journal of Political Economy, University of Chicago Press, vol. 90(3), pages 630-642, June.
    4. Hamermesh, Daniel S., 1997. "Some thoughts on replications and reviews," Labour Economics, Elsevier, vol. 4(2), pages 107-109, June.
    5. De Long, J Bradford & Lang, Kevin, 1992. "Are All Economic Hypotheses False?," Journal of Political Economy, University of Chicago Press, vol. 100(6), pages 1257-1272, December.
    6. Hamermesh, Daniel S & Biddle, Jeff E, 1994. "Beauty and the Labor Market," American Economic Review, American Economic Association, vol. 84(5), pages 1174-1194, December.
    7. Daniel S. Hamermesh & Peter Schmidt, 2003. "The Determinants of Econometric Society Fellows Elections," Econometrica, Econometric Society, vol. 71(1), pages 399-407, January.
    8. Leimer, Dean R & Lesnoy, Selig D, 1982. "Social Security and Private Saving: New Time-Series Evidence," Journal of Political Economy, University of Chicago Press, vol. 90(3), pages 606-629, June.
    9. Daniel S. Hamermesh, 2002. "International Labor Economics," Journal of Labor Economics, University of Chicago Press, vol. 20(4), pages 709-732, October.
    10. Kate L. Antonovics & Arthur S. Goldberger, 2005. "Does Increasing Women's Schooling Raise the Schooling of the Next Generation? Comment," American Economic Review, American Economic Association, vol. 95(5), pages 1738-1744, December.
    11. Glenn Ellison, 2002. "The Slowdown of the Economics Publishing Process," Journal of Political Economy, University of Chicago Press, vol. 110(5), pages 947-993, October.
    12. Jesse Rothstein, 2007. "Does Competition Among Public Schools Benefit Students and Taxpayers? Comment," American Economic Review, American Economic Association, vol. 97(5), pages 2026-2037, December.
    13. Gregory Grossman & Richard Sakwa & Rachel Walker & Daniel Nelson & Mike Bowker & Jeffrey Checkel & Elizabeth Coughlan & Derek Aldcroft & David Mason & Kieran Williams & Jan Drewnowski & Andrei Kuznets, 1997. "Reviews," Europe-Asia Studies, Taylor & Francis Journals, vol. 49(5), pages 905-941.
    14. Hunter, John E, 2001. "The Desperate Need for Replications," Journal of Consumer Research, Oxford University Press, vol. 28(1), pages 149-158, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    RePEc Biblio mentions

    As found on the RePEc Biblio, the curated bibliography for Economics:
    1. > Economics Profession > Publishing in Economics > Replication

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hussinger, Katrin & Pellens, Maikel, 2019. "Guilt by association: How scientific misconduct harms prior collaborators," Research Policy, Elsevier, vol. 48(2), pages 516-530.
    2. Bruno S. Frey, 2010. "Withering Academia," CREMA Working Paper Series 2010-19, Center for Research in Economics, Management and the Arts (CREMA).
    3. Koten, Silvester Van, 2020. "Forward premia in electricity markets: A replication study," Energy Economics, Elsevier, vol. 89(C).
    4. Grolleau, Gilles & Ibanez, Lisette & Mzoughi, Naoufel, 2020. "Moral judgment of environmental harm caused by a single versus multiple wrongdoers: A survey experiment," Ecological Economics, Elsevier, vol. 170(C).
    5. Daniels, Gerald Eric & Kakar, Venoo, 2018. "Normalized CES supply-side system approach: How to replicate Klump, McAdam, and Willman (2007)," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW), vol. 12, pages 1-11.
    6. Kaplan, Greg & Mitman, Kurt & Violante, Giovanni L., 2020. "Non-durable consumption and housing net worth in the Great Recession: Evidence from easily accessible data," Journal of Public Economics, Elsevier, vol. 189(C).
    7. Simonetta LONGHI & Peter NIJKAMP & Jacques POOT, 2008. "Meta-Analysis Of Empirical Evidence On The Labour Market Impacts Of Immigration," Region et Developpement, Region et Developpement, LEAD, Universite du Sud - Toulon Var, vol. 27, pages 161-191.
    8. Nicola Lacetera & Lorenzo Zirulia, 2011. "The Economics of Scientific Misconduct," Journal of Law, Economics, and Organization, Oxford University Press, vol. 27(3), pages 568-603.
    9. Daniels, Gerald Eric & Kakar, Venoo, 2017. "Normalized CES supply-side system approach: How to replicate Klump, McAdam, and Willman (Review of Economics and Statistics, 2007)," Economics Discussion Papers 2017-70, Kiel Institute for the World Economy (IfW).
    10. Santi Budria & Luis Diaz-Serrano & Ada Ferrer-i-Carbonell & Joop Hartog, 2013. "Risk attitude and wage growth: replicating Shaw (1996)," Empirical Economics, Springer, vol. 44(2), pages 981-1004, April.
    11. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    12. Kiri, Bralind & Lacetera, Nicola & Zirulia, Lorenzo, 2018. "Above a swamp: A theory of high-quality scientific production," Research Policy, Elsevier, vol. 47(5), pages 827-839.
    13. Owen, P. Dorian, 2018. "Replication to assess statistical adequacy," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW), vol. 12, pages 1-16.
    14. Furman, Jeffrey L. & Jensen, Kyle & Murray, Fiona, 2012. "Governing knowledge in the scientific community: Exploring the role of retractions in biomedicine," Research Policy, Elsevier, vol. 41(2), pages 276-290.
    15. Karl-Heinz Tödter, 2009. "Benford's Law as an Indicator of Fraud in Economics," German Economic Review, Verein für Socialpolitik, vol. 10, pages 339-351, August.
    16. Brian Goesling, "undated". "Making Sense of Replication Studies: Guidance for Teen Pregnancy Prevention Researchers," Mathematica Policy Research Reports 02bb275446464c37a4fddbcdf, Mathematica Policy Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 40(3), pages 715-733, August.
    2. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    3. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    4. Tito Boeri & Jan van Ours, 2013. "The Economics of Imperfect Labor Markets: Second Edition," Economics Books, Princeton University Press, edition 1, number 10142.
    5. Christensen, Garret & Miguel, Edward & Sturdy, Jennifer, 2017. "Transparency, Reproducibility, and the Credibility of Economics Research," MetaArXiv 9a3rw, Center for Open Science.
    6. Jonathan Gruber & Aaron Yelowitz, 1999. "Public Health Insurance and Private Savings," Journal of Political Economy, University of Chicago Press, vol. 107(6), pages 1249-1274, December.
    7. Rajeev Dehejia, 2013. "The Porous Dialectic: Experimental and Non-Experimental Methods in Development Economics," WIDER Working Paper Series wp-2013-011, World Institute for Development Economic Research (UNU-WIDER).
    8. Martin Schlotter & Guido Schwerdt & Ludger Woessmann, 2011. "Econometric methods for causal evaluation of education policies and practices: a non-technical guide," Education Economics, Taylor & Francis Journals, vol. 19(2), pages 109-137.
    9. Song, Yang, 2019. "Sorting, school performance and quality: Evidence from China," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 238-261.
    10. Annette N. Brown & Drew B. Cameron & Benjamin D. K. Wood, 2014. "Quality evidence for policymaking: I'll believe it when I see the replication," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 215-235, September.
    11. Phan Kim Dung, 2017. "The effects of minimum wage hikes on employment and wages in Vietnam’s micro, small, and medium enterprises," WIDER Working Paper Series 095, World Institute for Development Economic Research (UNU-WIDER).
    12. Lemos Sara, 2005. "Political Variables as Instruments for the Minimum Wage," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 4(1), pages 1-33, December.
    13. Mustafa U. Karakaplan & Levent Kutlu, 2019. "School district consolidation policies: endogenous cost inefficiency and saving reversals," Empirical Economics, Springer, vol. 56(5), pages 1729-1768, May.
    14. Franklin Mixon, 2002. "Social security trust fund flows and the welfare costs of rent seeking," Applied Economics, Taylor & Francis Journals, vol. 34(8), pages 975-979.
    15. Omar Al-Ubaydli & John List, 2013. "On the Generalizability of Experimental Results in Economics: With A Response To Camerer," Artefactual Field Experiments j0001, The Field Experiments Website.
    16. Philippe Jehiel & Laurent Lamy, 2018. "A Mechanism Design Approach to the Tiebout Hypothesis," Journal of Political Economy, University of Chicago Press, vol. 126(2), pages 735-760.
    17. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    18. Gaynor, Martin & Laudicella, Mauro & Propper, Carol, 2012. "Can governments do it better? Merger mania and hospital outcomes in the English NHS," Journal of Health Economics, Elsevier, vol. 31(3), pages 528-543.
    19. Henry G. Overman, 2010. "“Gis A Job”: What Use Geographical Information Systems In Spatial Economics?," Journal of Regional Science, Wiley Blackwell, vol. 50(1), pages 165-180, February.
    20. Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre (Seville site).

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C59 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Other

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:13026. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.