IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp10403.html
   My bibliography  Save this paper

Replication in Labor Economics: Evidence from Data, and What It Suggests

Author

Listed:
  • Hamermesh, Daniel S.

    (University of Texas at Austin)

Abstract

Examining the most heavily-cited publications in labor economics from the early 1990s, I show that few of over 3000 articles citing them directly replicates them. They are replicated more frequently using data from other time periods and economies, so that the validity of their central ideas has typically been verified. This pattern of scholarship suggests, beyond the currently required depositing of data and code upon publication, that there is little need for formal mechanisms for replication. The market for scholarship already produces replications of non-laboratory applied research.

Suggested Citation

  • Hamermesh, Daniel S., 2016. "Replication in Labor Economics: Evidence from Data, and What It Suggests," IZA Discussion Papers 10403, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp10403
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp10403.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. ., 2010. "Warranted Growth and the Role of the State," Chapters, in: Strategic Competition, Dynamics, and the Role of the State, chapter 5, Edward Elgar Publishing.
    2. Thomas Herndon & Michael Ash & Robert Pollin, 2014. "Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 38(2), pages 257-279.
    3. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    4. Carmen M. Reinhart & Kenneth S. Rogoff, 2010. "Growth in a Time of Debt," American Economic Review, American Economic Association, vol. 100(2), pages 573-578, May.
    5. Edward P. Lazear, 1990. "Job Security Provisions and Employment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 105(3), pages 699-726.
    6. Joonkyung Ha & Jong-Wha Lee & Lea Sumulong, 2010. "Rebalancing Growth in the Republic of Korea," Macroeconomics Working Papers 22753, East Asian Bureau of Economic Research.
    7. Daniel S. Hamermesh, 2007. "Replication in Economics," NBER Working Papers 13026, National Bureau of Economic Research, Inc.
    8. ., 2010. "A Review of the Literature on Growth," Chapters, in: Strategic Competition, Dynamics, and the Role of the State, chapter 3, Edward Elgar Publishing.
    9. Steven J. Davis & John Haltiwanger, 1992. "Gross Job Creation, Gross Job Destruction, and Employment Reallocation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(3), pages 819-863.
    10. Ashenfelter, Orley & Krueger, Alan B, 1994. "Estimates of the Economic Returns to Schooling from a New Sample of Twins," American Economic Review, American Economic Association, vol. 84(5), pages 1157-1173, December.
    11. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 40(3), pages 715-733, August.
    12. Neal, Derek A & Johnson, William R, 1996. "The Role of Premarket Factors in Black-White Wage Differences," Journal of Political Economy, University of Chicago Press, vol. 104(5), pages 869-895, October.
    13. Reinhart, Carmen & Rogoff, Kenneth, 2010. "Debt and Growth Revisited," MPRA Paper 24376, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marianne Saam, 2021. "Überlegungen zu Open Science in der Volkswirtschaftslehre [Reflections on Open Science in Economics]," Wirtschaftsdienst, Springer;ZBW - Leibniz Information Centre for Economics, vol. 101(10), pages 746-747, October.
    2. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 48(1), pages 62-83.
    3. Travis Richardson & Georgios Nalbantis & Tim Pawlowski, 2023. "Emotional Cues and the Demand for Televised Sports: Evidence from the UEFA Champions League," Journal of Sports Economics, , vol. 24(8), pages 993-1025, December.
    4. Hernández Alemán, Anastasia & León, Carmelo J., 2018. "La Réplica en el Análisis Económico Aplicado/Replication in Applied Economic Analysis," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 36, pages 317-332, Enero.
    5. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    6. Nicolas Vallois & Dorian Jullien, 2017. "Replication in experimental economics: A historical and quantitative approach focused on public good game experiments," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01651080, HAL.
    7. Christian Leuz, 2018. "Evidence-based policymaking: promise, challenges and opportunities for accounting and financial markets research," Accounting and Business Research, Taylor & Francis Journals, vol. 48(5), pages 582-608, July.
    8. Aditya Bhattacharjea, 2019. "Labour Market Flexibility in Indian Industry A Critical Survey of the Literature," Working papers 296, Centre for Development Economics, Delhi School of Economics.
    9. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
    10. Aditya BHATTACHARJEA, 2021. "Labour market flexibility in Indian manufacturing: A critical survey of the literature," International Labour Review, International Labour Organization, vol. 160(2), pages 197-217, June.
    11. Kenneth Button, 2020. "Studying the empirical implications of the liberalization of airport markets," Competition and Regulation in Network Industries, , vol. 21(3), pages 223-243, September.
    12. W. Robert Reed, 2017. "Replication in labor economics," IZA World of Labor, Institute of Labor Economics (IZA), pages 413-413, December.
    13. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    14. Abel Brodeur & Nikolai Cook & Carina Neisser, 2022. "P-Hacking, Data Type and Data-Sharing Policy," ECONtribute Discussion Papers Series 200, University of Bonn and University of Cologne, Germany.
    15. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    16. Christophe Hurlin & Christophe Pérignon, 2020. "Reproducibility Certification in Economics Research," Working Papers hal-02896404, HAL.
    17. Anderson, Richard G., 2017. "Should you choose to do so...: A replication paradigm," Economics Discussion Papers 2017-79, Kiel Institute for the World Economy (IfW Kiel).
    18. Nicolas Vallois & Dorian Jullien, 2017. "Replication in Experimental Economics: A Historical and Quantitative Approach Focused on Public Good Game Experiments," GREDEG Working Papers 2017-21, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    19. Josephson, Anna & Michler, Jeffrey D., 2018. "Viewpoint: Beasts of the field? Ethics in agricultural and applied economics," Food Policy, Elsevier, vol. 79(C), pages 1-11.
    20. Sylvérie Herbert & Hautahi Kingi & Flavio Stanchi & Lars Vilhubern, 2021. "The Reproducibility of Economics Research: A Case Study," Working papers 853, Banque de France.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Consiglio Andrea & Zenios Stavros A., 2015. "Risk Management Optimization for Sovereign Debt Restructuring," Journal of Globalization and Development, De Gruyter, vol. 6(2), pages 181-213, December.
    2. Gianluigi Pelloni & Marco Savioli, 2015. "Why Is Italy Doing So Badly?," Economic Affairs, Wiley Blackwell, vol. 35(3), pages 349-365, October.
    3. Di Sanzo Silvestro & Bella Mariano, 2015. "Public debt and growth in the euro area: evidence from parametric and nonparametric Granger causality," The B.E. Journal of Macroeconomics, De Gruyter, vol. 15(2), pages 631-648, July.
    4. repec:rim:rimpre:15_01 is not listed on IDEAS
    5. Lorenzo Forni & Geremia Palomba & Joana Pereira & Christine Richmond, 2021. "Sovereign debt restructuring and growth [Investment cycles and sovereign debt overhang]," Oxford Economic Papers, Oxford University Press, vol. 73(2), pages 671-697.
    6. Andrew C. Chang & Phillip Li, 2015. "Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"," Finance and Economics Discussion Series 2015-83, Board of Governors of the Federal Reserve System (U.S.).
    7. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    8. Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre.
    9. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," Research Policy, Elsevier, vol. 48(1), pages 62-83.
    10. Valérie Orozco & Christophe Bontemps & Élise Maigné & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2017. "How to make a pie? Reproducible Research for Empirical Economics & Econometrics," Post-Print hal-01939942, HAL.
    11. Christophe Hurlin & Christophe Pérignon, 2020. "Reproducibility Certification in Economics Research," Working Papers hal-02896404, HAL.
    12. Joseph E. Stiglitz, 2015. "Reconstructing Macroeconomic Theory to Manage Economic Policy," Palgrave Macmillan Books, in: Éloi Laurent & Jacques Cacheux (ed.), Fruitful Economics, chapter 1, pages 20-56, Palgrave Macmillan.
    13. Valérie Orozco & Christophe Bontemps & Elise Maigné & Virginie Piguet & Annie Hofstetter & Anne Lacroix & Fabrice Levert & Jean‐Marc Rousselle, 2020. "How To Make A Pie: Reproducible Research For Empirical Economics And Econometrics," Journal of Economic Surveys, Wiley Blackwell, vol. 34(5), pages 1134-1169, December.
    14. Annette N. Brown & Drew B. Cameron & Benjamin D. K. Wood, 2014. "Quality evidence for policymaking: I'll believe it when I see the replication," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 215-235, September.
    15. Daniel S. Hamermesh & Katie R. Genadek & Michael C. Burda, 2022. "Reply to “Racial Differences in Time at Work Not Working†by William A. Darity Jr. et al," ILR Review, Cornell University, ILR School, vol. 75(3), pages 573-577, May.
    16. Claudio Montenegro & Carmen Pagés, 2005. "Who Benefits from Labor Market Regulations? Chile 1960-1998," Central Banking, Analysis, and Economic Policies Book Series, in: Jorge Restrepo & Andrea Tokman R. & Norman Loayza (Series Editor) & Klaus Schmidt-Hebbel (Series Edi (ed.),Labor Markets and Institutions, edition 1, volume 8, chapter 4, pages 077-114, Central Bank of Chile.
    17. Markus Eberhardt & Andrea Filippo Presbitero, 2013. "This Time They're Different: Heterogeneity;and Nonlinearity in the Relationship;between Debt and Growth," Mo.Fi.R. Working Papers 92, Money and Finance Research group (Mo.Fi.R.) - Univ. Politecnica Marche - Dept. Economic and Social Sciences.
    18. Paulo André Camuri & Frederico G. Jayme Jr. & Ana Maria Hermeto, 2015. "Fiscal consolidation in developed and emerging economies [Fiscal consolidation in developed and emerging economies]," Nova Economia, Economics Department, Universidade Federal de Minas Gerais (Brazil), vol. 25(spe), pages 835-861, December.
    19. Kazuo Nishimura & Carine Nourry & Thomas Seegmuller & Alain Venditti, 2015. "On the (de)stabilizing effect of public debt in a Ramsey model with heterogeneous agents," International Journal of Economic Theory, The International Society for Economic Theory, vol. 11(1), pages 7-24, March.
    20. Swamy, Vighneswara, 2015. "Government Debt and Economic Growth – Decomposing the Cause and Effect Relationship," MPRA Paper 64105, University Library of Munich, Germany.
    21. Swamy, Vighneswara, 2015. "Government Debt and its Macroeconomic Determinants – An Empirical Investigation," MPRA Paper 64106, University Library of Munich, Germany.

    More about this item

    Keywords

    citation analysis; reliability of research; scientific method;
    All these keywords.

    JEL classification:

    • B21 - Schools of Economic Thought and Methodology - - History of Economic Thought since 1925 - - - Microeconomics
    • J01 - Labor and Demographic Economics - - General - - - Labor Economics: General
    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp10403. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.