IDEAS home Printed from https://ideas.repec.org/p/fip/fedgfe/2015-83.html
   My bibliography  Save this paper

Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"

Author

Abstract

No abstract is available for this item.

Suggested Citation

  • Andrew C. Chang & Phillip Li, 2015. "Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"," Finance and Economics Discussion Series 2015-83, Board of Governors of the Federal Reserve System (U.S.).
  • Handle: RePEc:fip:fedgfe:2015-83
    DOI: 10.17016/FEDS.2015.083
    as

    Download full text from publisher

    File URL: http://www.federalreserve.gov/econresdata/feds/2015/files/2015083pap.pdf
    File Function: Full text
    Download Restriction: no

    File URL: http://dx.doi.org/10.17016/FEDS.2015.083
    File Function: http://dx.doi.org/10.17016/FEDS.2015.083
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alan J. Auerbach & Yuriy Gorodnichenko, 2012. "Measuring the Output Responses to Fiscal Policy," American Economic Journal: Economic Policy, American Economic Association, vol. 4(2), pages 1-27, May.
    2. Karel Mertens & Morten O. Ravn, 2013. "The Dynamic Effects of Personal and Corporate Income Tax Changes in the United States," American Economic Review, American Economic Association, vol. 103(4), pages 1212-1247, June.
    3. Peter N. Ireland, 2009. "On the Welfare Cost of Inflation and the Recent Behavior of Money Demand," American Economic Review, American Economic Association, vol. 99(3), pages 1040-1052, June.
    4. Arvind Krishnamurthy & Annette Vissing-Jorgensen, 2012. "The Aggregate Demand for Treasury Debt," Journal of Political Economy, University of Chicago Press, vol. 120(2), pages 233-267.
    5. Thomas Herndon & Michael Ash & Robert Pollin, 2014. "Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff," Cambridge Journal of Economics, Oxford University Press, vol. 38(2), pages 257-279.
    6. Steven D. Levitt, 2002. "Using Electoral Cycles in Police Hiring to Estimate the Effects of Police on Crime: Reply," American Economic Review, American Economic Association, vol. 92(4), pages 1244-1250, September.
    7. Carlo Favero & Francesco Giavazzi, 2012. "Measuring Tax Multipliers: The Narrative Method in Fiscal VARs," American Economic Journal: Economic Policy, American Economic Association, vol. 4(2), pages 69-94, May.
    8. Valerie A. Ramey, 2011. "Identifying Government Spending Shocks: It's all in the Timing," The Quarterly Journal of Economics, Oxford University Press, vol. 126(1), pages 1-50.
    9. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    10. Giancarlo Corsetti & André Meier & Gernot J. Müller, 2012. "Fiscal Stimulus with Spending Reversals," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 878-895, November.
    11. Michelle Alexopoulos & Jon Cohen, 2011. "Volumes of evidence: examining technical change in the last century through a new lens," Canadian Journal of Economics, Canadian Economics Association, vol. 44(2), pages 413-450, May.
    12. Caroline M. Hoxby, 2000. "Does Competition among Public Schools Benefit Students and Taxpayers?," American Economic Review, American Economic Association, vol. 90(5), pages 1209-1238, December.
    13. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 40(3), pages 715-733, August.
    14. Karel Mertens & Morten Overgaard Ravn, 2011. "Understanding the Aggregate Effects of Anticipated and Unanticipated Tax Policy Shocks," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 14(1), pages 27-54, January.
    15. Christina D. Romer & David H. Romer, 2010. "The Macroeconomic Effects of Tax Changes: Estimates Based on a New Measure of Fiscal Shocks," American Economic Review, American Economic Association, vol. 100(3), pages 763-801, June.
    16. Andrew C. Chang & Phillip Li, 2018. "Measurement Error In Macroeconomic Data And Economics Research: Data Revisions, Gross Domestic Product, And Gross Domestic Income," Economic Inquiry, Western Economic Association International, vol. 56(3), pages 1846-1869, July.
    17. George J. Hall & Thomas J. Sargent, 2011. "Interest Rate Risk and Other Determinants of Post-WWII US Government Debt/GDP Dynamics," American Economic Journal: Macroeconomics, American Economic Association, vol. 3(3), pages 192-214, July.
    18. Kydland, Finn E & Prescott, Edward C, 1982. "Time to Build and Aggregate Fluctuations," Econometrica, Econometric Society, vol. 50(6), pages 1345-1370, November.
    19. Alan B. Krueger & David Card, 2000. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Reply," American Economic Review, American Economic Association, vol. 90(5), pages 1397-1420, December.
    20. Robert J. Barro, 1991. "Economic Growth in a Cross Section of Countries," The Quarterly Journal of Economics, Oxford University Press, vol. 106(2), pages 407-443.
    21. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    22. Barry Nalebuff & Ron Shachar, 1999. "Follow the Leader: Theory and Evidence on Political Participation," American Economic Review, American Economic Association, vol. 89(3), pages 525-547, June.
    23. Zeynep Senyuz, 2011. "Factor analysis of permanent and transitory dynamics of the US economy and the stock market," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(6), pages 975-998, September.
    24. Michelle Alexopoulos, 2011. "Read All about It!! What Happens Following a Technology Shock?," American Economic Review, American Economic Association, vol. 101(4), pages 1144-1179, June.
    25. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    26. William Wascher & David Neumark, 2000. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Comment," American Economic Review, American Economic Association, vol. 90(5), pages 1362-1396, December.
    27. Xavier Gabaix, 2011. "The Granular Origins of Aggregate Fluctuations," Econometrica, Econometric Society, vol. 79(3), pages 733-772, May.
    28. Lutz Kilian, 2009. "Not All Oil Price Shocks Are Alike: Disentangling Demand and Supply Shocks in the Crude Oil Market," American Economic Review, American Economic Association, vol. 99(3), pages 1053-1069, June.
    29. Todd E. Clark & Michael W. McCracken, 2010. "Averaging forecasts from VARs with uncertain instabilities," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 5-29, January.
    30. Justin McCrary, 2002. "Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime: Comment," American Economic Review, American Economic Association, vol. 92(4), pages 1236-1243, September.
    31. Fabio Canova & Luca Gambetti, 2010. "Do Expectations Matter? The Great Moderation Revisited," American Economic Journal: Macroeconomics, American Economic Association, vol. 2(3), pages 183-205, July.
    32. Robert J. Barro & Charles J. Redlick, 2011. "Macroeconomic Effects From Government Purchases and Taxes," The Quarterly Journal of Economics, Oxford University Press, vol. 126(1), pages 51-102.
    33. Alan J. Auerbach & Yuriy Gorodnichenko, 2013. "Corrigendum: Measuring the Output Responses to Fiscal Policy," American Economic Journal: Economic Policy, American Economic Association, vol. 5(3), pages 320-322, August.
    34. Garth Heutel, 2012. "How Should Environmental Policy Respond to Business Cycles? Optimal Policy under Persistent Productivity Shocks," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 15(2), pages 244-264, April.
    35. Christiane Baumeister & Gert Peersman, 2013. "Time-Varying Effects of Oil Supply Shocks on the US Economy," American Economic Journal: Macroeconomics, American Economic Association, vol. 5(4), pages 1-28, October.
    36. JonasD.M. Fisher & Ryan Peters, 2010. "Using Stock Returns to Identify Government Spending Shocks," Economic Journal, Royal Economic Society, vol. 120(544), pages 414-436, May.
    37. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    38. Dean Croushore & Tom Stark, 2003. "A Real-Time Data Set for Macroeconomists: Does the Data Vintage Matter?," The Review of Economics and Statistics, MIT Press, vol. 85(3), pages 605-617, August.
    39. Colleen Carey & Stephen H. Shore, 2013. "From the Peaks to the Valleys: Cross-State Evidence on Income Volatility over the Business Cycle," The Review of Economics and Statistics, MIT Press, vol. 95(2), pages 549-562, May.
    40. Levitt, Steven D, 1997. "Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime," American Economic Review, American Economic Association, vol. 87(3), pages 270-290, June.
    41. B. D. McCullough & H. D. Vinod, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Reply," American Economic Review, American Economic Association, vol. 94(1), pages 400-406, March.
    42. Anna Kormilitsina, 2011. "Oil Price Shocks and the Optimality of Monetary Policy," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 14(1), pages 199-223, January.
    43. Garth Heutel, 2012. "How Should Environmental Policy Respond to Business Cycles? Optimal Policy under Persistent Productivity Shocks," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 15(2), pages 244-264, April.
    44. Atsushi Inoue & Barbara Rossi, 2011. "Identifying the Sources of Instabilities in Macroeconomic Fluctuations," The Review of Economics and Statistics, MIT Press, vol. 93(4), pages 1186-1204, November.
    45. Caroline M. Hoxby, 2007. "Does Competition Among Public Schools Benefit Students and Taxpayers? Reply," American Economic Review, American Economic Association, vol. 97(5), pages 2038-2055, December.
    46. Sophocles Mavroeidis, 2010. "Monetary Policy Rules and Macroeconomic Stability: Some New Evidence," American Economic Review, American Economic Association, vol. 100(1), pages 491-503, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Economic failures
      by Inaki Villanueva in Applied economist on 2016-02-22 20:40:00

    RePEc Biblio mentions

    As found on the RePEc Biblio, the curated bibliography for Economics:
    1. > Economics Profession > Publishing in Economics > Replication

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andreas Schwab, 2018. "Investigating and Communicating the Uncertainty of Effects: The Power of Graphs," Entrepreneurship Theory and Practice, , vol. 42(6), pages 823-834, November.
    2. Hou, Kewei & Xue, Chen & Zhang, Lu, 2017. "Replicating Anomalies," Working Paper Series 2017-10, Ohio State University, Charles A. Dice Center for Research in Financial Economics.
    3. Muradchanian, Jasmine & Hoekstra, Rink & Kiers, Henk & van Ravenzwaaij, Don, 2020. "How Best to Quantify Replication Success? A Simulation Study on the Comparison of Replication Success Metrics," MetaArXiv wvdjf, Center for Open Science.
    4. Allan Drazen & Anna Dreber & Erkut Y. Ozbay & Erik Snowberg, 2019. "A Journal-Based Replication of "Being Chosen to Lead"," CESifo Working Paper Series 7942, CESifo.
    5. Maren Duvendack & Richard Palmer-Jones & W. Robert Reed, 2017. "What Is Meant by "Replication" and Why Does It Encounter Resistance in Economics?," American Economic Review, American Economic Association, vol. 107(5), pages 46-51, May.
    6. Chang, Andrew C., 2017. "A replication recipe: List your ingredients before you start cooking," Economics Discussion Papers 2017-74, Kiel Institute for the World Economy (IfW).
    7. Gary A. Hoover & Christian Hopp, 2017. "What Crisis? Taking Stock of Management Researchers' Experiences with and Views of Scholarly Misconduct," CESifo Working Paper Series 6611, CESifo.
    8. Adesina, Adedoyin & Akogun, Oladele & Dillon, Andrew & Friedman, Jed & Njobdi, Sani & Serneels, Pieter, 2017. "Robustness and External Validity: What do we Learn from Repeated Study Designs over Time?," 2018 Allied Social Sciences Association (ASSA) Annual Meeting, January 5-7, 2018, Philadelphia, Pennsylvania 266292, Agricultural and Applied Economics Association.
    9. Andrew C. Chang & Phillip Li, 2017. "A Preanalysis Plan to Replicate Sixty Economics Research Papers That Worked Half of the Time," American Economic Review, American Economic Association, vol. 107(5), pages 60-64, May.
    10. Tolossa Fufa Gulema & Zhou Xiaoyan, 2021. "Mediating Role of Innovation Capacity in the Relationship between Corporate governance and Firm Performance: evidence from Chinese listed firms," International Journal of Science and Business, IJSAB International, vol. 5(4), pages 105-122.
    11. Opoku-Agyemang, Kweku A., 2017. "A Human-Computer Interaction Approach for Integrity in Economics," SocArXiv ra3cs, Center for Open Science.
    12. Andrew Y. Chen & Rebecca Wasyk & Fabian Winkler, 2017. "A Likelihood-Based Comparison of Macro Asset Pricing Models," Finance and Economics Discussion Series 2017-024, Board of Governors of the Federal Reserve System (U.S.).
    13. Tyler Cowen & Alex Tabarrok, 2016. "A Skeptical View of the National Science Foundation's Role in Economic Research," Journal of Economic Perspectives, American Economic Association, vol. 30(3), pages 235-248, Summer.
    14. Sebastian Galiani & Paul Gertler & Mauricio Romero, 2017. "Incentives for Replication in Economics," NBER Working Papers 23576, National Bureau of Economic Research, Inc.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrew C. Chang & Phillip Li, 2018. "Measurement Error In Macroeconomic Data And Economics Research: Data Revisions, Gross Domestic Product, And Gross Domestic Income," Economic Inquiry, Western Economic Association International, vol. 56(3), pages 1846-1869, July.
    2. Ramey, V.A., 2016. "Macroeconomic Shocks and Their Propagation," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 71-162, Elsevier.
    3. Virkola, Tuomo, 2014. "Exchange Rate Regime, Fiscal Foresight and the Effectiveness of Fiscal Policy in a Small Open Economy," ETLA Reports 20, The Research Institute of the Finnish Economy.
    4. Mertens, Karel & Ravn, Morten O., 2014. "A reconciliation of SVAR and narrative estimates of tax multipliers," Journal of Monetary Economics, Elsevier, vol. 68(S), pages 1-19.
    5. Alesina, Alberto & Favero, Carlo & Giavazzi, Francesco, 2015. "The output effect of fiscal consolidation plans," Journal of International Economics, Elsevier, vol. 96(S1), pages 19-42.
    6. Klein, Mathias & Linnemann, Ludger, 2019. "Tax and spending shocks in the open economy: are the deficits twins?," Working Paper Series 377, Sveriges Riksbank (Central Bank of Sweden).
    7. Mathias Klein & Ludger Linnemann, 2019. "Tax and Spending Shocks in the Open Economy: Are the Deficits Twins?," Discussion Papers of DIW Berlin 1821, DIW Berlin, German Institute for Economic Research.
    8. Klein, Mathias & Linnemann, Ludger, 2019. "Tax and spending shocks in the open economy: Are the deficits twins?," European Economic Review, Elsevier, vol. 120(C).
    9. Fotiou, Alexandra & Shen, Wenyi & Yang, Shu-Chun S., 2020. "The fiscal state-dependent effects of capital income tax cuts," Journal of Economic Dynamics and Control, Elsevier, vol. 117(C).
    10. Efrem Castelnuovo & Guay Lim, 2019. "What Do We Know About the Macroeconomic Effects of Fiscal Policy? A Brief Survey of the Literature on Fiscal Multipliers," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 52(1), pages 78-93, March.
    11. Giancarlo Corsetti & Michael P. Devereux & Luigi Guiso & John Hassler & Gilles Saint-Paul & Hans-Werner Sinn & Jan-Egbert Sturm & Xavier Vives, 2010. "Chapter 3: From Fiscal Rescue to Global Debt," EEAG Report on the European Economy, CESifo, vol. 0, pages 71-100, February.
    12. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    13. Piotr Krajewski, 2017. "Regionalne zróżnicowanie oddziaływania wydatków rządowych na zatrudnienie – wnioski z analizy SVAR," Bank i Kredyt, Narodowy Bank Polski, vol. 48(1), pages 73-96.
    14. Karel Mertens & Morten O. Ravn, 2013. "The Dynamic Effects of Personal and Corporate Income Tax Changes in the United States," American Economic Review, American Economic Association, vol. 103(4), pages 1212-1247, June.
    15. Sanz, Carlos & Gonzalo Muñoz, Jesus & Alloza, Mario, 2019. "Dynamic Effects of Persistent Shocks," UC3M Working papers. Economics 29187, Universidad Carlos III de Madrid. Departamento de Economía.
    16. Rüth, Sebastian K. & Simon, Camilla, 2020. "How Do Income and the Debt Position of Households Propagate Public into Private Spending?," Working Papers 0676, University of Heidelberg, Department of Economics.
    17. B.D. McCullough & Kerry Anne McGeary & Teresa D. Harrison, 2008. "Do economics journal archives promote replicable research?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 41(4), pages 1406-1420, November.
    18. Nadav Ben Zeev, 2019. "Is There A Single Shock That Drives The Majority Of Business Cycle Fluctuations?," Working Papers 1906, Ben-Gurion University of the Negev, Department of Economics.
    19. Párraga Rodríguez, Susana, 2018. "The dynamic effects of public expenditure shocks in the United States," Journal of Macroeconomics, Elsevier, vol. 56(C), pages 340-360.
    20. Christian Bredemeier & Falko Juessen & Andreas Schabert, 2021. "Why Are Fiscal Multipliers Moderate Even Under Monetary Accommodation?," ECONtribute Discussion Papers Series 074, University of Bonn and University of Cologne, Germany.

    More about this item

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C80 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - General
    • C82 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Macroeconomic Data; Data Access
    • C87 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Econometric Software
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software
    • E01 - Macroeconomics and Monetary Economics - - General - - - Measurement and Data on National Income and Product Accounts and Wealth; Environmental Accounts

    NEP fields

    This paper has been announced in the following NEP Reports:

    Lists

    This item is featured on the following reading lists, Wikipedia, or ReplicationWiki pages:
    1. Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say ”Usually Not” (Critical Finance Review forthcoming) in ReplicationWiki

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedgfe:2015-83. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: https://edirc.repec.org/data/frbgvus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.