IDEAS home Printed from https://ideas.repec.org/a/taf/jecmet/v16y2009i4p393-408.html
   My bibliography  Save this article

Statistical vs. economic significance in economics and econometrics: further comments on McCloskey and Ziliak

Author

Listed:
  • Tom Engsted

Abstract

I comment on the controversy between McCloskey and Ziliak and Hoover and Siegler on statistical versus economic significance, in the March 2008 issue of the Journal of Economic Methodology. I argue that while McCloskey and Ziliak are right in emphasizing 'real error', i.e. non-sampling error that cannot be eliminated through specification testing, they fail to acknowledge those areas in economics, e.g. rational expectations macroeconomics and asset pricing, where researchers clearly distinguish between statistical and economic significance and where statistical testing plays a relatively minor role in model evaluation. In these areas models are treated as inherently misspecified and, consequently, are evaluated empirically by other methods than statistical tests. I also criticize McCloskey and Ziliak for their strong focus on the size of parameter estimates while neglecting the important question of how to obtain reliable estimates, and I argue that significance tests are useful tools in those cases where a statistical model serves as input in the quantification of an economic model. Finally, I provide a specific example from economics - asset return predictability - where the distinction between statistical and economic significance is well appreciated, but which also shows how statistical tests have contributed to our substantive economic understanding.

Suggested Citation

  • Tom Engsted, 2009. "Statistical vs. economic significance in economics and econometrics: further comments on McCloskey and Ziliak," Journal of Economic Methodology, Taylor & Francis Journals, vol. 16(4), pages 393-408.
  • Handle: RePEc:taf:jecmet:v:16:y:2009:i:4:p:393-408
    DOI: 10.1080/13501780903337339
    as

    Download full text from publisher

    File URL: http://www.tandfonline.com/doi/abs/10.1080/13501780903337339
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1080/13501780903337339?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Campbell, John Y. & Viceira, Luis M., 2002. "Strategic Asset Allocation: Portfolio Choice for Long-Term Investors," OUP Catalogue, Oxford University Press, number 9780198296942.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alexander Libman & Joachim Zweynert, 2014. "Ceremonial Science: The State of Russian Economics Seen Through the Lens of the Work of ‘Doctor of Science’ Candidates," Working Papers 337, Leibniz Institut für Ost- und Südosteuropaforschung (Institute for East and Southeast European Studies).
    2. Kim, Jae H. & Ji, Philip Inyeob, 2015. "Significance testing in empirical finance: A critical review and assessment," Journal of Empirical Finance, Elsevier, vol. 34(C), pages 1-14.
    3. Hendry David F & Mizon Grayham E, 2011. "Econometric Modelling of Time Series with Outlying Observations," Journal of Time Series Econometrics, De Gruyter, vol. 3(1), pages 1-26, February.
    4. Libman, Alexander & Zweynert, Joachim, 2014. "Ceremonial science: The state of Russian economics seen through the lens of the work of ‘Doctor of Science’ candidates," Economic Systems, Elsevier, vol. 38(3), pages 360-378.
    5. Peter J. Veazie, 2015. "Understanding Statistical Testing," SAGE Open, , vol. 5(1), pages 21582440145, January.
    6. Edsel Beja, 2014. "Income growth and happiness: reassessment of the Easterlin Paradox," International Review of Economics, Springer;Happiness Economics and Interpersonal Relations (HEIRS), vol. 61(4), pages 329-346, December.
    7. Jae H. Kim & Kamran Ahmed & Philip Inyeob Ji, 2018. "Significance Testing in Accounting Research: A Critical Evaluation Based on Evidence," Abacus, Accounting Foundation, University of Sydney, vol. 54(4), pages 524-546, December.
    8. Thomas Mayer, 2012. "Ziliak and McCloskey's Criticisms of Significance Tests: An Assessment," Econ Journal Watch, Econ Journal Watch, vol. 9(3), pages 256-297, September.
    9. Kim, Jae, 2015. "How to Choose the Level of Significance: A Pedagogical Note," MPRA Paper 66373, University Library of Munich, Germany.
    10. Kim, Jae & Choi, In, 2015. "Unit Roots in Economic and Financial Time Series: A Re-Evaluation based on Enlightened Judgement," MPRA Paper 68411, University Library of Munich, Germany.
    11. Edsel L. Beja Jr., 2018. "Testing the easterlin paradox: Results and policy implications," Journal of Behavioral Economics for Policy, Society for the Advancement of Behavioral Economics (SABE), vol. 2(2), pages 79-83, September.
    12. Thomas Mayer, 2012. "Ziliak and McClosky?s Criticisms of Significance Tests: A Damage Assessment," Working Papers 126, University of California, Davis, Department of Economics.
    13. Jae H. Kim & In Choi, 2021. "Choosing the Level of Significance: A Decision‐theoretic Approach," Abacus, Accounting Foundation, University of Sydney, vol. 57(1), pages 27-71, March.
    14. Hart Hodges & Steven E. Henson, 2017. "Weak Foundations in Economic Development Programs," Economic Development Quarterly, , vol. 31(2), pages 116-127, May.
    15. Plöckinger, Martin & Aschauer, Ewald & Hiebl, Martin R.W. & Rohatschek, Roman, 2016. "The influence of individual executives on corporate financial reporting: A review and outlook from the perspective of upper echelons theory," Journal of Accounting Literature, Elsevier, vol. 37(C), pages 55-75.
    16. Pinto, Hugo, 2011. "The role of econometrics in economic science: An essay about the monopolization of economic methodology by econometric methods," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 40(4), pages 436-443, August.
    17. Elisabeth Albertini & Fabienne Berger-Remy, 2019. "Intellectual Capital and Financial Performance: A Meta-Analysis and Research Agenda," Post-Print hal-02139763, HAL.
    18. Beja, Edsel Jr., 2018. "Testing the Easterlin Paradox: Results and Policy Implications," MPRA Paper 101075, University Library of Munich, Germany.
    19. Thomas Mayer, 2012. "Ziliak and McClosky?s Criticisms of Significance Tests: A Damage Assessment," Working Papers 61, University of California, Davis, Department of Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. John Y. Campbell, 2016. "Restoring Rational Choice: The Challenge of Consumer Financial Regulation," American Economic Review, American Economic Association, vol. 106(5), pages 1-30, May.
    2. Robert J. Shiller, 2005. "The Life-Cycle Personal Accounts Proposal for Social Security: An Evaluation," Cowles Foundation Discussion Papers 1504, Cowles Foundation for Research in Economics, Yale University.
    3. Guglielmo Maria Caporale & Alex Plastun, 2019. "Price overreactions in the cryptocurrency market," Journal of Economic Studies, Emerald Group Publishing Limited, vol. 46(5), pages 1137-1155, August.
    4. Ryan Edwards, 2013. "The cost of uncertain life span," Journal of Population Economics, Springer;European Society for Population Economics, vol. 26(4), pages 1485-1522, October.
    5. John Y. Campbell & Luis M. Viceira & Joshua S. White, 2003. "Foreign Currency for Long-Term Investors," Economic Journal, Royal Economic Society, vol. 113(486), pages 1-25, March.
    6. Luc Arrondel & Fr餩rique Savignac, 2015. "Risk management, housing and stockholding," Applied Economics, Taylor & Francis Journals, vol. 47(39), pages 4208-4227, August.
    7. Miao, Jianjun & Wang, Neng, 2007. "Investment, consumption, and hedging under incomplete markets," Journal of Financial Economics, Elsevier, vol. 86(3), pages 608-642, December.
    8. Willi Semmler & Raphaele Chappe, 2012. "Ponzi Finance And The Hedge Fund Industry," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 15(supp0), pages 1-25.
    9. Gordon L Clark & Ashby H B Monk, 2013. "Financial Institutions, Information, and Investing-At-A-Distance," Environment and Planning A, , vol. 45(6), pages 1318-1336, June.
    10. Jacob A. Bikker & Dirk W.G.A. Broeders & Dirk Jan de Dreu, 2010. "Stock Market Performance and Pension Fund Investment Policy: Rebalancing, Free Float, or Market Timing?," International Journal of Central Banking, International Journal of Central Banking, vol. 6(2), pages 53-79, June.
    11. Arts, Jori & Ponds, Eduard, 2016. "The Need for Flexible Take-ups of Home Equity and Pension Wealth in Retirement," Other publications TiSEM 4331ef5b-798f-48dd-8bb5-a, Tilburg University, School of Economics and Management.
    12. Hero Ashman & Seth Neumuller, 2020. "Can Income Differences Explain the Racial Wealth Gap: A Quantitative Analysis," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 35, pages 220-239, January.
    13. Karlis, Alexandros & Galanis, Girogos & Terovitis, Spyridon & Turner, Matthew, 2017. "Heterogeneity and Clustering of Defaults," Economic Research Papers 270011, University of Warwick - Department of Economics.
    14. Gregory H. MacKinnon & Ashraf Al Zaman, 2009. "Real Estate for the Long Term: The Effect of Return Predictability on Long‐Horizon Allocations," Real Estate Economics, American Real Estate and Urban Economics Association, vol. 37(1), pages 117-153, March.
    15. Pelizzon, Loriana & Weber, Guglielmo, 2009. "Efficient portfolios when housing needs change over the life cycle," Journal of Banking & Finance, Elsevier, vol. 33(11), pages 2110-2121, November.
    16. Straub, Ludwig & Ulbricht, Robert, 2015. "Endogenous Uncertainty and Credit Crunches," TSE Working Papers 15-604, Toulouse School of Economics (TSE), revised Dec 2017.
    17. John Y. Campbell & Robert J. Shiller & Luis M. Viceira, 2009. "Understanding Inflation-Indexed Bond Markets," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 40(1 (Spring), pages 79-138.
    18. Kiss, Tamás & Mazur, Stepan & Nguyen, Hoang, 2022. "Predicting returns and dividend growth — The role of non-Gaussian innovations," Finance Research Letters, Elsevier, vol. 46(PA).
    19. Esposito, Federico, 2022. "Demand risk and diversification through international trade," Journal of International Economics, Elsevier, vol. 135(C).
    20. Fabio C. Bagliano & Raffaele Corvino & Carolina Fugazza & Giovanna Nicodano, 2018. "Hedging Labor Income Risk over the Life-Cycle," Working papers 058, Department of Economics and Statistics (Dipartimento di Scienze Economico-Sociali e Matematico-Statistiche), University of Torino.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jecmet:v:16:y:2009:i:4:p:393-408. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/RJEC20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.