IDEAS home Printed from https://ideas.repec.org/p/fip/fedgif/959.html
   My bibliography  Save this paper

The fragility of sensitivity analysis: an encompassing perspective

Author

Abstract

Robustness and fragility in Leamer's sense are defined with respect to a particular coefficient over a class of models. This paper shows that inclusion of the data generation process in that class of models is neither necessary nor sufficient for robustness. This result holds even if the properly specified model has well-determined, statistically significant coefficients. The encompassing principle explains how this result can occur. Encompassing also provides a link to a more common-sense notion of robustness, which is still a desirable property empirically; and encompassing clarifies recent discussion on model averaging and the pooling of forecasts.

Suggested Citation

  • Neil R. Ericsson, 2008. "The fragility of sensitivity analysis: an encompassing perspective," International Finance Discussion Papers 959, Board of Governors of the Federal Reserve System (U.S.).
    • Handle: RePEc:fip:fedgif:959
    as

    Download full text from publisher

    File URL: http://www.federalreserve.gov/pubs/ifdp/2008/959/default.htm
    Download Restriction: no
    File URL: http://www.federalreserve.gov/pubs/ifdp/2008/959/ifdp959.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
    2. Freille, Sebastian & Haque, M. Emranul & Kneller, Richard, 2007. "A contribution to the empirics of press freedom and corruption," European Journal of Political Economy, Elsevier, vol. 23(4), pages 838-862, December.
    3. Hendry, David F, 1985. "Monetary Economic Myth and Econometric Reality," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 1(1), pages 72-84, Spring.
    4. Paruolo, Paolo, 1996. "On the determination of integration indices in I(2) systems," Journal of Econometrics, Elsevier, vol. 72(1-2), pages 313-356.
    5. Christian Bjørnskov & Axel Dreher & Justina Fischer, 2008. "Cross-country determinants of life satisfaction: exploring different determinants across groups in society," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 30(1), pages 119-173, January.
    6. David Hendry & Carlos Santos, 2007. "AUTOMATIC TESTS for SUPER EXOGENEITY," Working Papers de Economia (Economics Working Papers) 11, Católica Porto Business School, Universidade Católica Portuguesa.
    7. Yock Y. Chong & David F. Hendry, 1986. "Econometric Evaluation of Linear Macro-Economic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 671-690.
    8. David F. Hendry & Hans-Martin Krolzig, 2005. "The Properties of Automatic "GETS" Modelling," Economic Journal, Royal Economic Society, vol. 115(502), pages 32-61, March.
    9. Gouriéroux, Christian & Monfort, Alain, 1995. "Testing, Encompassing, and Simulating Dynamic Econometric Models," Econometric Theory, Cambridge University Press, vol. 11(2), pages 195-228, February.
    10. McAleer, Michael & Pagan, Adrian R & Volker, Paul A, 1985. "What Will Take the Con out of Econometrics?," American Economic Review, American Economic Association, vol. 75(3), pages 293-307, June.
    11. Levine, Ross & Renelt, David, 1992. "A Sensitivity Analysis of Cross-Country Growth Regressions," American Economic Review, American Economic Association, vol. 82(4), pages 942-963, September.
    12. Brunner, Karl & Meltzer, Allan H., 1976. "The Phillips curve," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 1-18, January.
    13. Mizon, Grayham E & Richard, Jean-Francois, 1986. "The Encompassing Principle and Its Application to Testing Non-nested Hypotheses," Econometrica, Econometric Society, vol. 54(3), pages 657-678, May.
    14. Edwards Jeffrey A & Sams Alfred & Yang Benhua, 2006. "A Refinement in the Specification of Empirical Macroeconomic Models as an Extension to the EBA Procedure," The B.E. Journal of Macroeconomics, De Gruyter, vol. 6(2), pages 1-26, October.
    15. Stephen M. Goldfeld, 1976. "The Case of the Missing Money," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 7(3), pages 683-740.
    16. Hendry, David F. & Pagan, Adrian R. & Sargan, J.Denis, 1984. "Dynamic specification," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 18, pages 1023-1100, Elsevier.
    17. Cooley, Thomas F & LeRoy, Stephen F, 1981. "Identification and Estimation of Money Demand," American Economic Review, American Economic Association, vol. 71(5), pages 825-844, December.
    18. Gilbert, Christopher L, 1986. "Professor Hendry's Econometric Methodology," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 48(3), pages 283-307, August.
    19. Kiviet, Jan F. & Phillips, Garry D. A., 1994. "Bias assessment and reduction in linear error-correction models," Journal of Econometrics, Elsevier, vol. 63(1), pages 215-243, July.
    20. Hendry, David F. & Ericsson, Neil R., 1991. "Modeling the demand for narrow money in the United Kingdom and the United States," European Economic Review, Elsevier, vol. 35(4), pages 833-881, May.
    21. Carlos Santos & David Hendry & Soren Johansen, 2008. "Automatic selection of indicators in a fully saturated regression," Computational Statistics, Springer, vol. 23(2), pages 317-335, April.
    22. Allen, Stuart D & Connolly, Robert A, 1989. "Financial Market Effects on Aggregate Money Demand: A Bayesian Analysis," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 21(2), pages 158-175, May.
    23. Danila Serra, 2006. "Empirical determinants of corruption: A sensitivity analysis," Public Choice, Springer, vol. 126(1), pages 225-256, January.
    24. Johansen, Soren, 1992. "Testing weak exogeneity and the order of cointegration in UK money demand data," Journal of Policy Modeling, Elsevier, vol. 14(3), pages 313-334, June.
    25. McAleer, Michael & Veall, Michael R, 1989. "How Fragile Are Fragile Inferences? A Re-evaluation of the Deterrent Effect of Capital Punishment," The Review of Economics and Statistics, MIT Press, vol. 71(1), pages 99-106, February.
    26. Engle, Robert F. & Hendry, David F., 1993. "Testing superexogeneity and invariance in regression models," Journal of Econometrics, Elsevier, vol. 56(1-2), pages 119-139, March.
    27. Ericsson, Neil R. & Campos, Julia & Tran, Hong-Anh, 1990. "Pc-Give and David Hendry'S Econometric Methodology," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 10(1), April.
    28. Hendry, D.F. & Richard, J.-F., 1987. "Recent developments in the theory of encompassing," LIDAM Discussion Papers CORE 1987022, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    29. Lu, Maozu & Mizon, Grayham E., 1996. "The Encompassing Principle and Hypothesis Testing," Econometric Theory, Cambridge University Press, vol. 12(5), pages 845-858, December.
    30. Magee, Lonnie, 1990. "The Asymptotic Variance of Extreme Bounds," The Review of Economics and Statistics, MIT Press, vol. 72(1), pages 182-184, February.
    31. Ericsson, Neil R., 1992. "Parameter constancy, mean square forecast errors, and measuring forecast performance: An exposition, extensions, and illustration," Journal of Policy Modeling, Elsevier, vol. 14(4), pages 465-495, August.
    32. Cuthbertson, Keith, 1988. "The Demand for M1: A Forward Looking Buffer Stock Model," Oxford Economic Papers, Oxford University Press, vol. 40(1), pages 110-131, March.
    33. Kevin D. Hoover & Stephen J. Perez, 2004. "Truth and Robustness in Cross‐country Growth Regressions," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 66(5), pages 765-798, December.
    34. Leamer, Edward E, 1997. "Revisiting Tobin's 1950 Study of Food Expenditure: Reply," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 12(5), pages 559-561, Sept.-Oct.
    35. Ericsson, Neil R & Marquez, Jaime, 1993. "Encompassing the Forecasts of U.S. Trade Balance Models," The Review of Economics and Statistics, MIT Press, vol. 75(1), pages 19-31, February.
    36. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    37. Wright, Jonathan H., 2008. "Bayesian Model Averaging and exchange rate forecasts," Journal of Econometrics, Elsevier, vol. 146(2), pages 329-341, October.
    38. Bontemps, Christophe & Mizon, Grayham E., 2001. "Congruence and encompassing," Discussion Paper Series In Economics And Econometrics 0107, Economics Division, School of Social Sciences, University of Southampton.
    39. Granger, Clive W. J. & Uhlig, Harald F., 1990. "Reasonable extreme-bounds analysis," Journal of Econometrics, Elsevier, vol. 44(1-2), pages 159-170.
    40. Hansen, Bruce E, 2002. "Tests for Parameter Instability in Regressions with I(1) Processes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 45-59, January.
    41. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    42. David F. Hendry, 2013. "Econometric Modelling: The ‘Consumption Function’ In Retrospect," Scottish Journal of Political Economy, Scottish Economic Society, vol. 60(5), pages 495-522, November.
    43. Spanos,Aris, 1986. "Statistical Foundations of Econometric Modelling," Cambridge Books, Cambridge University Press, number 9780521269124.
    44. Sala-i-Martin, Xavier, 1997. "I Just Ran Two Million Regressions," American Economic Review, American Economic Association, vol. 87(2), pages 178-183, May.
    45. Leamer, Edward E, 1997. "Revisiting Tobin's 1950 Study of Food Expenditure," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 12(5), pages 533-553, Sept.-Oct.
    46. David F. Hendry & Hans-Martin Krolzig, 1999. "Improving on 'Data mining reconsidered' by K.D. Hoover and S.J. Perez," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 202-219.
    47. Hendry, David F, 1988. "The Encompassing Implications of Feedback versus Feedforward Mechanisms in Econometrics," Oxford Economic Papers, Oxford University Press, vol. 40(1), pages 132-149, March.
    48. Leamer, Edward E & Leonard, Herman B, 1983. "Reporting the Fragility of Regression Estimates," The Review of Economics and Statistics, MIT Press, vol. 65(2), pages 306-317, May.
    49. David F. Hendry & Michael P. Clements, 2004. "Pooling of forecasts," Econometrics Journal, Royal Economic Society, vol. 7(1), pages 1-31, June.
    50. Søren Johansen & Bent Nielsen, 2008. "An analysis of the indicator saturation estimator as a robust regression," Discussion Papers 08-03, University of Copenhagen. Department of Economics.
    51. Stewart, Mark B., 1984. "Significance tests in the presence of model uncertainty and specification search," Economics Letters, Elsevier, vol. 16(3-4), pages 309-313.
    52. Lucas, Robert Jr, 1976. "Econometric policy evaluation: A critique," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 1(1), pages 19-46, January.
    53. Govaerts, Bernadette & Hendry, David F. & Richard, Jean-Francois, 1994. "Encompassing in stationary linear dynamic models," Journal of Econometrics, Elsevier, vol. 63(1), pages 245-270, July.
    54. Leamer, Edward E, 1985. "Sensitivity Analyses Would Help," American Economic Review, American Economic Association, vol. 75(3), pages 308-313, June.
    55. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Butler, Alexander W. & Keefe, Michael O'Connor & Kieschnick, Robert, 2014. "Robust determinants of IPO underpricing and their implications for IPO research," Journal of Corporate Finance, Elsevier, vol. 27(C), pages 367-383.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Julia Campos & Neil R. Ericsson & David F. Hendry, 2005. "General-to-specific modeling: an overview and selected bibliography," International Finance Discussion Papers 838, Board of Governors of the Federal Reserve System (U.S.).
    2. Neil R. Ericsson, 2001. "Forecast uncertainty in economic modeling," International Finance Discussion Papers 697, Board of Governors of the Federal Reserve System (U.S.).
    3. Neil R. Ericsson, 2000. "Predictable uncertainty in economic forecasting," International Finance Discussion Papers 695, Board of Governors of the Federal Reserve System (U.S.).
    4. David Hendry & Carlos Santos, 2010. "An Automatic Test of Super Exogeneity," Economics Series Working Papers 476, University of Oxford, Department of Economics.
    5. Ericsson, Neil R., 1992. "Cointegration, exogeneity, and policy analysis: An overview," Journal of Policy Modeling, Elsevier, vol. 14(3), pages 251-280, June.
    6. Cheong, ChongCheul, 2003. "Regime changes and econometric modeling of the demand for money in Korea," Economic Modelling, Elsevier, vol. 20(3), pages 437-453, May.
    7. Ericsson, Neil R., 2017. "Economic forecasting in theory and practice: An interview with David F. Hendry," International Journal of Forecasting, Elsevier, vol. 33(2), pages 523-542.
    8. Faust, Jon & Whiteman, Charles H., 1997. "General-to-specific procedures for fitting a data-admissible, theory-inspired, congruent, parsimonious, encompassing, weakly-exogenous, identified, structural model to the DGP: A translation and criti," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 47(1), pages 121-161, December.
    9. Yang, Haisheng & He, Jie & Chen, Shaoling, 2015. "The fragility of the Environmental Kuznets Curve: Revisiting the hypothesis with Chinese data via an “Extreme Bound Analysis”," Ecological Economics, Elsevier, vol. 109(C), pages 41-58.
    10. Harris Dellas & Kevin Hoover, 2003. "Truth and Robustness in Cross-country Growth Regressions," Working Papers 11, University of California, Davis, Department of Economics.
    11. Ericsson, Neil R & Hendry, David F & Mizon, Grayham E, 1998. "Exogeneity, Cointegration, and Economic Policy Analysis," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(4), pages 370-387, October.
    12. David F. Hendry & Grayham E. Mizon, 2016. "Improving the teaching of econometrics," Cogent Economics & Finance, Taylor & Francis Journals, vol. 4(1), pages 1170096-117, December.
    13. Bardsen, Gunnar & Eitrheim, Oyvind & Jansen, Eilev S. & Nymoen, Ragnar, 2005. "The Econometrics of Macroeconomic Modelling," OUP Catalogue, Oxford University Press, number 9780199246502.
    14. Neil R. Ericsson & Steven B. Kamin, 2008. "Constructive data mining: modeling Argentine broad money demand," International Finance Discussion Papers 943, Board of Governors of the Federal Reserve System (U.S.).
    15. Kevin D. Hoover & Stephen J. Perez, 2004. "Truth and Robustness in Cross‐country Growth Regressions," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 66(5), pages 765-798, December.
    16. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    17. Rup Singh, 2015. "Forces of economic growth in China, India, and other Asian countries," Asian-Pacific Economic Literature, The Crawford School, The Australian National University, vol. 29(1), pages 62-81, May.
    18. Hendry, David F & Ericsson, Neil R, 1991. "An Econometric Analysis of U.K. Money Demand in 'Monetary Trends in the United States and the United Kingdom' by Milton Friedman and Anna Schwartz," American Economic Review, American Economic Association, vol. 81(1), pages 8-38, March.
    19. Ericsson Neil R., 2008. "Comment on "Economic Forecasting in a Changing World" (by Michael Clements and David Hendry)," Capitalism and Society, De Gruyter, vol. 3(2), pages 1-18, October.
    20. Ahmed, Walid M.A., 2022. "Robust drivers of Bitcoin price movements: An extreme bounds analysis," The North American Journal of Economics and Finance, Elsevier, vol. 62(C).

    More about this item

    Keywords

    Econometric models; Sensitivity theory (Mathematics);

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedgif:959. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ryan Wolfslayer ; Keisha Fournillier (email available below). General contact details of provider: https://edirc.repec.org/data/frbgvus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.