IDEAS home Printed from https://ideas.repec.org/p/fip/fedlwp/2015-025.html
   My bibliography  Save this paper

Tests of Equal Accuracy for Nested Models with Estimated Factors

Author

Abstract

In this paper we develop asymptotics for tests of equal predictive ability between nested models when factor-augmented regression models are used to forecast. We provide conditions under which the estimation of the factors does not affect the asymptotic distributions developed in Clark and McCracken (2001) and McCracken (2007). This enables researchers to use the existing tabulated critical values when conducting inference. As an intermediate result, we derive the asymptotic properties of the principal components estimator over recursive windows. We provide simulation evidence on the finite sample effects of factor estimation and apply the tests to the case of forecasting excess returns to the S&P 500 Composite Index.

Suggested Citation

  • Silvia Goncalves & Michael W. McCracken & Benoit Perron, 2015. "Tests of Equal Accuracy for Nested Models with Estimated Factors," Working Papers 2015-25, Federal Reserve Bank of St. Louis.
  • Handle: RePEc:fip:fedlwp:2015-025
    as

    Download full text from publisher

    File URL: https://research.stlouisfed.org/wp/2015/2015-025.pdf
    File Function: Full text
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    2. Gonçalves, Sílvia & Perron, Benoit, 2014. "Bootstrapping factor-augmented regression models," Journal of Econometrics, Elsevier, vol. 182(1), pages 156-173.
    3. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    4. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    5. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    6. Shintani, Mototsugu, 2005. "Nonlinear Forecasting Analysis Using Diffusion Indexes: An Application to Japan," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 37(3), pages 517-538, June.
    7. repec:taf:jnlbes:v:30:y:2012:i:1:p:53-66 is not listed on IDEAS
    8. Todd Clark & Michael McCracken, 2005. "Evaluating Direct Multistep Forecasts," Econometric Reviews, Taylor & Francis Journals, vol. 24(4), pages 369-404.
    9. Peter Reinhard Hansen & Allan Timmermann, 2015. "Equivalence Between Out‐of‐Sample Forecast Comparisons and Wald Statistics," Econometrica, Econometric Society, vol. 83, pages 2485-2505, November.
    10. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    11. Matteo Ciccarelli & Benoît Mojon, 2010. "Global Inflation," The Review of Economics and Statistics, MIT Press, vol. 92(3), pages 524-535, August.
    12. Elena Andreou & Eric Ghysels & Andros Kourtellos, 2013. "Should Macroeconomic Forecasters Use Daily Financial Data and How?," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(2), pages 240-251, April.
    13. James H. Stock & Mark W.Watson, 2003. "Forecasting Output and Inflation: The Role of Asset Prices," Journal of Economic Literature, American Economic Association, vol. 41(3), pages 788-829, September.
    14. Cheng, Xu & Hansen, Bruce E., 2015. "Forecasting with factor-augmented regression: A frequentist model averaging approach," Journal of Econometrics, Elsevier, vol. 186(2), pages 280-293.
    15. Jack Fosten, 2016. "Forecast evaluation with factor-augmented models," University of East Anglia School of Economics Working Paper Series 2016-05, School of Economics, University of East Anglia, Norwich, UK..
    16. Çakmaklı, Cem & van Dijk, Dick, 2016. "Getting the most out of macroeconomic information for predicting excess stock returns," International Journal of Forecasting, Elsevier, vol. 32(3), pages 650-668.
    17. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    18. Han, Xu & Inoue, Atsushi, 2015. "Tests For Parameter Instability In Dynamic Factor Models," Econometric Theory, Cambridge University Press, vol. 31(5), pages 1117-1152, October.
    19. Murphy, Kevin M & Topel, Robert H, 2002. "Estimation and Inference in Two-Step Econometric Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 88-97, January.
    20. Bai, Jushan & Ng, Serena, 2013. "Principal components estimation and identification of static factors," Journal of Econometrics, Elsevier, vol. 176(1), pages 18-29.
    21. Adrian Pagan, 1986. "Two Stage and Related Estimators and Their Applications," Review of Economic Studies, Oxford University Press, vol. 53(4), pages 517-538.
    22. Hjalmarsson, Erik, 2010. "Predicting Global Stock Returns," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 45(1), pages 49-80, February.
    23. Sydney C. Ludvigson & Serena Ng, 2009. "A Factor Analysis of Bond Risk Premia," NBER Working Papers 15188, National Bureau of Economic Research, Inc.
    24. Hofmann, Boris, 2009. "Do monetary indicators lead euro area inflation?," Journal of International Money and Finance, Elsevier, vol. 28(7), pages 1165-1181, November.
    25. Chamberlain, Gary & Rothschild, Michael, 1983. "Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets," Econometrica, Econometric Society, vol. 51(5), pages 1281-1304, September.
    26. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    27. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    28. Jennie Bai, 2010. "Equity premium predictions with adaptive macro indexes," Staff Reports 475, Federal Reserve Bank of New York.
    29. Cem Cakmakli & Dick van Dijk, 2010. "Getting the Most out of Macroeconomic Information for Predicting Stock Returns and Volatility," Tinbergen Institute Discussion Papers 10-115/4, Tinbergen Institute.
    30. Jushan Bai, 2009. "Panel Data Models With Interactive Fixed Effects," Econometrica, Econometric Society, vol. 77(4), pages 1229-1279, July.
    31. Pagan, Adrian, 1984. "Econometric Issues in the Analysis of Regressions with Generated Regressors," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 25(1), pages 221-247, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    2. Luiz Renato Lima & Lucas Lúcio Godeiro & Mohammed Mohsin, 2021. "Time-Varying Dictionary and the Predictive Power of FED Minutes," Computational Economics, Springer;Society for Computational Economics, vol. 57(1), pages 149-181, January.
    3. Alessandro Giovannelli & Daniele Massacci & Stefano Soccorsi, 2020. "Forecasting Stock Returns with Large Dimensional Factor Models," Working Papers 305661169, Lancaster University Management School, Economics Department.
    4. Andreou, Elena & Ghysels, Eric, 2021. "Predicting the VIX and the volatility risk premium: The role of short-run funding spreads Volatility Factors," Journal of Econometrics, Elsevier, vol. 220(2), pages 366-398.
    5. In Choi & Hanbat Jeong, 2020. "Differencing versus nondifferencing in factor‐based forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(6), pages 728-750, September.
    6. Antoine A. Djogbenou, 2021. "Model selection in factor-augmented regressions with estimated factors," Econometric Reviews, Taylor & Francis Journals, vol. 40(5), pages 470-503, April.
    7. Daniel Borup & Bent Jesper Christensen & Yunus Emre Ergemen, 2019. "Assessing predictive accuracy in panel data models with long-range dependence," CREATES Research Papers 2019-04, Department of Economics and Business Economics, Aarhus University.
    8. Daniel Borup & Martin Thyrsgaard, 2017. "Statistical tests for equal predictive ability across multiple forecasting methods," CREATES Research Papers 2017-19, Department of Economics and Business Economics, Aarhus University.
    9. Fosten, Jack, 2017. "Confidence intervals in regressions with estimated factors and idiosyncratic components," Economics Letters, Elsevier, vol. 157(C), pages 71-74.
    10. Michael W. McCracken, 2020. "Tests of Conditional Predictive Ability: Existence, Size, and Power," Working Papers 2020-050, Federal Reserve Bank of St. Louis.
    11. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2020. "Predicting bond return predictability," CREATES Research Papers 2020-09, Department of Economics and Business Economics, Aarhus University.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    2. Jack Fosten, 2016. "Forecast evaluation with factor-augmented models," University of East Anglia School of Economics Working Paper Series 2016-05, School of Economics, University of East Anglia, Norwich, UK..
    3. Varlam Kutateladze, 2021. "The Kernel Trick for Nonlinear Factor Modeling," Papers 2103.01266, arXiv.org.
    4. Firmin Doko Tchatoka & Qazi Haque, 2020. "On bootstrapping tests of equal forecast accuracy for nested models," Economics Discussion / Working Papers 20-06, The University of Western Australia, Department of Economics.
    5. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    6. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," ISER Discussion Paper 1053r, Institute of Social and Economic Research, Osaka University, revised Mar 2020.
    7. In Choi & Hanbat Jeong, 2020. "Differencing versus nondifferencing in factor‐based forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(6), pages 728-750, September.
    8. Norman R. Swanson & Weiqi Xiong, 2018. "Big data analytics in economics: What have we learned so far, and where should we go from here?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 51(3), pages 695-746, August.
    9. Mayer, Walter J. & Liu, Feng & Dang, Xin, 2017. "Improving the power of the Diebold–Mariano–West test for least squares predictions," International Journal of Forecasting, Elsevier, vol. 33(3), pages 618-626.
    10. Clark, Todd E. & McCracken, Michael W., 2015. "Nested forecast model comparisons: A new approach to testing equal accuracy," Journal of Econometrics, Elsevier, vol. 186(1), pages 160-177.
    11. Mogliani, Matteo & Darné, Olivier & Pluyaud, Bertrand, 2017. "The new MIBA model: Real-time nowcasting of French GDP using the Banque de France's monthly business survey," Economic Modelling, Elsevier, vol. 64(C), pages 26-39.
    12. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    13. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    14. Todd E. Clark & Michael W. McCracken, 2009. "Combining Forecasts from Nested Models," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(3), pages 303-329, June.
    15. McGurk, Zachary, 2020. "US real estate inflation prediction: Exchange rates and net foreign assets," The Quarterly Review of Economics and Finance, Elsevier, vol. 75(C), pages 53-66.
    16. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," DSSR Discussion Papers 96, Graduate School of Economics and Management, Tohoku University.
    17. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, vol. 29(1), pages 13-27.
    18. Yoshimasa Uematsu & Takashi Yamagata, 2020. "Inference in Weak Factor Models," ISER Discussion Paper 1080, Institute of Social and Economic Research, Osaka University.
    19. Clark, Todd E. & McCracken, Michael W., 2009. "Tests of Equal Predictive Ability With Real-Time Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 441-454.
    20. Daniel Borup & Jonas N. Eriksen & Mads M. Kjær & Martin Thyrsgaard, 2020. "Predicting bond return predictability," CREATES Research Papers 2020-09, Department of Economics and Business Economics, Aarhus University.

    More about this item

    Keywords

    factor model; out-of-sample forecasts; recursive estimation;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fip:fedlwp:2015-025. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/frbslus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/frbslus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.