IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/105368.html
   My bibliography  Save this paper

"Go wild for a while!": A new asymptotically Normal test for forecast evaluation in nested models

Author

Listed:
  • Pincheira, Pablo
  • Hardy, Nicolás
  • Muñoz, Felipe

Abstract

In this paper we present a new asymptotically normal test for out-of-sample evaluation in nested models. Our approach is a simple modification of a traditional encompassing test that is commonly known as Clark and West test (CW). The key point of our strategy is to introduce an independent random variable that prevents the traditional CW test from becoming degenerate under the null hypothesis of equal predictive ability. Using the approach developed by West (1996), we show that in our test the impact of parameter estimation uncertainty vanishes asymptotically. Using a variety of Monte Carlo simulations in iterated multi-step-ahead forecasts we evaluate our test and CW in terms of size and power. These simulations reveal that our approach is reasonably well-sized even at long horizons when CW may present severe size distortions. In terms of power, results are mixed but CW has an edge over our approach. Finally, we illustrate the use of our test with an empirical application in the context of the commodity currencies literature.

Suggested Citation

  • Pincheira, Pablo & Hardy, Nicolás & Muñoz, Felipe, 2021. ""Go wild for a while!": A new asymptotically Normal test for forecast evaluation in nested models," MPRA Paper 105368, University Library of Munich, Germany.
  • Handle: RePEc:pra:mprapa:105368
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/105368/1/MPRA_paper_105368.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Charles Engel & Kenneth D. West, 2005. "Exchange Rates and Fundamentals," Journal of Political Economy, University of Chicago Press, vol. 113(3), pages 485-517, June.
    2. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    3. Yu-Chin Chen & Kenneth S. Rogoff & Barbara Rossi, 2010. "Can Exchange Rates Forecast Commodity Prices?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(3), pages 1145-1194.
    4. Newey, Whitney K & West, Kenneth D, 1987. "Hypothesis Testing with Efficient Method of Moments Estimation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 28(3), pages 777-787, October.
    5. Yock Y. Chong & David F. Hendry, 1986. "Econometric Evaluation of Linear Macro-Economic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 53(4), pages 671-690.
    6. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    7. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, vol. 29(1), pages 13-27.
    8. Stambaugh, Robert F., 1999. "Predictive regressions," Journal of Financial Economics, Elsevier, vol. 54(3), pages 375-421, December.
    9. Whitney K. Newey & Kenneth D. West, 1994. "Automatic Lag Selection in Covariance Matrix Estimation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(4), pages 631-653.
    10. Pincheira, Pablo & Hardy, Nicolás, 2021. "Forecasting aluminum prices with commodity currencies," Resources Policy, Elsevier, vol. 73(C).
    11. Harvey, David I & Leybourne, Stephen J & Newbold, Paul, 1998. "Tests for Forecast Encompassing," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(2), pages 254-259, April.
    12. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
    13. Pincheira, Pablo M. & West, Kenneth D., 2016. "A comparison of some out-of-sample tests of predictability in iterated multi-step-ahead forecasts," Research in Economics, Elsevier, vol. 70(2), pages 304-319.
    14. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    15. Campbell, John Y & Shiller, Robert J, 1987. "Cointegration and Tests of Present Value Models," Journal of Political Economy, University of Chicago Press, vol. 95(5), pages 1062-1088, October.
    16. Pincheira, Pablo & Hardy, Nicolas, 2018. "The predictive relationship between exchange rate expectations and base metal prices," MPRA Paper 89423, University Library of Munich, Germany.
    17. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    18. Pincheira Brown, Pablo & Hardy, Nicolás, 2019. "Forecasting base metal prices with the Chilean exchange rate," Resources Policy, Elsevier, vol. 62(C), pages 256-281.
    19. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    20. Andrews, Donald W K, 1991. "Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation," Econometrica, Econometric Society, vol. 59(3), pages 817-858, May.
    21. Francis X. Diebold, 2015. "Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 33(1), pages 1-1, January.
    22. Gregory Mankiw, N. & Shapiro, Matthew D., 1986. "Do we reject too often? : Small sample properties of tests of rational expectations models," Economics Letters, Elsevier, vol. 20(2), pages 139-145.
    23. Clark, Todd E. & McCracken, Michael W., 2005. "The power of tests of predictive ability in the presence of structural breaks," Journal of Econometrics, Elsevier, vol. 124(1), pages 1-31, January.
    24. Unknown, 1986. "Letters," Choices: The Magazine of Food, Farm, and Resource Issues, Agricultural and Applied Economics Association, vol. 1(4), pages 1-9.
    25. Nelson, Charles R & Kim, Myung J, 1993. "Predictable Stock Returns: The Role of Small Sample Bias," Journal of Finance, American Finance Association, vol. 48(2), pages 641-661, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Valentina Corradi & Sainan Jin & Norman R. Swanson, 2023. "Robust forecast superiority testing with an application to assessing pools of expert forecasters," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 596-622, June.
    2. Pincheira, Pablo & Hardy, Nicolas, 2022. "Correlation Based Tests of Predictability," MPRA Paper 112014, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pablo Pincheira & Nicolás Hardy & Felipe Muñoz, 2021. "“Go Wild for a While!”: A New Test for Forecast Evaluation in Nested Models," Mathematics, MDPI, vol. 9(18), pages 1-28, September.
    2. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    3. Pincheira, Pablo M. & West, Kenneth D., 2016. "A comparison of some out-of-sample tests of predictability in iterated multi-step-ahead forecasts," Research in Economics, Elsevier, vol. 70(2), pages 304-319.
    4. Pablo Pincheira Brown, 2022. "A Power Booster Factor for Out-of-Sample Tests of Predictability," Revista Economía, Fondo Editorial - Pontificia Universidad Católica del Perú, vol. 45(89), pages 150-183.
    5. Nicolás Magner & Nicolás Hardy, 2022. "Cryptocurrency Forecasting: More Evidence of the Meese-Rogoff Puzzle," Mathematics, MDPI, vol. 10(13), pages 1-27, July.
    6. Todd E. Clark & Kenneth D. West, 2005. "Using Out-of-Sample Mean Squared Prediction Errors to Test the Martingale Difference," NBER Technical Working Papers 0305, National Bureau of Economic Research, Inc.
    7. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, vol. 29(1), pages 13-27.
    8. Clark, Todd E. & West, Kenneth D., 2006. "Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 155-186.
    9. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    10. Pincheira, Pablo & Hardy, Nicolas, 2022. "Correlation Based Tests of Predictability," MPRA Paper 112014, University Library of Munich, Germany.
    11. Pablo Pincheira-Brown & Nicolás Hardy & Cristobal Henrriquez & Ignacio Tapia & Andrea Bentancor, 2023. "Forecasting Base Metal Prices with an International Stock Index," Czech Journal of Economics and Finance (Finance a uver), Charles University Prague, Faculty of Social Sciences, vol. 73(3), pages 277-302, October.
    12. Pincheira, Pablo & Hardy, Nicolás, 2021. "Forecasting aluminum prices with commodity currencies," Resources Policy, Elsevier, vol. 73(C).
    13. West, Kenneth D., 2006. "Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 3, pages 99-134, Elsevier.
    14. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    15. Clark, Todd E. & West, Kenneth D., 2007. "Approximately normal tests for equal predictive accuracy in nested models," Journal of Econometrics, Elsevier, vol. 138(1), pages 291-311, May.
    16. Rapach, David E. & Wohar, Mark E., 2006. "In-sample vs. out-of-sample tests of stock return predictability in the context of data mining," Journal of Empirical Finance, Elsevier, vol. 13(2), pages 231-247, March.
    17. Brooks, Chris & Burke, Simon P. & Stanescu, Silvia, 2016. "Finite sample weighting of recursive forecast errors," International Journal of Forecasting, Elsevier, vol. 32(2), pages 458-474.
    18. Pincheira-Brown, Pablo & Bentancor, Andrea & Hardy, Nicolás & Jarsun, Nabil, 2022. "Forecasting fuel prices with the Chilean exchange rate: Going beyond the commodity currency hypothesis," Energy Economics, Elsevier, vol. 106(C).
    19. Rapach, David E. & Wohar, Mark E. & Rangvid, Jesper, 2005. "Macro variables and international stock return predictability," International Journal of Forecasting, Elsevier, vol. 21(1), pages 137-166.
    20. Joseph P. Byrne & Dimitris Korobilis & Pinho J. Ribeiro, 2018. "On The Sources Of Uncertainty In Exchange Rate Predictability," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 59(1), pages 329-357, February.

    More about this item

    Keywords

    forecasting; random walk; out-of-sample; prediction; mean square prediction error;
    All these keywords.

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • G17 - Financial Economics - - General Financial Markets - - - Financial Forecasting and Simulation

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:105368. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.