Nested forecast model comparisons: a new approach to testing equal accuracy
AbstractThis paper develops bootstrap methods for testing whether, in a finite sample, competing out-of-sample forecasts from nested models are equally accurate. Most prior work on forecast tests for nested models has focused on a null hypothesis of equal accuracy in population basically, whether coefficients on the extra variables in the larger, nesting model are zero. We instead use an asymptotic approximation that treats the coefficients as non-zero but small, such that, in a finite sample, forecasts from the small model are expected to be as accurate as forecasts from the large model. Under that approximation, we derive the limiting distributions of pairwise tests of equal mean square error, and develop bootstrap methods for estimating critical values. Monte Carlo experiments show that our proposed procedures have good size and power properties for the null of equal finite-sample forecast accuracy. We illustrate the use of the procedures with applications to forecasting stock returns and inflation.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Federal Reserve Bank of Kansas City in its series Research Working Paper with number RWP 09-11.
Date of creation: 2009
Date of revision:
Other versions of this item:
- Todd E. Clark & Michael W. McCracken, 2009. "Nested forecast model comparisons: a new approach to testing equal accuracy," Working Papers 2009-050, Federal Reserve Bank of St. Louis.
- NEP-ALL-2009-09-11 (All new papers)
- NEP-CBA-2009-09-11 (Central Banking)
- NEP-ECM-2009-09-11 (Econometrics)
- NEP-ETS-2009-09-11 (Econometric Time Series)
- NEP-FOR-2009-09-11 (Forecasting)
- NEP-MIC-2009-09-11 (Microeconomics)
You can help add them by filling out this form.
CitEc Project, subscribe to its RSS feed for this item.
- Francesco Ravazzolo & Philip Rothman, 2013.
"Oil and U.S. GDP: A Real‐Time Out‐of‐Sample Examination,"
Journal of Money, Credit and Banking,
Blackwell Publishing, vol. 45(2-3), pages 449-463, 03.
- Francesco Ravazzolo & Philip Rothman, 2010. "Oil and US GDP: A real-time out-of-sample examination," Working Paper 2010/18, Norges Bank.
- Francesco Ravazzolo & Philip Rothman, 2011. "Oil and US GDP: A Real-Time out-of Sample Examination," Working Papers 0004, Centre for Applied Macro- and Petroleum economics (CAMP), BI Norwegian Business School.
- Ron Alquist & Lutz Kilian & Robert J. Vigfusson, 2011.
"Forecasting the price of oil,"
International Finance Discussion Papers
1022, Board of Governors of the Federal Reserve System (U.S.).
- Todd E. Clark & Michael W. McCracken, 2011.
"Advances in forecast evaluation,"
2011-025, Federal Reserve Bank of St. Louis.
- Arai, Natsuki, 2014. "Using forecast evaluation to improve the accuracy of the Greenbook forecast," International Journal of Forecasting, Elsevier, vol. 30(1), pages 12-19.
- Andrea CARRIERO & Todd E. CLARK & Massimiliano MARCELLINO, 2012.
"Common Drifting Volatility in Large Bayesian VARs,"
Economics Working Papers
ECO2012/08, European University Institute.
- Carriero, Andrea & Clark, Todd & Marcellino, Massimiliano, 2012. "Common Drifting Volatility in Large Bayesian VARs," CEPR Discussion Papers 8894, C.E.P.R. Discussion Papers.
- Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2012. "Common drifting volatility in large Bayesian VARs," Working Paper 1206, Federal Reserve Bank of Cleveland.
- Lutz Kilian & Robert J. Vigfusson, 2012.
"Do oil prices help forecast U.S. real GDP? the role of nonlinearities and asymmetries,"
International Finance Discussion Papers
1050, Board of Governors of the Federal Reserve System (U.S.).
- Lutz Kilian & Robert J. Vigfusson, 2013. "Do Oil Prices Help Forecast U.S. Real GDP? The Role of Nonlinearities and Asymmetries," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(1), pages 78-93, January.
- Kilian, Lutz & Vigfusson, Robert J., 2012. "Do Oil Prices Help Forecast U.S. Real GDP? The Role of Nonlinearities and Asymmetries," CEPR Discussion Papers 8980, C.E.P.R. Discussion Papers.
- Calhoun, Gray, 2014. "Out-Of-Sample Comparisons of Overfit Models," Staff General Research Papers 32462, Iowa State University, Department of Economics.
- Guérin, Pierre & Maurin, Laurent & Mohr, Matthias, 2011. "Trend-cycle decomposition of output and euro area inflation forecasts: a real-time approach based on model combination," Working Paper Series 1384, European Central Bank.
- Todd E. Clark & Michael W. McCracken, 2010. "Reality checks and nested forecast model comparisons," Working Papers 2010-032, Federal Reserve Bank of St. Louis.
- Todd E. Clark & Taeyoung Doh, 2011.
"A Bayesian evaluation of alternative models of trend inflation,"
1134, Federal Reserve Bank of Cleveland.
- Todd E.Clark & Taeyoung Doh, 2011. "A Bayesian evaluation of alternative models of trend inflation," Research Working Paper RWP 11-16, Federal Reserve Bank of Kansas City.
- Nikolsko-Rzhevskyy, Alex & Prodan, Ruxandra, 2012. "Markov switching and exchange rate predictability," International Journal of Forecasting, Elsevier, vol. 28(2), pages 353-365.
- Todd E. Clark & Michael W. McCracken, 2010. "Testing for unconditional predictive ability," Working Papers 2010-031, Federal Reserve Bank of St. Louis.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Lu Dayrit).
If references are entirely missing, you can add them using this form.