IDEAS home Printed from https://ideas.repec.org/p/aah/create/2013-16.html

Nonlinear Forecasting With Many Predictors Using Kernel Ridge Regression

Author

Listed:
  • Peter Exterkate

    (Aarhus University and CREATES)

  • Patrick J.F. Groenen

    (Econometric Institute, Erasmus University Rotterdam)

  • Christiaan Heij

    (Econometric Institute, Erasmus University Rotterdam)

  • Dick van Dijk

    (Econometric Institute, Erasmus University Rotterdam)

Abstract

This paper puts forward kernel ridge regression as an approach for forecasting with many predictors that are related nonlinearly to the target variable. In kernel ridge regression, the observed predictor variables are mapped nonlinearly into a high-dimensional space, where estimation of the predictive regression model is based on a shrinkage estimator to avoid overfitting. We extend the kernel ridge regression methodology to enable its use for economic time-series forecasting, by including lags of the dependent variable or other individual variables as predictors, as typically desired in macroeconomic and financial applications. Monte Carlo simulations as well as an empirical application to various key measures of real economic activity confirm that kernel ridge regression can produce more accurate forecasts than traditional linear and nonlinear methods for dealing with many predictors based on principal component regression.

Suggested Citation

  • Peter Exterkate & Patrick J.F. Groenen & Christiaan Heij & Dick van Dijk, 2013. "Nonlinear Forecasting With Many Predictors Using Kernel Ridge Regression," CREATES Research Papers 2013-16, Department of Economics and Business Economics, Aarhus University.
  • Handle: RePEc:aah:create:2013-16
    as

    Download full text from publisher

    File URL: https://repec.econ.au.dk/repec/creates/rp/13/rp13_16.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bruno Cara Giovannetti, 2013. "Nonlinear Forecasting Using Factor‐Augmented Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 32(1), pages 32-40, January.
    2. Hansen, Bruce E. & Racine, Jeffrey S., 2012. "Jackknife model averaging," Journal of Econometrics, Elsevier, vol. 167(1), pages 38-46.
    3. Engle, Robert F. & White (the late), Halbert (ed.), 1999. "Cointegration, Causality, and Forecasting: Festschrift in Honour of Clive W. J. Granger," OUP Catalogue, Oxford University Press, number 9780198296836.
    4. Jonathan H. Wright, 2009. "Forecasting US inflation by Bayesian model averaging," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 28(2), pages 131-144.
    5. Timo Teräsvirta & Marcelo C. Medeiros & Gianluigi Rech, 2006. "Building neural network models for time series: a statistical approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(1), pages 49-75.
    6. Groen, Jan J.J. & Kapetanios, George, 2016. "Revisiting useful approaches to data-rich macroeconomic forecasting," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 221-239.
    7. Terasvirta, Timo & van Dijk, Dick & Medeiros, Marcelo C., 2005. "Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A re-examination," International Journal of Forecasting, Elsevier, vol. 21(4), pages 755-774.
    8. Bai, Jushan & Ng, Serena, 2008. "Forecasting economic time series using targeted predictors," Journal of Econometrics, Elsevier, vol. 146(2), pages 304-317, October.
    9. Pagan,Adrian & Ullah,Aman, 1999. "Nonparametric Econometrics," Cambridge Books, Cambridge University Press, number 9780521586115, Enero-Abr.
    10. Marta Bańbura & Domenico Giannone & Lucrezia Reichlin, 2010. "Large Bayesian vector auto regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 71-92, January.
    11. De Mol, Christine & Giannone, Domenico & Reichlin, Lucrezia, 2008. "Forecasting using a large number of predictors: Is Bayesian shrinkage a valid alternative to principal components?," Journal of Econometrics, Elsevier, vol. 146(2), pages 318-328, October.
    12. Elliott, Graham & Gargano, Antonio & Timmermann, Allan, 2013. "Complete subset regressions," Journal of Econometrics, Elsevier, vol. 177(2), pages 357-373.
    13. Stock, James H. & Watson, Mark W., 2006. "Forecasting with Many Predictors," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 10, pages 515-554, Elsevier.
    14. Racine, Jeff, 2000. "Consistent cross-validatory model-selection for dependent data: hv-block cross-validation," Journal of Econometrics, Elsevier, vol. 99(1), pages 39-61, November.
    15. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    16. Huiyu Huang & Tae-Hwy Lee, 2010. "To Combine Forecasts or to Combine Information?," Econometric Reviews, Taylor & Francis Journals, vol. 29(5-6), pages 534-570.
    17. James H. Stock & Mark W. Watson, 2007. "Erratum to "Why Has U.S. Inflation Become Harder to Forecast?"," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(7), pages 1849-1849, October.
    18. Andrea Carriero & George Kapetanios & Massimiliano Marcellino, 2011. "Forecasting large datasets with Bayesian reduced rank multivariate models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(5), pages 735-761, August.
    19. Sydney C. Ludvigson & Serena Ng, 2009. "Macro Factors in Bond Risk Premia," The Review of Financial Studies, Society for Financial Studies, vol. 22(12), pages 5027-5067, December.
    20. James H. Stock & Mark W. Watson, 2012. "Generalized Shrinkage Methods for Forecasting Using Many Predictors," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(4), pages 481-493, June.
    21. David E. Rapach & Jack K. Strauss & Guofu Zhou, 2010. "Out-of-Sample Equity Premium Prediction: Combination Forecasts and Links to the Real Economy," The Review of Financial Studies, Society for Financial Studies, vol. 23(2), pages 821-862, February.
    22. Cem Cakmakli & Dick van Dijk, 2010. "Getting the Most out of Macroeconomic Information for Predicting Stock Returns and Volatility," Tinbergen Institute Discussion Papers 10-115/4, Tinbergen Institute.
    23. Marco Aiolfi & Carlo Ambrogio Favero, "undated". "Model Uncertainty, Thick Modelling and the predictability of Stock Returns," Working Papers 221, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    24. Faust, Jon & Wright, Jonathan H., 2009. "Comparing Greenbook and Reduced Form Forecasts Using a Large Realtime Dataset," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 468-479.
    25. Swanson, Norman R & White, Halbert, 1995. "A Model-Selection Approach to Assessing the Information in the Term Structure Using Linear Models and Artificial Neural Networks," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 265-275, July.
    26. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    27. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    28. James H. Stock & Mark W. Watson, 2005. "Implications of Dynamic Factor Models for VAR Analysis," NBER Working Papers 11467, National Bureau of Economic Research, Inc.
    29. Kim, Hyun Hak & Swanson, Norman R., 2014. "Forecasting financial and macroeconomic variables using data reduction methods: New empirical evidence," Journal of Econometrics, Elsevier, vol. 178(P2), pages 352-367.
    30. Terasvirta, Timo, 2006. "Forecasting economic variables with nonlinear models," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 8, pages 413-457, Elsevier.
    31. Clements, Michael P. & Hendry, David F. (ed.), 2011. "The Oxford Handbook of Economic Forecasting," OUP Catalogue, Oxford University Press, number 9780195398649.
    32. Ludvigson, Sydney C. & Ng, Serena, 2007. "The empirical risk-return relation: A factor analysis approach," Journal of Financial Economics, Elsevier, vol. 83(1), pages 171-222, January.
    33. White, Halbert, 2006. "Approximate Nonlinear Forecasting Methods," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 9, pages 459-512, Elsevier.
    34. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    35. Peter Exterkate, 2012. "Model Selection in Kernel Ridge Regression," CREATES Research Papers 2012-10, Department of Economics and Business Economics, Aarhus University.
    36. Groen, Jan J.J. & Kapetanios, George, 2016. "Revisiting useful approaches to data-rich macroeconomic forecasting," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 221-239.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cheng, Xu & Hansen, Bruce E., 2015. "Forecasting with factor-augmented regression: A frequentist model averaging approach," Journal of Econometrics, Elsevier, vol. 186(2), pages 280-293.
    2. Barbara Rossi, 2021. "Forecasting in the Presence of Instabilities: How We Know Whether Models Predict Well and How to Improve Them," Journal of Economic Literature, American Economic Association, vol. 59(4), pages 1135-1190, December.
    3. Çakmaklı, Cem & van Dijk, Dick, 2016. "Getting the most out of macroeconomic information for predicting excess stock returns," International Journal of Forecasting, Elsevier, vol. 32(3), pages 650-668.
    4. Peter Exterkate, 2012. "Model Selection in Kernel Ridge Regression," CREATES Research Papers 2012-10, Department of Economics and Business Economics, Aarhus University.
    5. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    6. Liu, Yang & Swanson, Norman R., 2024. "An assessment of the marginal predictive content of economic uncertainty indexes and business conditions predictors," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1391-1409.
    7. Anders Bredahl Kock & Timo Teräsvirta, 2016. "Forecasting Macroeconomic Variables Using Neural Network Models and Three Automated Model Selection Techniques," Econometric Reviews, Taylor & Francis Journals, vol. 35(8-10), pages 1753-1779, December.
    8. Rachidi Kotchoni & Maxime Leroux & Dalibor Stevanovic, 2019. "Macroeconomic forecast accuracy in a data‐rich environment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(7), pages 1050-1072, November.
    9. Luciani, Matteo, 2014. "Forecasting with approximate dynamic factor models: The role of non-pervasive shocks," International Journal of Forecasting, Elsevier, vol. 30(1), pages 20-29.
    10. Xu Cheng & Bruce E. Hansen, 2012. "Forecasting with Factor-Augmented Regression: A Frequentist Model Averaging Approach, Second Version," PIER Working Paper Archive 13-061, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 03 Sep 2013.
    11. Chen, Qitong & Hong, Yongmiao & Li, Haiqi, 2024. "Time-varying forecast combination for factor-augmented regressions with smooth structural changes," Journal of Econometrics, Elsevier, vol. 240(1).
    12. Kim, Hyun Hak & Swanson, Norman R., 2018. "Mining big data using parsimonious factor, machine learning, variable selection and shrinkage methods," International Journal of Forecasting, Elsevier, vol. 34(2), pages 339-354.
    13. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    14. Rossi, Barbara, 2013. "Advances in Forecasting under Instability," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1203-1324, Elsevier.
    15. Bräuning, Falk & Koopman, Siem Jan, 2014. "Forecasting macroeconomic variables using collapsed dynamic factor analysis," International Journal of Forecasting, Elsevier, vol. 30(3), pages 572-584.
    16. Kim, Hyun Hak & Swanson, Norman R., 2014. "Forecasting financial and macroeconomic variables using data reduction methods: New empirical evidence," Journal of Econometrics, Elsevier, vol. 178(P2), pages 352-367.
    17. Kelly, Bryan & Pruitt, Seth, 2015. "The three-pass regression filter: A new approach to forecasting using many predictors," Journal of Econometrics, Elsevier, vol. 186(2), pages 294-316.
    18. Bae, Juhee, 2024. "Factor-augmented forecasting in big data," International Journal of Forecasting, Elsevier, vol. 40(4), pages 1660-1688.
    19. Kutateladze, Varlam, 2022. "The kernel trick for nonlinear factor modeling," International Journal of Forecasting, Elsevier, vol. 38(1), pages 165-177.
    20. Alessandro Girardi & Roberto Golinelli & Carmine Pappalardo, 2017. "The role of indicator selection in nowcasting euro-area GDP in pseudo-real time," Empirical Economics, Springer, vol. 53(1), pages 79-99, August.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • E27 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Forecasting and Simulation: Models and Applications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aah:create:2013-16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: http://www.econ.au.dk/afn/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.