IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v32y2016i3p736-753.html
   My bibliography  Save this article

Nonlinear forecasting with many predictors using kernel ridge regression

Author

Listed:
  • Exterkate, Peter
  • Groenen, Patrick J.F.
  • Heij, Christiaan
  • van Dijk, Dick

Abstract

This paper puts forward kernel ridge regression as an approach for forecasting with many predictors that are related to the target variable nonlinearly. In kernel ridge regression, the observed predictor variables are mapped nonlinearly into a high-dimensional space, where estimation of the predictive regression model is based on a shrinkage estimator in order to avoid overfitting. We extend the kernel ridge regression methodology to enable its use for economic time series forecasting, by including lags of the dependent variable or other individual variables as predictors, as is typically desired in macroeconomic and financial applications. Both Monte Carlo simulations and an empirical application to various key measures of real economic activity confirm that kernel ridge regression can produce more accurate forecasts than traditional linear and nonlinear methods for dealing with many predictors based on principal components.

Suggested Citation

  • Exterkate, Peter & Groenen, Patrick J.F. & Heij, Christiaan & van Dijk, Dick, 2016. "Nonlinear forecasting with many predictors using kernel ridge regression," International Journal of Forecasting, Elsevier, vol. 32(3), pages 736-753.
  • Handle: RePEc:eee:intfor:v:32:y:2016:i:3:p:736-753
    DOI: 10.1016/j.ijforecast.2015.11.017
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207016000182
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ijforecast.2015.11.017?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Marta Bańbura, 2008. "Large Bayesian VARs," 2008 Meeting Papers 334, Society for Economic Dynamics.
    2. Pagan,Adrian & Ullah,Aman, 1999. "Nonparametric Econometrics," Cambridge Books, Cambridge University Press, number 9780521355643.
    3. James H. Stock & Mark W. Watson, 2005. "Implications of Dynamic Factor Models for VAR Analysis," NBER Working Papers 11467, National Bureau of Economic Research, Inc.
    4. Marta Banbura & Domenico Giannone & Lucrezia Reichlin, 2010. "Large Bayesian vector auto regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 71-92.
    5. Huiyu Huang & Tae-Hwy Lee, 2010. "To Combine Forecasts or to Combine Information?," Econometric Reviews, Taylor & Francis Journals, vol. 29(5-6), pages 534-570.
    6. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    7. Groen, Jan J.J. & Kapetanios, George, 2016. "Revisiting useful approaches to data-rich macroeconomic forecasting," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 221-239.
    8. De Mol, Christine & Giannone, Domenico & Reichlin, Lucrezia, 2006. "Forecasting using a large number of predictors: is Bayesian regression a valid alternative to principal components?," Discussion Paper Series 1: Economic Studies 2006,32, Deutsche Bundesbank.
    9. Bruno Cara Giovannetti, 2013. "Nonlinear Forecasting Using Factor‐Augmented Models," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 32(1), pages 32-40, January.
    10. Jonathan H. Wright, 2009. "Forecasting US inflation by Bayesian model averaging," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 28(2), pages 131-144.
    11. Bai, Jushan & Ng, Serena, 2008. "Forecasting economic time series using targeted predictors," Journal of Econometrics, Elsevier, vol. 146(2), pages 304-317, October.
    12. Elliott, Graham & Gargano, Antonio & Timmermann, Allan, 2013. "Complete subset regressions," Journal of Econometrics, Elsevier, vol. 177(2), pages 357-373.
    13. Ludvigson, Sydney C. & Ng, Serena, 2007. "The empirical risk-return relation: A factor analysis approach," Journal of Financial Economics, Elsevier, vol. 83(1), pages 171-222, January.
    14. Stock, James H. & Watson, Mark W., 2006. "Forecasting with Many Predictors," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 10, pages 515-554, Elsevier.
    15. Faust, Jon & Wright, Jonathan H., 2009. "Comparing Greenbook and Reduced Form Forecasts Using a Large Realtime Dataset," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(4), pages 468-479.
    16. Andrea Carriero & George Kapetanios & Massimiliano Marcellino, 2011. "Forecasting large datasets with Bayesian reduced rank multivariate models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(5), pages 735-761, August.
    17. Marco Aiolfi & Carlo Ambrogio Favero, "undated". "Model Uncertainty, Thick Modelling and the predictability of Stock Returns," Working Papers 221, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    18. Clements, Michael P. & Hendry, David F. (ed.), 2011. "The Oxford Handbook of Economic Forecasting," OUP Catalogue, Oxford University Press, number 9780195398649.
    19. Terasvirta, Timo & van Dijk, Dick & Medeiros, Marcelo C., 2005. "Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A re-examination," International Journal of Forecasting, Elsevier, vol. 21(4), pages 755-774.
    20. James H. Stock & Mark W. Watson, 2012. "Generalized Shrinkage Methods for Forecasting Using Many Predictors," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(4), pages 481-493, June.
    21. Marta Banbura & Domenico Giannone & Lucrezia Reichlin, 2010. "Large Bayesian vector auto regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 71-92.
    22. Sydney C. Ludvigson & Serena Ng, 2009. "Macro Factors in Bond Risk Premia," Review of Financial Studies, Society for Financial Studies, vol. 22(12), pages 5027-5067, December.
    23. Kim, Hyun Hak & Swanson, Norman R., 2014. "Forecasting financial and macroeconomic variables using data reduction methods: New empirical evidence," Journal of Econometrics, Elsevier, vol. 178(P2), pages 352-367.
    24. Engle, Robert F. & White (the late), Halbert (ed.), 1999. "Cointegration, Causality, and Forecasting: Festschrift in Honour of Clive W. J. Granger," OUP Catalogue, Oxford University Press, number 9780198296836.
    25. James H. Stock & Mark W. Watson, 2007. "Why Has U.S. Inflation Become Harder to Forecast?," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 39(s1), pages 3-33, February.
    26. De Mol, Christine & Giannone, Domenico & Reichlin, Lucrezia, 2008. "Forecasting using a large number of predictors: Is Bayesian shrinkage a valid alternative to principal components?," Journal of Econometrics, Elsevier, vol. 146(2), pages 318-328, October.
    27. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    28. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    29. Terasvirta, Timo, 2006. "Forecasting economic variables with nonlinear models," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 8, pages 413-457, Elsevier.
    30. Peter Exterkate, 2012. "Model Selection in Kernel Ridge Regression," CREATES Research Papers 2012-10, Department of Economics and Business Economics, Aarhus University.
    31. Hansen, Bruce E. & Racine, Jeffrey S., 2012. "Jackknife model averaging," Journal of Econometrics, Elsevier, vol. 167(1), pages 38-46.
    32. Timo Teräsvirta & Marcelo C. Medeiros & Gianluigi Rech, 2006. "Building neural network models for time series: a statistical approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(1), pages 49-75.
    33. Racine, Jeff, 2000. "Consistent cross-validatory model-selection for dependent data: hv-block cross-validation," Journal of Econometrics, Elsevier, vol. 99(1), pages 39-61, November.
    34. David E. Rapach & Jack K. Strauss & Guofu Zhou, 2010. "Out-of-Sample Equity Premium Prediction: Combination Forecasts and Links to the Real Economy," Review of Financial Studies, Society for Financial Studies, vol. 23(2), pages 821-862, February.
    35. Cem Cakmakli & Dick van Dijk, 2010. "Getting the Most out of Macroeconomic Information for Predicting Stock Returns and Volatility," Tinbergen Institute Discussion Papers 10-115/4, Tinbergen Institute.
    36. Swanson, Norman R & White, Halbert, 1995. "A Model-Selection Approach to Assessing the Information in the Term Structure Using Linear Models and Artificial Neural Networks," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 265-275, July.
    37. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    38. White, Halbert, 2006. "Approximate Nonlinear Forecasting Methods," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 9, pages 459-512, Elsevier.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Thierry Moudiki & Frédéric Planchet & Areski Cousin, 2018. "Multiple Time Series Forecasting Using Quasi-Randomized Functional Link Neural Networks," Risks, MDPI, vol. 6(1), pages 1-20, March.
    2. Sergei V. Akopov, 2018. "“Duty” and “Blame” in Russian Official Symbolic Representations of Sovereignty (1994-2018)," HSE Working papers WP BRP 61/PS/2018, National Research University Higher School of Economics.
    3. Alessandro Giovannelli, 2012. "Nonlinear Forecasting Using Large Datasets: Evidences on US and Euro Area Economies," CEIS Research Paper 255, Tor Vergata University, CEIS, revised 08 Nov 2012.
    4. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    5. A. Frenkel’ A. & N. Volkova N. & A. Surkov A. & E. Romanyuk I. & А. Френкель А. & Н. Волкова Н. & А. Сурков А. & Э. Романюк И., 2018. "Использование Методов Гребневой Регрессии При Объединении Прогнозов // The Application Of Ridge Regression Methods When Combining Forecasts," Финансы: теория и практика/Finance: Theory and Practice // Finance: Theory and Practice, ФГОБУВО Финансовый университет при Правительстве Российской Федерации // Financial University under The Government of Russian Federation, vol. 22(4), pages 6-17.
    6. Oslandsbotn, Andreas & Kereta, Željko & Naumova, Valeriya & Freund, Yoav & Cloninger, Alexander, 2022. "StreaMRAK a streaming multi-resolution adaptive kernel algorithm," Applied Mathematics and Computation, Elsevier, vol. 426(C).
    7. Xu Gong & Boqiang Lin, 2018. "Structural breaks and volatility forecasting in the copper futures market," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 38(3), pages 290-339, March.
    8. Wei, Yu & Liang, Chao & Li, Yan & Zhang, Xunhui & Wei, Guiwu, 2020. "Can CBOE gold and silver implied volatility help to forecast gold futures volatility in China? Evidence based on HAR and Ridge regression models," Finance Research Letters, Elsevier, vol. 35(C).
    9. Peter Exterkate, 2012. "Model Selection in Kernel Ridge Regression," CREATES Research Papers 2012-10, Department of Economics and Business Economics, Aarhus University.
    10. Hauzenberger, Niko & Huber, Florian & Klieber, Karin, 2023. "Real-time inflation forecasting using non-linear dimension reduction techniques," International Journal of Forecasting, Elsevier, vol. 39(2), pages 901-921.
    11. Tian Han & Ying Wang & Xiao Wang & Kang Chen & Huaiwu Peng & Zhenxin Gao & Lanxin Cui & Wentong Sun & Qinke Peng, 2023. "Mixed Multi-Pattern Regression for DNI Prediction in Arid Desert Areas," Sustainability, MDPI, vol. 15(17), pages 1-16, August.
    12. Markus Vogl, 2022. "Quantitative modelling frontiers: a literature review on the evolution in financial and risk modelling after the financial crisis (2008–2019)," SN Business & Economics, Springer, vol. 2(12), pages 1-69, December.
    13. Daiki Maki & Yasushi Ota, 2019. "Robust tests for ARCH in the presence of the misspecified conditional mean: A comparison of nonparametric approches," Papers 1907.12752, arXiv.org, revised Sep 2019.
    14. Kutateladze, Varlam, 2022. "The kernel trick for nonlinear factor modeling," International Journal of Forecasting, Elsevier, vol. 38(1), pages 165-177.
    15. Varlam Kutateladze, 2021. "The Kernel Trick for Nonlinear Factor Modeling," Papers 2103.01266, arXiv.org.
    16. Milan Fičura, 2019. "Forecasting Foreign Exchange Rate Movements with k-Nearest-Neighbour, Ridge Regression and Feed-Forward Neural Networks," FFA Working Papers 1.001, Prague University of Economics and Business, revised 24 Nov 2019.
    17. Cheng, Kai & Lu, Zhenzhou, 2018. "Sparse polynomial chaos expansion based on D-MORPH regression," Applied Mathematics and Computation, Elsevier, vol. 323(C), pages 17-30.
    18. Saeed Salah & Husain R. Alsamamra & Jawad H. Shoqeir, 2022. "Exploring Wind Speed for Energy Considerations in Eastern Jerusalem-Palestine Using Machine-Learning Algorithms," Energies, MDPI, vol. 15(7), pages 1-16, April.
    19. Christophe Croux & Peter Exterkate, 2011. "Sparse and Robust Factor Modelling," Tinbergen Institute Discussion Papers 11-122/4, Tinbergen Institute.
    20. Yoshiki Nakajima & Naoya Sueishi, 2022. "Forecasting the Japanese macroeconomy using high-dimensional data," The Japanese Economic Review, Springer, vol. 73(2), pages 299-324, April.
    21. Peter Exterkate, 2011. "Modelling Issues in Kernel Ridge Regression," Tinbergen Institute Discussion Papers 11-138/4, Tinbergen Institute.
    22. Wojciech Victor Fulmyk, 2023. "Nonlinear Granger Causality using Kernel Ridge Regression," Papers 2309.05107, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Barbara Rossi, 2019. "Forecasting in the presence of instabilities: How do we know whether models predict well and how to improve them," Economics Working Papers 1711, Department of Economics and Business, Universitat Pompeu Fabra, revised Jul 2021.
    2. Marine Carrasco & Barbara Rossi, 2016. "In-Sample Inference and Forecasting in Misspecified Factor Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 313-338, July.
    3. Luciani, Matteo, 2014. "Forecasting with approximate dynamic factor models: The role of non-pervasive shocks," International Journal of Forecasting, Elsevier, vol. 30(1), pages 20-29.
    4. Rachidi Kotchoni & Maxime Leroux & Dalibor Stevanovic, 2019. "Macroeconomic forecast accuracy in a data‐rich environment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(7), pages 1050-1072, November.
    5. Smeekes, Stephan & Wijler, Etienne, 2018. "Macroeconomic forecasting using penalized regression methods," International Journal of Forecasting, Elsevier, vol. 34(3), pages 408-430.
    6. Karlsson, Sune, 2013. "Forecasting with Bayesian Vector Autoregression," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 791-897, Elsevier.
    7. Dias, Gustavo Fruet & Kapetanios, George, 2018. "Estimation and forecasting in vector autoregressive moving average models for rich datasets," Journal of Econometrics, Elsevier, vol. 202(1), pages 75-91.
    8. Alessandro Girardi & Roberto Golinelli & Carmine Pappalardo, 2017. "The role of indicator selection in nowcasting euro-area GDP in pseudo-real time," Empirical Economics, Springer, vol. 53(1), pages 79-99, August.
    9. Rusnák, Marek, 2016. "Nowcasting Czech GDP in real time," Economic Modelling, Elsevier, vol. 54(C), pages 26-39.
    10. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    11. Cheng, Xu & Hansen, Bruce E., 2015. "Forecasting with factor-augmented regression: A frequentist model averaging approach," Journal of Econometrics, Elsevier, vol. 186(2), pages 280-293.
    12. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2015. "Realtime nowcasting with a Bayesian mixed frequency model with stochastic volatility," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 837-862, October.
    13. Andrea Carriero & Francesco Corsello & Massimiliano Marcellino, 2022. "The global component of inflation volatility," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(4), pages 700-721, June.
    14. Berg, Tim O. & Henzel, Steffen R., 2015. "Point and density forecasts for the euro area using Bayesian VARs," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1067-1095.
    15. Alexander Chudik & M. Hashem Pesaran, 2016. "Theory And Practice Of Gvar Modelling," Journal of Economic Surveys, Wiley Blackwell, vol. 30(1), pages 165-197, February.
    16. Cubadda, Gianluca & Guardabascio, Barbara, 2012. "A medium-N approach to macroeconomic forecasting," Economic Modelling, Elsevier, vol. 29(4), pages 1099-1105.
    17. Kelly, Bryan & Pruitt, Seth, 2015. "The three-pass regression filter: A new approach to forecasting using many predictors," Journal of Econometrics, Elsevier, vol. 186(2), pages 294-316.
    18. Peter Exterkate, 2012. "Model Selection in Kernel Ridge Regression," CREATES Research Papers 2012-10, Department of Economics and Business Economics, Aarhus University.
    19. Chris Bloor & Troy Matheson, 2010. "Analysing shock transmission in a data-rich environment: a large BVAR for New Zealand," Empirical Economics, Springer, vol. 39(2), pages 537-558, October.
    20. Petrevski, Goran & Exterkate, Peter & Tevdovski, Dragan & Bogoev, Jane, 2015. "The transmission of foreign shocks to South Eastern European economies: A Bayesian VAR approach," Economic Systems, Elsevier, vol. 39(4), pages 632-643.

    More about this item

    Keywords

    High dimensionality; Nonlinear forecasting; Ridge regression; Kernel methods;
    All these keywords.

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • E27 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Forecasting and Simulation: Models and Applications

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:32:y:2016:i:3:p:736-753. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.