IDEAS home Printed from https://ideas.repec.org/p/tin/wpaper/20110138.html
   My bibliography  Save this paper

Modelling Issues in Kernel Ridge Regression

Author

Listed:
  • Peter Exterkate

    (Erasmus University Rotterdam)

Abstract

Kernel ridge regression is gaining popularity as a data-rich nonlinear forecasting tool, which is applicable in many different contexts. This paper investigates the influence of the choice of kernel and the setting of tuning parameters on forecast accuracy. We review several popular kernels, including polynomial kernels, the Gaussian kernel, and the Sinc kernel. We interpret the latter two kernels in terms of their smoothing properties, and we relate the tuning parameters associated to all these kernels to smoothness measures of the prediction function and to the signal-to-noise ratio. Based on these interpretations, we provide guidelines for selecting the tuning parameters from small grids using cross-validation. A Monte Carlo study confirms the practical usefulness of these rules of thumb. Finally, the flexible and smooth functional forms provided by the Gaussian and Sinc kernels makes them widely applicable, and we recommend their use instead of the pop ular polynomial kernels in general settings, in which no information on the data-generating process is available.

Suggested Citation

  • Peter Exterkate, 2011. "Modelling Issues in Kernel Ridge Regression," Tinbergen Institute Discussion Papers 11-138/4, Tinbergen Institute.
    • Handle: RePEc:tin:wpaper:20110138
    as

    Download full text from publisher

    File URL: https://papers.tinbergen.nl/11138.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Exterkate, Peter & Groenen, Patrick J.F. & Heij, Christiaan & van Dijk, Dick, 2016. "Nonlinear forecasting with many predictors using kernel ridge regression," International Journal of Forecasting, Elsevier, vol. 32(3), pages 736-753.
    2. Ludvigson, Sydney C. & Ng, Serena, 2007. "The empirical risk-return relation: A factor analysis approach," Journal of Financial Economics, Elsevier, vol. 83(1), pages 171-222, January.
    3. Terasvirta, Timo & van Dijk, Dick & Medeiros, Marcelo C., 2005. "Linear models, smooth transition autoregressions, and neural networks for forecasting macroeconomic time series: A re-examination," International Journal of Forecasting, Elsevier, vol. 21(4), pages 755-774.
    4. Sydney C. Ludvigson & Serena Ng, 2009. "Macro Factors in Bond Risk Premia," The Review of Financial Studies, Society for Financial Studies, vol. 22(12), pages 5027-5067, December.
    5. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    6. Timo Teräsvirta & Marcelo C. Medeiros & Gianluigi Rech, 2006. "Building neural network models for time series: a statistical approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(1), pages 49-75.
    7. White, Halbert, 2006. "Approximate Nonlinear Forecasting Methods," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 1, chapter 9, pages 459-512, Elsevier.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Kernel Ridge Regression – Example Computation I
      by Clive Jones in Business Forecasting on 2012-07-27 00:23:25
    2. Kernel Ridge Regression – A Toy Example
      by Clive Jones in Business Forecasting on 2014-03-02 03:10:25

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Exterkate, 2012. "Model Selection in Kernel Ridge Regression," CREATES Research Papers 2012-10, Department of Economics and Business Economics, Aarhus University.
    2. Exterkate, Peter & Groenen, Patrick J.F. & Heij, Christiaan & van Dijk, Dick, 2016. "Nonlinear forecasting with many predictors using kernel ridge regression," International Journal of Forecasting, Elsevier, vol. 32(3), pages 736-753.
    3. Christophe Croux & Peter Exterkate, 2011. "Sparse and Robust Factor Modelling," Tinbergen Institute Discussion Papers 11-122/4, Tinbergen Institute.
    4. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    5. Trucíos, Carlos & Mazzeu, João H.G. & Hotta, Luiz K. & Valls Pereira, Pedro L. & Hallin, Marc, 2021. "Robustness and the general dynamic factor model with infinite-dimensional space: Identification, estimation, and forecasting," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1520-1534.
    6. Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2019. "A diagnostic criterion for approximate factor structure," Journal of Econometrics, Elsevier, vol. 212(2), pages 503-521.
    7. repec:gnv:wpaper:unige:76321 is not listed on IDEAS
    8. Maio, Paulo & Philip, Dennis, 2015. "Macro variables and the components of stock returns," Journal of Empirical Finance, Elsevier, vol. 33(C), pages 287-308.
    9. Yoshiki Nakajima & Naoya Sueishi, 2022. "Forecasting the Japanese macroeconomy using high-dimensional data," The Japanese Economic Review, Springer, vol. 73(2), pages 299-324, April.
    10. repec:dau:papers:123456789/11663 is not listed on IDEAS
    11. Carlos Cesar Trucios-Maza & João H. G Mazzeu & Luis K. Hotta & Pedro L. Valls Pereira & Marc Hallin, 2019. "On the robustness of the general dynamic factor model with infinite-dimensional space: identification, estimation, and forecasting," Working Papers ECARES 2019-32, ULB -- Universite Libre de Bruxelles.
    12. Hammerschmid, Regina & Lohre, Harald, 2018. "Regime shifts and stock return predictability," International Review of Economics & Finance, Elsevier, vol. 56(C), pages 138-160.
    13. Patrick Gagliardini & Elisa Ossola & Olivier Scaillet, 2016. "Time‐Varying Risk Premium in Large Cross‐Sectional Equity Data Sets," Econometrica, Econometric Society, vol. 84, pages 985-1046, May.
    14. repec:ipg:wpaper:19 is not listed on IDEAS
    15. Terence Tai-Leung Chong & Nasha Li & Lin Zou, 2017. "A New Approach to Modeling Sector Stock Returns in China," Chinese Economy, Taylor & Francis Journals, vol. 50(5), pages 305-322, September.
    16. Forni, Mario & Cavicchioli, Maddalena & Lippi, Marco & Zaffaroni, Paolo, 2016. "Eigenvalue Ratio Estimators for the Number of Common Factors," CEPR Discussion Papers 11440, C.E.P.R. Discussion Papers.
    17. Roberta Fiori & Simonetta Iannotti, 2010. "On the interaction between market and credit risk: a factor-augmented vector autoregressive (FAVAR) approach," Temi di discussione (Economic working papers) 779, Bank of Italy, Economic Research and International Relations Area.
    18. Fan, Jianqing & Xue, Lingzhou & Yao, Jiawei, 2017. "Sufficient forecasting using factor models," Journal of Econometrics, Elsevier, vol. 201(2), pages 292-306.
    19. Yannick Le Pen & Benoît Sévi, 2013. "Futures Trading and the Excess Comovement of Commodity Prices," Working Papers halshs-00793724, HAL.
    20. repec:ipg:wpaper:2013-019 is not listed on IDEAS
    21. Buncic, Daniel & Tischhauser, Martin, 2017. "Macroeconomic factors and equity premium predictability," International Review of Economics & Finance, Elsevier, vol. 51(C), pages 621-644.
    22. Chang, Ya-Ting & Gau, Yin-Feng & Hsu, Chih-Chiang, 2017. "Liquidity Commonality in Foreign Exchange Markets During the Global Financial Crisis and the Sovereign Debt Crisis: Effects of Macroeconomic and Quantitative Easing Announcements," The North American Journal of Economics and Finance, Elsevier, vol. 42(C), pages 172-192.
    23. Florian Gunsilius & Susanne M. Schennach, 2019. "Independent nonlinear component analysis," CeMMAP working papers CWP46/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    24. Simon Freyaldenhoven, 2020. "Identification Through Sparsity in Factor Models," Working Papers 20-25, Federal Reserve Bank of Philadelphia.

    More about this item

    Keywords

    nonlinear forecasting; shrinkage estimation; kernel methods; high dimensionality;
    All these keywords.

    JEL classification:

    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tin:wpaper:20110138. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tinbergen Office +31 (0)10-4088900 (email available below). General contact details of provider: https://edirc.repec.org/data/tinbenl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.