IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v101y2010i4p839-849.html
   My bibliography  Save this article

Model selection by sequentially normalized least squares

Author

Listed:
  • Rissanen, Jorma
  • Roos, Teemu
  • Myllymäki, Petri

Abstract

Model selection by means of the predictive least squares (PLS) principle has been thoroughly studied in the context of regression model selection and autoregressive (AR) model order estimation. We introduce a new criterion based on sequentially minimized squared deviations, which are smaller than both the usual least squares and the squared prediction errors used in PLS. We also prove that our criterion has a probabilistic interpretation as a model which is asymptotically optimal within the given class of distributions by reaching the lower bound on the logarithmic prediction errors, given by the so called stochastic complexity, and approximated by BIC. This holds when the regressor (design) matrix is non-random or determined by the observed data as in AR models. The advantages of the criterion include the fact that it can be evaluated efficiently and exactly, without asymptotic approximations, and importantly, there are no adjustable hyper-parameters, which makes it applicable to both small and large amounts of data.

Suggested Citation

  • Rissanen, Jorma & Roos, Teemu & Myllymäki, Petri, 2010. "Model selection by sequentially normalized least squares," Journal of Multivariate Analysis, Elsevier, vol. 101(4), pages 839-849, April.
    • Handle: RePEc:eee:jmvana:v:101:y:2010:i:4:p:839-849
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047-259X(09)00240-1
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hansen M. H & Yu B., 2001. "Model Selection and the Principle of Minimum Description Length," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 746-774, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jussi Määttä & Daniel F. Schmidt & Teemu Roos, 2016. "Subset Selection in Linear Regression using Sequentially Normalized Least Squares: Asymptotic Theory," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(2), pages 382-395, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Seongkyoon Jeong & Jae Young Choi, 2012. "The taxonomy of research collaboration in science and technology: evidence from mechanical research through probabilistic clustering analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 719-735, June.
    2. Poskitt, D.S. & Sengarapillai, Arivalzahan, 2013. "Description length and dimensionality reduction in functional data analysis," Computational Statistics & Data Analysis, Elsevier, vol. 58(C), pages 98-113.
    3. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    4. Njindan Iyke, Bernard, 2015. "Macro Determinants of the Real Exchange Rate in a Small Open Small Island Economy: Evidence from Mauritius via BMA," MPRA Paper 68968, University Library of Munich, Germany.
    5. Paul L. Bowen & Robert A. O'Farrell & Fiona H. Rohde, 2009. "An Empirical Investigation of End-User Query Development: The Effects of Improved Model Expressiveness vs. Complexity," Information Systems Research, INFORMS, vol. 20(4), pages 565-584, December.
    6. Brockwell, P. J. & Dahlhaus, R., 2004. "Generalized Levinson-Durbin and Burg algorithms," Journal of Econometrics, Elsevier, vol. 118(1-2), pages 129-149.
    7. Magkonis, Georgios & Zekente, Kalliopi-Maria, 2020. "Inflation-output trade-off: Old measures, new determinants?," Journal of Macroeconomics, Elsevier, vol. 65(C).
    8. Branimir Jovanovic, 2017. "Growth forecast errors and government investment and consumption multipliers," International Review of Applied Economics, Taylor & Francis Journals, vol. 31(1), pages 83-107, January.
    9. Karol Szafranek & Marek Kwas & Grzegorz Szafrański & Zuzanna Wośko, 2020. "Common Determinants of Credit Default Swap Premia in the North American Oil and Gas Industry. A Panel BMA Approach," Energies, MDPI, vol. 13(23), pages 1-23, November.
    10. Klaus Wohlrabe & Teresa Buchen, 2014. "Assessing the Macroeconomic Forecasting Performance of Boosting: Evidence for the United States, the Euro Area and Germany," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(4), pages 231-242, July.
    11. Joris Mulder & James O. Berger & Víctor Peña & M. J. Bayarri, 2021. "On the prevalence of information inconsistency in normal linear models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(1), pages 103-132, March.
    12. Sylvain Barde, 2017. "A Practical, Accurate, Information Criterion for Nth Order Markov Processes," Computational Economics, Springer;Society for Computational Economics, vol. 50(2), pages 281-324, August.
    13. repec:hal:spmain:info:hdl:2441/5fafm6me7k8omq5jbo61urqq27 is not listed on IDEAS
    14. Marcos Prates & Renato Assunção & Marcelo Costa, 2012. "Flexible scan statistic test to detect disease clusters in hierarchical trees," Computational Statistics, Springer, vol. 27(4), pages 715-737, December.
    15. Jan G. De Gooijer & Ao Yuan, 2008. "MDL Mean Function Selection in Semiparametric Kernel Regression Models," Tinbergen Institute Discussion Papers 08-046/4, Tinbergen Institute.
    16. Christian Pierdzioch & Rangan Gupta & Hossein Hassani & Emmanuel Silva, 2018. "Forecasting Changes of Economic Inequality: A Boosting Approach," Working Papers 201868, University of Pretoria, Department of Economics.
    17. Yue, Mu & Li, Jialiang & Cheng, Ming-Yen, 2019. "Two-step sparse boosting for high-dimensional longitudinal data with varying coefficients," Computational Statistics & Data Analysis, Elsevier, vol. 131(C), pages 222-234.
    18. Massimo Marinacci, 2015. "Model Uncertainty," Journal of the European Economic Association, European Economic Association, vol. 13(6), pages 1022-1100, December.
    19. Ivan Chang, Yuan-Chin & Huang, Yufen & Huang, Yu-Pai, 2010. "Early stopping in L2Boosting," Computational Statistics & Data Analysis, Elsevier, vol. 54(10), pages 2203-2213, October.
    20. Branimir Jovanovic, 2012. "How Policy Actions Affect Short-term Post-crisis Recovery?," CEIS Research Paper 253, Tor Vergata University, CEIS, revised 05 Oct 2012.
    21. Paul Hofmarcher & Jesús Crespo Cuaresma & Bettina Grün & Kurt Hornik, 2015. "Last Night a Shrinkage Saved My Life: Economic Growth, Model Uncertainty and Correlated Regressors," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 34(2), pages 133-144, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:101:y:2010:i:4:p:839-849. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.