IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i12p3194-3211.html
   My bibliography  Save this article

Model selection strategies for identifying most relevant covariates in homoscedastic linear models

Author

Listed:
  • Min, Aleksey
  • Holzmann, Hajo
  • Czado, Claudia

Abstract

A new method in two variations for the identification of most relevant covariates in linear models with homoscedastic errors is proposed. In contrast to many known selection criteria, the method is based on an interpretable scaled quantity. This quantity measures a maximal relative error one makes by selecting covariates from a given set of all available covariates. The proposed model selection procedures rely on asymptotic normality of test statistics, and therefore normality of the errors in the regression model is not required. In a simulation study the performance of the suggested methods along with the performance of the standard model selection criteria AIC, BIC, Lasso and relaxed Lasso is examined. The simulation study illustrates the favorable performance of the proposed method as compared to the above reference criteria, especially when regression effects possess influence of several orders in magnitude. The accuracy of the normal approximation to the test statistics is also investigated; it has been already satisfactory for sample sizes 50 and 100. As an illustration the US college spending data from 1994 is analyzed.

Suggested Citation

  • Min, Aleksey & Holzmann, Hajo & Czado, Claudia, 2010. "Model selection strategies for identifying most relevant covariates in homoscedastic linear models," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3194-3211, December.
    • Handle: RePEc:eee:csdana:v:54:y:2010:i:12:p:3194-3211
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(09)00322-3
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Meinshausen, Nicolai, 2007. "Relaxed Lasso," Computational Statistics & Data Analysis, Elsevier, vol. 52(1), pages 374-393, September.
    2. Yancey, T A & Judge, G G & Bock, M E, 1973. "Wallace's Weak Mean Square Error Criterion for Testing Linear Restrictions in Regression: A Tighter Bound," Econometrica, Econometric Society, vol. 41(6), pages 1203-1206, November.
    3. David J. Spiegelhalter & Nicola G. Best & Bradley P. Carlin & Angelika Van Der Linde, 2002. "Bayesian measures of model complexity and fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 583-639, October.
    4. Wang, Hansheng & Leng, Chenlei, 2008. "A note on adaptive group lasso," Computational Statistics & Data Analysis, Elsevier, vol. 52(12), pages 5277-5286, August.
    5. Holger Dette & Axel Munk, 2003. "Some Methodological Aspects of Validation of Models in Nonparametric Regression," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 57(2), pages 207-244, May.
    6. Wallace, T D, 1972. "Weaker Criteria and Tests for Linear Restrictions in Regression," Econometrica, Econometric Society, vol. 40(4), pages 689-698, July.
    7. Leeb, Hannes & Pötscher, Benedikt M., 2003. "The Finite-Sample Distribution Of Post-Model-Selection Estimators And Uniform Versus Nonuniform Approximations," Econometric Theory, Cambridge University Press, vol. 19(1), pages 100-142, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ruth M. Pfeiffer & Andrew Redd & Raymond J. Carroll, 2017. "On the impact of model selection on predictor identification and parameter inference," Computational Statistics, Springer, vol. 32(2), pages 667-690, June.
    2. Tutz, Gerhard & Pößnecker, Wolfgang & Uhlmann, Lorenz, 2015. "Variable selection in general multinomial logit models," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 207-222.
    3. Buddhavarapu, Prasad & Bansal, Prateek & Prozzi, Jorge A., 2021. "A new spatial count data model with time-varying parameters," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 566-586.
    4. Mumtaz, Haroon & Theodoridis, Konstantinos, 2017. "Common and country specific economic uncertainty," Journal of International Economics, Elsevier, vol. 105(C), pages 205-216.
    5. Jesse Elliott & Zemin Bai & Shu-Ching Hsieh & Shannon E Kelly & Li Chen & Becky Skidmore & Said Yousef & Carine Zheng & David J Stewart & George A Wells, 2020. "ALK inhibitors for non-small cell lung cancer: A systematic review and network meta-analysis," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-18, February.
    6. Christina Leuker & Thorsten Pachur & Ralph Hertwig & Timothy J. Pleskac, 2019. "Do people exploit risk–reward structures to simplify information processing in risky choice?," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 76-94, August.
    7. Francois Olivier & Laval Guillaume, 2011. "Deviance Information Criteria for Model Selection in Approximate Bayesian Computation," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-25, July.
    8. El Ghouch, Anouar & Genton, Marc G. & Bouezmarni , Taoufik, 2012. "Measuring the Discrepancy of a Parametric Model via Local Polynomial Smoothing," LIDAM Discussion Papers ISBA 2012001, Université catholique de Louvain, Institute of Statistics, Biostatistics and Actuarial Sciences (ISBA).
    9. Raggi, Davide & Bordignon, Silvano, 2012. "Long memory and nonlinearities in realized volatility: A Markov switching approach," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3730-3742.
    10. Angelica Gianfreda & Francesco Ravazzolo & Luca Rossini, 2023. "Large Time‐Varying Volatility Models for Hourly Electricity Prices," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 85(3), pages 545-573, June.
    11. Rubio, F.J. & Steel, M.F.J., 2011. "Inference for grouped data with a truncated skew-Laplace distribution," Computational Statistics & Data Analysis, Elsevier, vol. 55(12), pages 3218-3231, December.
    12. Mkhadri, Abdallah & Ouhourane, Mohamed, 2013. "An extended variable inclusion and shrinkage algorithm for correlated variables," Computational Statistics & Data Analysis, Elsevier, vol. 57(1), pages 631-644.
    13. Alessandri, Piergiorgio & Mumtaz, Haroon, 2019. "Financial regimes and uncertainty shocks," Journal of Monetary Economics, Elsevier, vol. 101(C), pages 31-46.
    14. Padilla, Juan L. & Azevedo, Caio L.N. & Lachos, Victor H., 2018. "Multidimensional multiple group IRT models with skew normal latent trait distributions," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 250-268.
    15. Svetlana V. Tishkovskaya & Paul G. Blackwell, 2021. "Bayesian estimation of heterogeneous environments from animal movement data," Environmetrics, John Wiley & Sons, Ltd., vol. 32(6), September.
    16. David Macro & Jeroen Weesie, 2016. "Inequalities between Others Do Matter: Evidence from Multiplayer Dictator Games," Games, MDPI, vol. 7(2), pages 1-23, April.
    17. Tautenhahn, Susanne & Heilmeier, Hermann & Jung, Martin & Kahl, Anja & Kattge, Jens & Moffat, Antje & Wirth, Christian, 2012. "Beyond distance-invariant survival in inverse recruitment modeling: A case study in Siberian Pinus sylvestris forests," Ecological Modelling, Elsevier, vol. 233(C), pages 90-103.
    18. Julian P. T. Higgins & Simon G. Thompson & David J. Spiegelhalter, 2009. "A re‐evaluation of random‐effects meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 137-159, January.
    19. Simon Mak & Derek Bingham & Yi Lu, 2016. "A regional compound Poisson process for hurricane and tropical storm damage," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 65(5), pages 677-703, November.
    20. Xi, Yanhui & Peng, Hui & Qin, Yemei & Xie, Wenbiao & Chen, Xiaohong, 2015. "Bayesian analysis of heavy-tailed market microstructure model and its application in stock markets," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 117(C), pages 141-153.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:12:p:3194-3211. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.