IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v206y2025ics0167947324002081.html
   My bibliography  Save this article

Component selection and variable selection for mixture regression models

Author

Listed:
  • Qi, Xuefei
  • Xu, Xingbai
  • Feng, Zhenghui
  • Peng, Heng

Abstract

Finite mixture regression models are commonly used to account for heterogeneity in populations and situations where the assumptions required for standard regression models may not hold. To expand the range of applicable distributions for components beyond the Gaussian distribution, other distributions, such as the exponential power distribution, the skew-normal distribution, and so on, are explored. To enable simultaneous model estimation, order selection, and variable selection, a penalized likelihood estimation approach that imposes penalties on both the mixing proportions and regression coefficients, which we call the double-penalized likelihood method is proposed in this paper. Four double-penalized likelihood functions and their performance are studied. The consistency of estimators, order selection, and variable selection are investigated. A modified expectation–maximization algorithm is proposed to implement the double-penalized likelihood method. Numerical simulations demonstrate the effectiveness of our proposed method and algorithm. Finally, the results of real data analysis are presented to illustrate the application of our approach. Overall, our study contributes to the development of mixture regression models and provides a useful tool for model and variable selection.

Suggested Citation

  • Qi, Xuefei & Xu, Xingbai & Feng, Zhenghui & Peng, Heng, 2025. "Component selection and variable selection for mixture regression models," Computational Statistics & Data Analysis, Elsevier, vol. 206(C).
    • Handle: RePEc:eee:csdana:v:206:y:2025:i:c:s0167947324002081
    DOI: 10.1016/j.csda.2024.108124
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947324002081
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2024.108124?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Hajo Holzmann & Axel Munk & Tilmann Gneiting, 2006. "Identifiability of Finite Mixtures of Elliptical Distributions," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 33(4), pages 753-763, December.
    2. Shiyi Tang & Jiali Zheng, 2022. "Penalized estimation in finite mixture of ultra-high dimensional regression models," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 51(17), pages 5971-5992, September.
    3. repec:bla:biomet:v:71:y:2015:i:4:p:1081-1089 is not listed on IDEAS
    4. Junhui Yin & Liucang Wu & Lin Dai, 2020. "Variable selection in finite mixture of regression models using the skew-normal distribution," Journal of Applied Statistics, Taylor & Francis Journals, vol. 47(16), pages 2941-2960, December.
    5. S. Faria & F. Gon�alves, 2013. "Financial data modeling by Poisson mixture regression," Journal of Applied Statistics, Taylor & Francis Journals, vol. 40(10), pages 2150-2162, October.
    6. Song, Weixing & Yao, Weixin & Xing, Yanru, 2014. "Robust mixture regression model fitting by Laplace distribution," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 128-137.
    7. Mian Huang & Weixin Yao, 2012. "Mixture of Regression Models With Varying Mixing Proportions: A Semiparametric Approach," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 711-724, June.
    8. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    9. Gupta, Mayetri & Ibrahim, Joseph G., 2007. "Variable Selection in Regression Mixture Modeling for the Discovery of Gene Regulatory Networks," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 867-880, September.
    10. Chen, Jiahua & Khalili, Abbas, 2008. "Order Selection in Finite Mixture Models With a Nonsmooth Penalty," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1674-1683.
    11. Goldfeld, Stephen M. & Quandt, Richard E., 1973. "A Markov model for switching regressions," Journal of Econometrics, Elsevier, vol. 1(1), pages 3-15, March.
    12. Mian Huang & Runze Li & Shaoli Wang, 2013. "Nonparametric Mixture of Regression Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(503), pages 929-941, September.
    13. Biernacki, Christophe & Celeux, Gilles & Govaert, Gerard, 2003. "Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate Gaussian mixture models," Computational Statistics & Data Analysis, Elsevier, vol. 41(3-4), pages 561-575, January.
    14. Martin G. Larson & Gregg E. Dinse, 1985. "A Mixture Model for the Regression Analysis of Competing Risks Data," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 34(3), pages 201-211, November.
    15. Chen, Xiao & Feng, Zhenghui & Peng, Heng, 2023. "Estimation and order selection for multivariate exponential power mixture models," Journal of Multivariate Analysis, Elsevier, vol. 195(C).
    16. Yunlu Jiang & Yu Conglian & Ji Qinghua, 2018. "Model selection for the localized mixture of experts models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 45(11), pages 1994-2006, August.
    17. Conglian Yu & Xiyang Wang, 2020. "A new model selection procedure for finite mixture regression models," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 49(18), pages 4347-4366, September.
    18. N. Atienza & J. Garcia-Heras & J. Muñoz-Pichardo, 2006. "A new condition for identifiability of finite mixture distributions," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 63(2), pages 215-221, April.
    19. Xinyi Wang & Zhenghui Feng, 2023. "Component selection for exponential power mixture models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 50(2), pages 291-314, January.
    20. Otiniano, C.E.G. & Rathie, P.N. & Ozelim, L.C.S.M., 2015. "On the identifiability of finite mixture of Skew-Normal and Skew-t distributions," Statistics & Probability Letters, Elsevier, vol. 106(C), pages 103-108.
    21. Khalili, Abbas & Chen, Jiahua, 2007. "Variable Selection in Finite Mixture of Regression Models," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1025-1038, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hou, Yanxi & Li, Jiahong & Gao, Guangyuan, 2025. "Insurance loss modeling with gradient tree-boosted mixture models," Insurance: Mathematics and Economics, Elsevier, vol. 121(C), pages 45-62.
    2. Pei, Youquan & Peng, Heng & Xu, Jinfeng, 2024. "A latent class Cox model for heterogeneous time-to-event data," Journal of Econometrics, Elsevier, vol. 239(2).
    3. Chun Yu & Weixin Yao & Guangren Yang, 2020. "A Selective Overview and Comparison of Robust Mixture Regression Estimators," International Statistical Review, International Statistical Institute, vol. 88(1), pages 176-202, April.
    4. Ye, Mao & Lu, Zhao-Hua & Li, Yimei & Song, Xinyuan, 2019. "Finite mixture of varying coefficient model: Estimation and component selection," Journal of Multivariate Analysis, Elsevier, vol. 171(C), pages 452-474.
    5. Sphiwe B. Skhosana & Salomon M. Millard & Frans H. J. Kanfer, 2023. "A Novel EM-Type Algorithm to Estimate Semi-Parametric Mixtures of Partially Linear Models," Mathematics, MDPI, vol. 11(5), pages 1-20, February.
    6. Abbas Khalili & Farhad Shokoohi & Masoud Asgharian & Shili Lin, 2023. "Sparse estimation in semiparametric finite mixture of varying coefficient regression models," Biometrics, The International Biometric Society, vol. 79(4), pages 3445-3457, December.
    7. Lee, Kuo-Jung & Chen, Ray-Bing & Wu, Ying Nian, 2016. "Bayesian variable selection for finite mixture model of linear regressions," Computational Statistics & Data Analysis, Elsevier, vol. 95(C), pages 1-16.
    8. Ping Zeng & Yongyue Wei & Yang Zhao & Jin Liu & Liya Liu & Ruyang Zhang & Jianwei Gou & Shuiping Huang & Feng Chen, 2014. "Variable selection approach for zero-inflated count data via adaptive lasso," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(4), pages 879-894, April.
    9. Yan Li & Chun Yu & Yize Zhao & Weixin Yao & Robert H. Aseltine & Kun Chen, 2022. "Pursuing sources of heterogeneity in modeling clustered population," Biometrics, The International Biometric Society, vol. 78(2), pages 716-729, June.
    10. Sijia Xiang & Weixin Yao, 2018. "Semiparametric mixtures of nonparametric regressions," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 70(1), pages 131-154, February.
    11. Gustavo Alexis Sabillón & Luiz Gabriel Fernandes Cotrim & Daiane Aparecida Zuanetti, 2023. "A data-driven reversible jump for estimating a finite mixture of regression models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 32(1), pages 350-369, March.
    12. You, Na & Dai, Hongsheng & Wang, Xueqin & Yu, Qingyun, 2024. "Sequential estimation for mixture of regression models for heterogeneous population," Computational Statistics & Data Analysis, Elsevier, vol. 194(C).
    13. Marco Berrettini & Giuliano Galimberti & Saverio Ranciati, 2023. "Semiparametric finite mixture of regression models with Bayesian P-splines," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(3), pages 745-775, September.
    14. Maruotti, Antonello & Punzo, Antonio, 2017. "Model-based time-varying clustering of multivariate longitudinal data with covariates and outliers," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 475-496.
    15. Lee, Kuo-Jung & Feldkircher, Martin & Chen, Yi-Chi, 2021. "Variable selection in finite mixture of regression models with an unknown number of components," Computational Statistics & Data Analysis, Elsevier, vol. 158(C).
    16. Saverio Ranciati & Giuliano Galimberti & Gabriele Soffritti, 2019. "Bayesian variable selection in linear regression models with non-normal errors," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 28(2), pages 323-358, June.
    17. Yao, Weixin & Wei, Yan & Yu, Chun, 2014. "Robust mixture regression using the t-distribution," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 116-127.
    18. Xue, Jiacheng & Yao, Weixin, 2022. "Machine Learning Embedded Semiparametric Mixtures of Regressions with Covariate-Varying Mixing Proportions," Econometrics and Statistics, Elsevier, vol. 22(C), pages 159-171.
    19. Meng Li & Sijia Xiang & Weixin Yao, 2016. "Robust estimation of the number of components for mixtures of linear regression models," Computational Statistics, Springer, vol. 31(4), pages 1539-1555, December.
    20. Custodio João, Igor & Lucas, André & Schaumburg, Julia & Schwaab, Bernd, 2023. "Dynamic clustering of multivariate panel data," Journal of Econometrics, Elsevier, vol. 237(2).

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:206:y:2025:i:c:s0167947324002081. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.