IDEAS home Printed from https://ideas.repec.org/a/spr/stmapp/v30y2021i1d10.1007_s10260-020-00523-9.html
   My bibliography  Save this article

Covariance matrix estimation of the maximum likelihood estimator in multivariate clusterwise linear regression

Author

Listed:
  • Giuliano Galimberti

    (University of Bologna)

  • Lorenzo Nuzzi
  • Gabriele Soffritti

    (University of Bologna)

Abstract

The expectation-maximisation algorithm is employed to perform maximum likelihood estimation in a wide range of situations, including regression analysis based on clusterwise regression models. A disadvantage of using this algorithm is that it is unable to provide an assessment of the sample variability of the maximum likelihood estimator. This inability is a consequence of the fact that the algorithm does not require deriving an analytical expression for the Hessian matrix, thus preventing from a direct evaluation of the asymptotic covariance matrix of the estimator. A solution to this problem when performing linear regression analysis through a multivariate Gaussian clusterwise regression model is developed. Two estimators of the asymptotic covariance matrix of the maximum likelihood estimator are proposed. In practical applications their use makes it possible to avoid resorting to bootstrap techniques and general purpose mathematical optimisers. The performances of these estimators are evaluated in analysing small simulated and real datasets; the obtained results illustrate their usefulness and effectiveness in practical applications. From a theoretical point of view, under suitable conditions, the proposed estimators are shown to be consistent.

Suggested Citation

  • Giuliano Galimberti & Lorenzo Nuzzi & Gabriele Soffritti, 2021. "Covariance matrix estimation of the maximum likelihood estimator in multivariate clusterwise linear regression," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 30(1), pages 235-268, March.
    • Handle: RePEc:spr:stmapp:v:30:y:2021:i:1:d:10.1007_s10260-020-00523-9
    DOI: 10.1007/s10260-020-00523-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10260-020-00523-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10260-020-00523-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bowden, Roger J, 1973. "The Theory of Parametric Identification," Econometrica, Econometric Society, vol. 41(6), pages 1069-1074, November.
    2. Benaglia, Tatiana & Chauveau, Didier & Hunter, David R. & Young, Derek S., 2009. "mixtools: An R Package for Analyzing Mixture Models," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 32(i06).
    3. T. Rolf Turner, 2000. "Estimating the propagation rate of a viral infection of potato plants via mixtures of regressions," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 49(3), pages 371-384.
    4. Utkarsh J. Dang & Antonio Punzo & Paul D. McNicholas & Salvatore Ingrassia & Ryan P. Browne, 2017. "Multivariate Response and Parsimony for Gaussian Cluster-Weighted Models," Journal of Classification, Springer;The Classification Society, vol. 34(1), pages 4-34, April.
    5. Grün, Bettina & Leisch, Friedrich, 2008. "FlexMix Version 2: Finite Mixtures with Concomitant Variables and Varying and Constant Parameters," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 28(i04).
    6. Tang, Qingguo & Karunamuni, Rohana J., 2013. "Minimum distance estimation in a finite mixture regression model," Journal of Multivariate Analysis, Elsevier, vol. 120(C), pages 185-204.
    7. Boldea, Otilia & Magnus, Jan R., 2009. "Maximum Likelihood Estimation of the Multivariate Normal Mixture Model," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1539-1549.
    8. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
    9. Gerhard Arminger & Petra Stein & Jörg Wittenberg, 1999. "Mixtures of conditional mean- and covariance-structure models," Psychometrika, Springer;The Psychometric Society, vol. 64(4), pages 475-494, December.
    10. Michel Wedel, 2002. "Concomitant variables in finite mixture models," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 56(3), pages 362-375, August.
    11. Cathy Maugis & Gilles Celeux & Marie-Laure Martin-Magniette, 2009. "Variable Selection for Clustering with Gaussian Mixture Models," Biometrics, The International Biometric Society, vol. 65(3), pages 701-709, September.
    12. Judith A. Chevalier & Anil K. Kashyap & Peter E. Rossi, 2003. "Why Don't Prices Rise During Periods of Peak Demand? Evidence from Scanner Data," American Economic Review, American Economic Association, vol. 93(1), pages 15-37, March.
    13. García-Escudero, L.A. & Gordaliza, A. & Mayo-Iscar, A. & San Martín, R., 2010. "Robust clusterwise linear regression through trimming," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3057-3069, December.
    14. Adam Tashman & Robert Frey, 2009. "Modeling risk in arbitrage strategies using finite mixtures," Quantitative Finance, Taylor & Francis Journals, vol. 9(5), pages 495-503.
    15. Fair, Ray C & Jaffee, Dwight M, 1972. "Methods of Estimation for Markets in Disequilibrium," Econometrica, Econometric Society, vol. 40(3), pages 497-514, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gabriele Soffritti, 2021. "Estimating the Covariance Matrix of the Maximum Likelihood Estimator Under Linear Cluster-Weighted Models," Journal of Classification, Springer;The Classification Society, vol. 38(3), pages 594-625, October.
    2. Diani, Cecilia & Galimberti, Giuliano & Soffritti, Gabriele, 2022. "Multivariate cluster-weighted models based on seemingly unrelated linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 171(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabriele Perrone & Gabriele Soffritti, 2023. "Seemingly unrelated clusterwise linear regression for contaminated data," Statistical Papers, Springer, vol. 64(3), pages 883-921, June.
    2. Giuliano Galimberti & Gabriele Soffritti, 2020. "Seemingly unrelated clusterwise linear regression," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 14(2), pages 235-260, June.
    3. Diani, Cecilia & Galimberti, Giuliano & Soffritti, Gabriele, 2022. "Multivariate cluster-weighted models based on seemingly unrelated linear regression," Computational Statistics & Data Analysis, Elsevier, vol. 171(C).
    4. Salvatore Ingrassia & Antonio Punzo, 2020. "Cluster Validation for Mixtures of Regressions via the Total Sum of Squares Decomposition," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 526-547, July.
    5. Gabriele Soffritti, 2021. "Estimating the Covariance Matrix of the Maximum Likelihood Estimator Under Linear Cluster-Weighted Models," Journal of Classification, Springer;The Classification Society, vol. 38(3), pages 594-625, October.
    6. Antonio Punzo & Paul. D. McNicholas, 2017. "Robust Clustering in Regression Analysis via the Contaminated Gaussian Cluster-Weighted Model," Journal of Classification, Springer;The Classification Society, vol. 34(2), pages 249-293, July.
    7. Angelo Mazza & Antonio Punzo, 2020. "Mixtures of multivariate contaminated normal regression models," Statistical Papers, Springer, vol. 61(2), pages 787-822, April.
    8. Keefe Murphy & Thomas Brendan Murphy, 2020. "Gaussian parsimonious clustering models with covariates and a noise component," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 14(2), pages 293-325, June.
    9. Nalan Basturk & Richard Paap & Dick van Dijk, 2008. "Structural Differences in Economic Growth," Tinbergen Institute Discussion Papers 08-085/4, Tinbergen Institute.
    10. Wu, Qiang & Yao, Weixin, 2016. "Mixtures of quantile regressions," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 162-176.
    11. Salvatore Ingrassia & Antonio Punzo & Giorgio Vittadini & Simona Minotti, 2015. "Erratum to: The Generalized Linear Mixed Cluster-Weighted Model," Journal of Classification, Springer;The Classification Society, vol. 32(2), pages 327-355, July.
    12. Meng Li & Sijia Xiang & Weixin Yao, 2016. "Robust estimation of the number of components for mixtures of linear regression models," Computational Statistics, Springer, vol. 31(4), pages 1539-1555, December.
    13. Roberto Colombi & Sabrina Giordano, 2019. "Likelihood-based tests for a class of misspecified finite mixture models for ordinal categorical data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 28(4), pages 1175-1202, December.
    14. Derek S. Young & Xi Chen & Dilrukshi C. Hewage & Ricardo Nilo-Poyanco, 2019. "Finite mixture-of-gamma distributions: estimation, inference, and model-based clustering," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 13(4), pages 1053-1082, December.
    15. Kim, Jae-Young, 2014. "An alternative quasi likelihood approach, Bayesian analysis and data-based inference for model specification," Journal of Econometrics, Elsevier, vol. 178(P1), pages 132-145.
    16. Rainer Schlittgen, 2011. "A weighted least-squares approach to clusterwise regression," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 95(2), pages 205-217, June.
    17. Lo, Yungtai, 2011. "Bias from misspecification of the component variances in a normal mixture," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2739-2747, September.
    18. Dolnicar, Sara & Grün, Bettina & Leisch, Friedrich, 2016. "Increasing sample size compensates for data problems in segmentation studies," Journal of Business Research, Elsevier, vol. 69(2), pages 992-999.
    19. Ye He & Ling Zhou & Yingcun Xia & Huazhen Lin, 2023. "Center‐augmented ℓ2‐type regularization for subgroup learning," Biometrics, The International Biometric Society, vol. 79(3), pages 2157-2170, September.
    20. Ang Shan & Fengkai Yang, 2021. "Bayesian Inference for Finite Mixture Regression Model Based on Non-Iterative Algorithm," Mathematics, MDPI, vol. 9(6), pages 1-13, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stmapp:v:30:y:2021:i:1:d:10.1007_s10260-020-00523-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.