IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0131739.html
   My bibliography  Save this article

Overfitting Bayesian Mixture Models with an Unknown Number of Components

Author

Listed:
  • Zoé van Havre
  • Nicole White
  • Judith Rousseau
  • Kerrie Mengersen

Abstract

This paper proposes solutions to three issues pertaining to the estimation of finite mixture models with an unknown number of components: the non-identifiability induced by overfitting the number of components, the mixing limitations of standard Markov Chain Monte Carlo (MCMC) sampling techniques, and the related label switching problem. An overfitting approach is used to estimate the number of components in a finite mixture model via a Zmix algorithm. Zmix provides a bridge between multidimensional samplers and test based estimation methods, whereby priors are chosen to encourage extra groups to have weights approaching zero. MCMC sampling is made possible by the implementation of prior parallel tempering, an extension of parallel tempering. Zmix can accurately estimate the number of components, posterior parameter estimates and allocation probabilities given a sufficiently large sample size. The results will reflect uncertainty in the final model and will report the range of possible candidate models and their respective estimated probabilities from a single run. Label switching is resolved with a computationally light-weight method, Zswitch, developed for overfitted mixtures by exploiting the intuitiveness of allocation-based relabelling algorithms and the precision of label-invariant loss functions. Four simulation studies are included to illustrate Zmix and Zswitch, as well as three case studies from the literature. All methods are available as part of the R package Zmix, which can currently be applied to univariate Gaussian mixture models.

Suggested Citation

  • Zoé van Havre & Nicole White & Judith Rousseau & Kerrie Mengersen, 2015. "Overfitting Bayesian Mixture Models with an Unknown Number of Components," PLOS ONE, Public Library of Science, vol. 10(7), pages 1-27, July.
    • Handle: RePEc:plo:pone00:0131739
    DOI: 10.1371/journal.pone.0131739
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0131739
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0131739&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0131739?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. repec:dau:papers:123456789/6069 is not listed on IDEAS
    2. Lewin Alex & Bochkina Natalia & Richardson Sylvia, 2007. "Fully Bayesian Mixture Model for Differential Gene Expression: Simulations and Model Checks," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 6(1), pages 1-28, December.
    3. James J. Heckman & Christopher R. Taber, 1994. "Econometric Mixture Models and More General Models for Unobservables in Duration Analysis," NBER Technical Working Papers 0157, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Papastamoulis, Panagiotis, 2018. "Overfitting Bayesian mixtures of factor analyzers with an unknown number of components," Computational Statistics & Data Analysis, Elsevier, vol. 124(C), pages 220-234.
    2. Burghardt, Elliot & Sewell, Daniel & Cavanaugh, Joseph, 2022. "Agglomerative and divisive hierarchical Bayesian clustering," Computational Statistics & Data Analysis, Elsevier, vol. 176(C).
    3. Tin Lok James Ng & Andrew Zammit-Mangion, 2024. "Mixture modeling with normalizing flows for spherical density estimation," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 18(1), pages 103-120, March.
    4. Sylvia Frühwirth-Schnatter & Gertraud Malsiner-Walli, 2019. "From here to infinity: sparse finite versus Dirichlet process mixtures in model-based clustering," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 13(1), pages 33-64, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jaap H. Abbring & Gerard J. van den Berg, 2000. "The Non-Parametric Identification of the Mixed Proportional Hazards Competing Risks Model," Tinbergen Institute Discussion Papers 00-066/3, Tinbergen Institute.
    2. Xiaohong Chen & James J. Heckman & Edward Vytlacil, 2000. "Identification and SQRT N Efficient Estimation of Semiparametric Panel Data Models with Binary Dependent Variables and a Latent Factor," Econometric Society World Congress 2000 Contributed Papers 1567, Econometric Society.
    3. Jaap H. Abbring & Gerard J. Van Den Berg, 2007. "The unobserved heterogeneity distribution in duration analysis," Biometrika, Biometrika Trust, vol. 94(1), pages 87-99.
    4. Bo E. Honore & Aureo de Paula, 2007. "Interdependent Durations, Second Version," PIER Working Paper Archive 08-044, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 01 Nov 2008.
    5. Thomas Andren & Daniela Andren, 2006. "Assessing the employment effects of vocational training using a one-factor model," Applied Economics, Taylor & Francis Journals, vol. 38(21), pages 2469-2486.
    6. Abbring, Jaap H. & Berg, Gerard J. van den, 2000. "The non-parametric identification of the mixed proportional hazards competing risks model," Serie Research Memoranda 0024, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics.
    7. Nobel Prize Committee, 2000. "The Scientific Contributions of James Heckman and Daniel McFadden," Nobel Prize in Economics documents 2000-2, Nobel Prize Committee.
    8. Van den Berg, Gerard J., 2001. "Duration models: specification, identification and multiple durations," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 5, chapter 55, pages 3381-3460, Elsevier.
    9. Abbring, Jaap H & van den Berg, Gerard J, 2005. "Social experiments and intrumental variables with duration outcomes," Working Paper Series 2005:11, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    10. Jaap H. Abbring, 2012. "Mixed Hitting‐Time Models," Econometrica, Econometric Society, vol. 80(2), pages 783-819, March.
    11. Falavigna Falavigna & Elena Ragazzi & Lisa Sella, "undated". "Vocational training and labour market: inclusion or segregation paths? An integrated approach on immigrant trainees in Piedmont," CERIS Working Paper 201425, CNR-IRCrES Research Institute on Sustainable Economic Growth - Torino (TO) ITALY - former Institute for Economic Research on Firms and Growth - Moncalieri (TO) ITALY.
    12. Jaap H. Abbring, 2010. "Identification of Dynamic Discrete Choice Models," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 367-394, September.
    13. Jaap Abbring & James Heckman, 2008. "Dynamic policy analysis," CeMMAP working papers CWP05/08, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    14. Botosaru, Irene, 2011. "A Duration Model with Dynamic Unobserved Heterogeneity," TSE Working Papers 11-262, Toulouse School of Economics (TSE), revised Nov 2013.
    15. Prieto-Rodriguez, Juan & Vecco, Marilena, 2021. "Reading between the lines in the art market: Lack of transparency and price heterogeneity as an indicator of multiple equilibria," Economic Modelling, Elsevier, vol. 102(C).
    16. Vinícius Diniz Mayrink & Flávio B. Gonçalves, 2020. "Identifying atypically expressed chromosome regions using RNA-Seq data," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 29(3), pages 619-649, September.
    17. Bo E. Honoré & Aureo de Paula, 2009. ""Interdependent Durations" Third Version," PIER Working Paper Archive 09-039, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 01 Feb 2008.
    18. Brinch,C., 2000. "Identification of structural duration dependence and unobserved heterogeneity with time-varying," Memorandum 20/2000, Oslo University, Department of Economics.
    19. Peng, Yingwei & Zhang, Jiajia, 2008. "Identifiability of a mixture cure frailty model," Statistics & Probability Letters, Elsevier, vol. 78(16), pages 2604-2608, November.
    20. Jaap H. Abbring, 0000. "Mixed Hitting-Time Models," Tinbergen Institute Discussion Papers 07-057/3, Tinbergen Institute, revised 11 Aug 2009.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0131739. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.