IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v66y2025i6d10.1007_s00362-025-01749-z.html
   My bibliography  Save this article

Convergence of the EM algorithm in KL distance for overspecified Gaussian mixtures

Author

Listed:
  • Alan Legg

    (Purdue University Fort Wayne)

  • Artur Pak

    (Nazarbayev University
    Institute of Mathematics and Mathematical Modeling)

  • Igor Melnykov

    (University of Minnesota Duluth)

  • Arman Bolatov

    (Mohamed bin Zayed University of Artificial Intelligence
    Institute of Mathematics and Mathematical Modeling)

  • Zhenisbek Assylbekov

    (Purdue University Fort Wayne
    Institute of Mathematics and Mathematical Modeling)

Abstract

We present a study of the convergence properties of the Expectation-Maximization (EM) algorithm when applied to an overspecified model. In particular, we consider fitting a balanced mixture of two Gaussians to data originating from a single Gaussian. We provide theoretical bounds on the Kullback–Leibler (KL) divergence between the fitted and true distributions. An important feature is concavity and radiality of the expected log-likelihood function on a hypersurface induced by the EM algorithm, which greatly simplifies the analysis. We also show how our result on KL divergence can be used to upperbound the error rate of a mixture discriminant analysis classifier trained by the EM algorithm.

Suggested Citation

  • Alan Legg & Artur Pak & Igor Melnykov & Arman Bolatov & Zhenisbek Assylbekov, 2025. "Convergence of the EM algorithm in KL distance for overspecified Gaussian mixtures," Statistical Papers, Springer, vol. 66(6), pages 1-33, October.
    • Handle: RePEc:spr:stpapr:v:66:y:2025:i:6:d:10.1007_s00362-025-01749-z
    DOI: 10.1007/s00362-025-01749-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-025-01749-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-025-01749-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:66:y:2025:i:6:d:10.1007_s00362-025-01749-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.