IDEAS home Printed from https://ideas.repec.org/a/bla/istatr/v90y2022is1ps52-s66.html
   My bibliography  Save this article

A Legacy of EM Algorithms

Author

Listed:
  • Kenneth Lange
  • Hua Zhou

Abstract

Nan Laird has an enormous and growing impact on computational statistics. Her paper with Dempster and Rubin on the expectation‐maximisation (EM) algorithm is the second most cited paper in statistics. Her papers and book on longitudinal modelling are nearly as impressive. In this brief survey, we revisit the derivation of some of her most useful algorithms from the perspective of the minorisation‐maximisation (MM) principle. The MM principle generalises the EM principle and frees it from the shackles of missing data and conditional expectations. Instead, the focus shifts to the construction of surrogate functions via standard mathematical inequalities. The MM principle can deliver a classical EM algorithm with less fuss or an entirely new algorithm with a faster rate of convergence. In any case, the MM principle enriches our understanding of the EM principle and suggests new algorithms of considerable potential in high‐dimensional settings where standard algorithms such as Newton's method and Fisher scoring falter.

Suggested Citation

  • Kenneth Lange & Hua Zhou, 2022. "A Legacy of EM Algorithms," International Statistical Review, International Statistical Institute, vol. 90(S1), pages 52-66, December.
    • Handle: RePEc:bla:istatr:v:90:y:2022:i:s1:p:s52-s66
    DOI: 10.1111/insr.12526
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/insr.12526
    Download Restriction: no
    File URL: https://libkey.io/10.1111/insr.12526?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Dankmar Böhning & Bruce Lindsay, 1988. "Monotonicity of quadratic-approximation algorithms," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 40(4), pages 641-663, December.
    2. de Leeuw, Jan & Lange, Kenneth, 2009. "Sharp quadratic majorization in one dimension," Computational Statistics & Data Analysis, Elsevier, vol. 53(7), pages 2471-2484, May.
    3. Glanz, Hunter & Carvalho, Luis, 2018. "An expectation–maximization algorithm for the matrix normal distribution with an application in remote sensing," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 31-48.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wang, Fa, 2017. "Maximum likelihood estimation and inference for high dimensional nonlinear factor models with application to factor-augmented regressions," MPRA Paper 93484, University Library of Munich, Germany, revised 19 May 2019.
    2. de Leeuw, Jan, 2006. "Principal component analysis of binary data by iterated singular value decomposition," Computational Statistics & Data Analysis, Elsevier, vol. 50(1), pages 21-39, January.
    3. Roussille, Nina & Scuderi, Benjamin, 2023. "Bidding for Talent: A Test of Conduct in a High-Wage Labor Market," IZA Discussion Papers 16352, Institute of Labor Economics (IZA).
    4. Om Prakash Yadav & Shashwati Ray, 2021. "A novel method of preprocessing and modeling ECG signals with Lagrange–Chebyshev interpolating polynomials," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 12(3), pages 377-390, June.
    5. Wang, Fa, 2022. "Maximum likelihood estimation and inference for high dimensional generalized factor models with application to factor-augmented regressions," Journal of Econometrics, Elsevier, vol. 229(1), pages 180-200.
    6. Utkarsh J. Dang & Michael P.B. Gallaugher & Ryan P. Browne & Paul D. McNicholas, 2023. "Model-Based Clustering and Classification Using Mixtures of Multivariate Skewed Power Exponential Distributions," Journal of Classification, Springer;The Classification Society, vol. 40(1), pages 145-167, April.
    7. Unkel, S. & Trendafilov, N.T., 2010. "A majorization algorithm for simultaneous parameter estimation in robust exploratory factor analysis," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3348-3358, December.
    8. Tian, Guo-Liang & Tang, Man-Lai & Liu, Chunling, 2012. "Accelerating the quadratic lower-bound algorithm via optimizing the shrinkage parameter," Computational Statistics & Data Analysis, Elsevier, vol. 56(2), pages 255-265.
    9. Tian, Guo-Liang & Tang, Man-Lai & Fang, Hong-Bin & Tan, Ming, 2008. "Efficient methods for estimating constrained parameters with applications to regularized (lasso) logistic regression," Computational Statistics & Data Analysis, Elsevier, vol. 52(7), pages 3528-3542, March.
    10. Kim, Nam-Hwui & Browne, Ryan P., 2021. "In the pursuit of sparseness: A new rank-preserving penalty for a finite mixture of factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 160(C).
    11. Dankmar Böhning, 1992. "Multinomial logistic regression algorithm," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 44(1), pages 197-200, March.
    12. Bohning, Dankmar, 1999. "The lower bound method in probit regression," Computational Statistics & Data Analysis, Elsevier, vol. 30(1), pages 13-17, March.
    13. Liu, Wenchen & Tang, Yincai & Wu, Xianyi, 2020. "Separating variables to accelerate non-convex regularized optimization," Computational Statistics & Data Analysis, Elsevier, vol. 147(C).
    14. Bansal, Prateek & Daziano, Ricardo A & Guerra, Erick, 2018. "Minorization-Maximization (MM) algorithms for semiparametric logit models: Bottlenecks, extensions, and comparisons," Transportation Research Part B: Methodological, Elsevier, vol. 115(C), pages 17-40.
    15. Lloyd-Jones, Luke R. & Nguyen, Hien D. & McLachlan, Geoffrey J., 2018. "A globally convergent algorithm for lasso-penalized mixture of linear regression models," Computational Statistics & Data Analysis, Elsevier, vol. 119(C), pages 19-38.
    16. de Leeuw, Jan & Lange, Kenneth, 2009. "Sharp quadratic majorization in one dimension," Computational Statistics & Data Analysis, Elsevier, vol. 53(7), pages 2471-2484, May.
    17. Jonathan James, 2012. "A tractable estimator for general mixed multinomial logit models," Working Papers (Old Series) 1219, Federal Reserve Bank of Cleveland.
    18. Bissantz, Nicolai & Dümbgen, Lutz & Munk, Axel & Stratmann, Bernd, 2008. "Convergence analysis of generalized iteratively reweighted least squares algorithms on convex function spaces," Technical Reports 2008,25, Technische Universität Dortmund, Sonderforschungsbereich 475: Komplexitätsreduktion in multivariaten Datenstrukturen.
    19. Lee, Sangin & Kwon, Sunghoon & Kim, Yongdai, 2016. "A modified local quadratic approximation algorithm for penalized optimization problems," Computational Statistics & Data Analysis, Elsevier, vol. 94(C), pages 275-286.
    20. Takayuki Kawashima & Hironori Fujisawa, 2023. "Robust regression against heavy heterogeneous contamination," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 86(4), pages 421-442, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:istatr:v:90:y:2022:i:s1:p:s52-s66. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/isiiinl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.