IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v71y2009i3p593-613.html
   My bibliography  Save this article

On‐line expectation–maximization algorithm for latent data models

Author

Listed:
  • Olivier Cappé
  • Eric Moulines

Abstract

Summary. We propose a generic on‐line (also sometimes called adaptive or recursive) version of the expectation–maximization (EM) algorithm applicable to latent variable models of independent observations. Compared with the algorithm of Titterington, this approach is more directly connected to the usual EM algorithm and does not rely on integration with respect to the complete‐data distribution. The resulting algorithm is usually simpler and is shown to achieve convergence to the stationary points of the Kullback–Leibler divergence between the marginal distribution of the observation and the model distribution at the optimal rate, i.e. that of the maximum likelihood estimator. In addition, the approach proposed is also suitable for conditional (or regression) models, as illustrated in the case of the mixture of linear regressions model.

Suggested Citation

  • Olivier Cappé & Eric Moulines, 2009. "On‐line expectation–maximization algorithm for latent data models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(3), pages 593-613, June.
  • Handle: RePEc:bla:jorssb:v:71:y:2009:i:3:p:593-613
    DOI: 10.1111/j.1467-9868.2009.00698.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9868.2009.00698.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1467-9868.2009.00698.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Leisch, Friedrich, 2004. "FlexMix: A General Framework for Finite Mixture Models and Latent Class Regression in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 11(i08).
    2. Liu, Z. & Almhana, J. & Choulakian, V. & McGorman, R., 2006. "Online EM algorithm for mixture with application to internet traffic modeling," Computational Statistics & Data Analysis, Elsevier, vol. 50(4), pages 1052-1071, February.
    3. Wang, Shaojun & Zhao, Yunxin, 2006. "Almost sure convergence of Titterington's recursive estimator for mixture models," Statistics & Probability Letters, Elsevier, vol. 76(18), pages 2001-2006, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shinohara, Shuji & Okamoto, Hiroshi & Manome, Nobuhito & Gunji, Pegio-Yukio & Nakajima, Yoshihiro & Moriyama, Toru & Chung, Ung-il, 2022. "Simulation of foraging behavior using a decision-making agent with Bayesian and inverse Bayesian inference: Temporal correlations and power laws in displacement patterns," Chaos, Solitons & Fractals, Elsevier, vol. 157(C).
    2. Maire, Florian & Moulines, Eric & Lefebvre, Sidonie, 2017. "Online EM for functional data," Computational Statistics & Data Analysis, Elsevier, vol. 111(C), pages 27-47.
    3. L. Ippel & M. C. Kaptein & J. K. Vermunt, 2019. "Estimating Multilevel Models on Data Streams," Psychometrika, Springer;The Psychometric Society, vol. 84(1), pages 41-64, March.
    4. Amy L Cochran & Josh M Cisler, 2019. "A flexible and generalizable model of online latent-state learning," PLOS Computational Biology, Public Library of Science, vol. 15(9), pages 1-31, September.
    5. Shuji Shinohara & Nobuhito Manome & Kouta Suzuki & Ung-il Chung & Tatsuji Takahashi & Hiroshi Okamoto & Yukio Pegio Gunji & Yoshihiro Nakajima & Shunji Mitsuyoshi, 2020. "A new method of Bayesian causal inference in non-stationary environments," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-22, May.
    6. Donna Henderson & Gerton Lunter, 2020. "Efficient inference in state-space models through adaptive learning in online Monte Carlo expectation maximization," Computational Statistics, Springer, vol. 35(3), pages 1319-1344, September.
    7. Johannes Bill & Samuel J. Gershman & Jan Drugowitsch, 2022. "Visual motion perception as online hierarchical inference," Nature Communications, Nature, vol. 13(1), pages 1-17, December.
    8. Ippel, L. & Kaptein, M.C. & Vermunt, J.K., 2016. "Estimating random-intercept models on data streams," Computational Statistics & Data Analysis, Elsevier, vol. 104(C), pages 169-182.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Maire, Florian & Moulines, Eric & Lefebvre, Sidonie, 2017. "Online EM for functional data," Computational Statistics & Data Analysis, Elsevier, vol. 111(C), pages 27-47.
    2. Christian Kleiber & Achim Zeileis, 2016. "Visualizing Count Data Regressions Using Rootograms," The American Statistician, Taylor & Francis Journals, vol. 70(3), pages 296-303, July.
    3. Lebret, Rémi & Iovleff, Serge & Langrognet, Florent & Biernacki, Christophe & Celeux, Gilles & Govaert, Gérard, 2015. "Rmixmod: The R Package of the Model-Based Unsupervised, Supervised, and Semi-Supervised Classification Mixmod Library," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i06).
    4. Grün, Bettina & Kosmidis, Ioannis & Zeileis, Achim, 2012. "Extended Beta Regression in R: Shaken, Stirred, Mixed, and Partitioned," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i11).
    5. Peter Willemé, 2017. "Working Paper 14-17 - Modelling unobserved heterogeneity in distribution - Finite mixtures of the Johnson family of distributions," Working Papers 1714, Federal Planning Bureau, Belgium.
    6. Marc A. Scott & Kaushik Mohan & Jacques‐Antoine Gauthier, 2020. "Model‐based clustering and analysis of life history data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 1231-1251, June.
    7. Fabian Dvorak, 2020. "stratEst: Strategy Estimation in R," TWI Research Paper Series 119, Thurgauer Wirtschaftsinstitut, Universität Konstanz.
    8. Grun, Bettina & Leisch, Friedrich, 2007. "Fitting finite mixtures of generalized linear regressions in R," Computational Statistics & Data Analysis, Elsevier, vol. 51(11), pages 5247-5252, July.
    9. Zeileis, Achim & Kleiber, Christian & Jackman, Simon, 2008. "Regression Models for Count Data in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i08).
    10. Frick, Hannah & Strobl, Carolin & Leisch, Friedrich & Zeileis, Achim, 2012. "Flexible Rasch Mixture Models with Package psychomix," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i07).
    11. Raoofpanah, Iman & Zamudio, César & Groening, Christopher, 2023. "Review reader segmentation based on the heterogeneous impacts of review and reviewer attributes on review helpfulness: A study involving ZIP code data," Journal of Retailing and Consumer Services, Elsevier, vol. 72(C).
    12. Salvatore Ingrassia & Simona Minotti & Giorgio Vittadini, 2012. "Local Statistical Modeling via a Cluster-Weighted Approach with Elliptical Distributions," Journal of Classification, Springer;The Classification Society, vol. 29(3), pages 363-401, October.
    13. Krzysztof Rusek & Agnieszka Kleszcz & Albert Cabellos-Aparicio, 2023. "Bayesian inference of spatial and temporal relations in AI patents for EU countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3313-3335, June.
    14. Ahonen, Ilmari & Nevalainen, Jaakko & Larocque, Denis, 2019. "Prediction with a flexible finite mixture-of-regressions," Computational Statistics & Data Analysis, Elsevier, vol. 132(C), pages 212-224.
    15. Sato, Aki-Hiro, 2012. "Patterns of regional travel behavior: An analysis of Japanese hotel reservation data," International Review of Financial Analysis, Elsevier, vol. 23(C), pages 55-65.
    16. Kim, Sung Hoo & Mokhtarian, Patricia L., 2023. "Finite mixture (or latent class) modeling in transportation: Trends, usage, potential, and future directions," Transportation Research Part B: Methodological, Elsevier, vol. 172(C), pages 134-173.
    17. Ciarleglio, Adam & Todd Ogden, R., 2016. "Wavelet-based scalar-on-function finite mixture regression models," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 86-96.
    18. Maik Dehnert & Josephine Schumann, 2022. "Uncovering the digitalization impact on consumer decision-making for checking accounts in banking," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(3), pages 1503-1528, September.
    19. Chun-Yu Chang & Yueh-Tseng Hou & Yung-Jiun Chien & Yu-Long Chen & Po-Chen Lin & Chien-Sheng Chen & Meng-Yu Wu, 2020. "Two-Thumb or Two-Finger Technique in Infant Cardiopulmonary Resuscitation by a Single Rescuer? A Meta-Analysis with GOSH Analysis," IJERPH, MDPI, vol. 17(14), pages 1-19, July.
    20. Prates, Marcos Oliveira & Lachos, Victor Hugo & Barbosa Cabral, Celso Rômulo, 2013. "mixsmsn: Fitting Finite Mixture of Scale Mixture of Skew-Normal Distributions," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 54(i12).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:71:y:2009:i:3:p:593-613. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.