IDEAS home Printed from https://ideas.repec.org/a/bla/istatr/v82y2014i2p296-308.html

Numerical Maximisation of Likelihood: A Neglected Alternative to EM?

Author

Listed:
  • Iain L. MacDonald

Abstract

type="main" xml:id="insr12041-abs-0001"> There is by now a long tradition of using the EM algorithm to find maximum-likelihood estimates (MLEs) when the data are incomplete in any of a wide range of ways, even when the observed-data likelihood can easily be evaluated and numerical maximisation of that likelihood is available as a conceptually simple route to the MLEs. It is rare in the literature to see numerical maximisation employed if EM is possible. But with excellent general-purpose numerical optimisers now available free, there is no longer any reason, as a matter of course, to avoid direct numerical maximisation of likelihood. In this tutorial, I present seven examples of models in which numerical maximisation of likelihood appears to have some advantages over the use of EM as a route to MLEs. The mathematical and coding effort is minimal, as there is no need to derive and code the E and M steps, only a likelihood evaluator. In all the examples, the unconstrained optimiser nlm available in R is used, and transformations are used to impose constraints on parameters. I suggest therefore that the following question be asked of proposed new applications of EM: Can the MLEs be found more simply and directly by using a general-purpose numerical optimiser?

Suggested Citation

  • Iain L. MacDonald, 2014. "Numerical Maximisation of Likelihood: A Neglected Alternative to EM?," International Statistical Review, International Statistical Institute, vol. 82(2), pages 296-308, August.
    • Handle: RePEc:bla:istatr:v:82:y:2014:i:2:p:296-308
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/insr.12041
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Walter Zucchini & David Raubenheimer & Iain L. MacDonald, 2008. "Modeling Time Series of Animal Behavior by Means of a Latent‐State Model with Feedback," Biometrics, The International Biometric Society, vol. 64(3), pages 807-815, September.
    2. Altman, Rachel MacKay, 2007. "Mixed Hidden Markov Models: An Extension of the Hidden Markov Model to the Longitudinal Data Setting," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 201-210, March.
    3. A. Narayanan, 1991. "Maximum Likelihood Estimation of the Parameters of the Dirichlet Distribution," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 40(2), pages 365-374, June.
    4. Bruce A. Craig & Peter P. Sendi, 2002. "Estimation of the transition matrix of a discrete‐time Markov chain," Health Economics, John Wiley & Sons, Ltd., vol. 11(1), pages 33-42, January.
    5. A. Azzalini & A. Capitanio, 1999. "Statistical applications of the multivariate skew normal distribution," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(3), pages 579-602.
    6. Turner, Rolf, 2008. "Direct maximization of the likelihood of a hidden Markov model," Computational Statistics & Data Analysis, Elsevier, vol. 52(9), pages 4147-4160, May.
    7. Langrock, Roland & MacDonald, Iain L. & Zucchini, Walter, 2012. "Some nonstandard stochastic volatility models and their estimation using structured hidden Markov models," Journal of Empirical Finance, Elsevier, vol. 19(1), pages 147-161.
    8. Jan Bulla & Andreas Berzel, 2008. "Computational issues in parameter estimation for stationary hidden Markov models," Computational Statistics, Springer, vol. 23(1), pages 1-18, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Roland Langrock & Thomas Kneib & Alexander Sohn & Stacy L. DeRuiter, 2015. "Nonparametric inference in hidden Markov models using P-splines," Biometrics, The International Biometric Society, vol. 71(2), pages 520-528, June.
    2. Shi, Yue & Punzo, Antonio & Otneim, Håkon & Maruotti, Antonello, 2023. "Hidden semi-Markov models for rainfall-related insurance claims," Discussion Papers 2023/17, Norwegian School of Economics, Department of Business and Management Science.
    3. Cremaschini, Alessandro & Maruotti, Antonello, 2023. "A finite mixture analysis of structural breaks in the G-7 gross domestic product series," Research in Economics, Elsevier, vol. 77(1), pages 76-90.
    4. Ting Wang & Jiancang Zhuang & Kazushige Obara & Hiroshi Tsuruoka, 2017. "Hidden Markov modelling of sparse time series from non-volcanic tremor observations," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 66(4), pages 691-715, August.
    5. Marios Kokkodis & Panagiotis G. Ipeirotis, 2023. "The Good, the Bad, and the Unhirable: Recommending Job Applicants in Online Labor Markets," Management Science, INFORMS, vol. 69(11), pages 6969-6987, November.
    6. Subhajit Chattopadhyay, 2025. "Finite mixture copulas for modeling dependence in longitudinal count data," METRON, Springer;Sapienza Università di Roma, vol. 83(2), pages 183-212, August.
    7. Shi, Yue & Punzo, Antonio & Otneim, Håkon & Maruotti, Antonello, 2025. "Hidden semi-Markov models for rainfall-related insurance claims," Insurance: Mathematics and Economics, Elsevier, vol. 120(C), pages 91-106.
    8. Toby A. Patterson & Alison Parton & Roland Langrock & Paul G. Blackwell & Len Thomas & Ruth King, 2017. "Statistical modelling of individual animal movement: an overview of key methods and a discussion of practical challenges," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 101(4), pages 399-438, October.
    9. Antonello Maruotti & Antonio Punzo, 2021. "Initialization of Hidden Markov and Semi‐Markov Models: A Critical Evaluation of Several Strategies," International Statistical Review, International Statistical Institute, vol. 89(3), pages 447-480, December.
    10. Maruotti, Antonello & Punzo, Antonio, 2017. "Model-based time-varying clustering of multivariate longitudinal data with covariates and outliers," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 475-496.
    11. Gordon Anderson & Alessio Farcomeni & Grazia Pittau & Roberto Zelli, 2017. "Rectangular latent Markov models for time-specific clustering," Working Papers tecipa-589, University of Toronto, Department of Economics.
    12. Roland Langrock & Timo Adam & Vianey Leos‐Barajas & Sina Mews & David L. Miller & Yannis P. Papastamatiou, 2018. "Spline‐based nonparametric inference in general state‐switching models," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 72(3), pages 179-200, August.
    13. Gordon Anderson & Alessio Farcomeni & Maria Grazia Pittau & Roberto Zelli, 2019. "Rectangular latent Markov models for time‐specific clustering, with an analysis of the wellbeing of nations," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 68(3), pages 603-621, April.
    14. Punzo, Antonio & Bagnato, Luca & Maruotti, Antonello, 2018. "Compound unimodal distributions for insurance losses," Insurance: Mathematics and Economics, Elsevier, vol. 81(C), pages 95-107.
    15. Christian H. Weiß & Osama Swidan, 2025. "Hidden-Markov models for ordinal time series," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 109(2), pages 217-239, June.
    16. Iain L. MacDonald & Brendon M. Lapham, 2016. "Even More Direct Calculation of the Variance of a Maximum Penalized-Likelihood Estimator," The American Statistician, Taylor & Francis Journals, vol. 70(1), pages 114-118, February.
    17. Maruotti, Antonello & Petrella, Lea & Sposito, Luca, 2021. "Hidden semi-Markov-switching quantile regression for time series," Computational Statistics & Data Analysis, Elsevier, vol. 159(C).
    18. Antonello Maruotti, 2015. "Handling non-ignorable dropouts in longitudinal data: a conditional model based on a latent Markov heterogeneity structure," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(1), pages 84-109, March.
    19. Alessio Farcomeni, 2015. "Generalized Linear Mixed Models Based on Latent Markov Heterogeneity Structures," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 42(4), pages 1127-1135, December.
    20. Koslik, Jan-Ole, 2025. "Hidden semi-Markov models with inhomogeneous state dwell-time distributions," Computational Statistics & Data Analysis, Elsevier, vol. 209(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roland Langrock & Timo Adam & Vianey Leos‐Barajas & Sina Mews & David L. Miller & Yannis P. Papastamatiou, 2018. "Spline‐based nonparametric inference in general state‐switching models," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 72(3), pages 179-200, August.
    2. Roland Langrock & Thomas Kneib & Alexander Sohn & Stacy L. DeRuiter, 2015. "Nonparametric inference in hidden Markov models using P-splines," Biometrics, The International Biometric Society, vol. 71(2), pages 520-528, June.
    3. Kristian Gundersen & Timothée Bacri & Jan Bulla & Sondre Hølleland & Antonello Maruotti & Bård Støve, 2024. "Testing for time‐varying nonlinear dependence structures: Regime‐switching and local Gaussian correlation," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 51(3), pages 1012-1060, September.
    4. Zhou, Jie & Song, Xinyuan & Sun, Liuquan, 2020. "Continuous time hidden Markov model for longitudinal data," Journal of Multivariate Analysis, Elsevier, vol. 179(C).
    5. F. Bartolucci & A. Farcomeni & F. Pennoni, 2014. "Latent Markov models: a review of a general framework for the analysis of longitudinal data with covariates," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 23(3), pages 433-465, September.
    6. Maruotti, Antonello & Punzo, Antonio, 2017. "Model-based time-varying clustering of multivariate longitudinal data with covariates and outliers," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 475-496.
    7. Catania, Leopoldo & Di Mari, Roberto, 2021. "Hierarchical Markov-switching models for multivariate integer-valued time-series," Journal of Econometrics, Elsevier, vol. 221(1), pages 118-137.
    8. Gordon Anderson & Alessio Farcomeni & Maria Grazia Pittau & Roberto Zelli, 2019. "Rectangular latent Markov models for time‐specific clustering, with an analysis of the wellbeing of nations," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 68(3), pages 603-621, April.
    9. Gordon Anderson & Alessio Farcomeni & Grazia Pittau & Roberto Zelli, 2017. "Rectangular latent Markov models for time-specific clustering," Working Papers tecipa-589, University of Toronto, Department of Economics.
    10. Antonello Maruotti & Antonio Punzo, 2021. "Initialization of Hidden Markov and Semi‐Markov Models: A Critical Evaluation of Several Strategies," International Statistical Review, International Statistical Institute, vol. 89(3), pages 447-480, December.
    11. Ari Hyytinen & Frode Steen & Otto Toivanen, 2018. "Cartels Uncovered," American Economic Journal: Microeconomics, American Economic Association, vol. 10(4), pages 190-222, November.
    12. Raúl Alejandro Morán-Vásquez & Edwin Zarrazola & Daya K. Nagar, 2022. "Some Statistical Aspects of the Truncated Multivariate Skew- t Distribution," Mathematics, MDPI, vol. 10(15), pages 1-14, August.
    13. Lucia Zanotto & Vladimir Canudas-Romo & Stefano Mazzuco, 2021. "A Mixture-Function Mortality Model: Illustration of the Evolution of Premature Mortality," European Journal of Population, Springer;European Association for Population Studies, vol. 37(1), pages 1-27, March.
    14. Ferraz, V.R.S. & Moura, F.A.S., 2012. "Small area estimation using skew normal models," Computational Statistics & Data Analysis, Elsevier, vol. 56(10), pages 2864-2874.
    15. Omid Karimi, 2024. "A Hamiltonian Monte Carlo EM algorithm for generalized linear mixed models with spatial skew latent variables," Statistical Papers, Springer, vol. 65(2), pages 1065-1084, April.
    16. Padilla, Juan L. & Azevedo, Caio L.N. & Lachos, Victor H., 2018. "Multidimensional multiple group IRT models with skew normal latent trait distributions," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 250-268.
    17. Li, Baokun & Tian, Weizhong & Wang, Tonghui, 2018. "Remarks for the singular multivariate skew-normal distribution and its quadratic forms," Statistics & Probability Letters, Elsevier, vol. 137(C), pages 105-112.
    18. Elisa Bocchialini & Demetrio Miloslavo Bova & Ilaria Colivicchi & Federica Ielasi, 2024. "Environmental-, Social-, and Governance-Oriented Pension Funds for Young Contributors: A Win–Win Option," Sustainability, MDPI, vol. 16(24), pages 1-19, December.
    19. Xuedong Chen & Qianying Zeng & Qiankun Song, 2014. "Likelihood Inference of Nonlinear Models Based on a Class of Flexible Skewed Distributions," Abstract and Applied Analysis, John Wiley & Sons, vol. 2014(1).
    20. Risha Gidwani & Louise B. Russell, 2020. "Correction to: Estimating Transition Probabilities from Published Evidence: A Tutorial for Decision Modelers," PharmacoEconomics, Springer, vol. 38(11), pages 1277-1277, November.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:istatr:v:82:y:2014:i:2:p:296-308. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/isiiinl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.