IDEAS home Printed from https://ideas.repec.org/a/bla/scjsta/v37y2010i4p612-631.html
   My bibliography  Save this article

On the Bumpy Road to the Dominant Mode

Author

Listed:
  • HUA ZHOU
  • KENNETH L. LANGE

Abstract

. Maximum likelihood estimation in many classical statistical problems is beset by multimodality. This article explores several variations of deterministic annealing that tend to avoid inferior modes and find the dominant mode. In Bayesian settings, annealing can be tailored to find the dominant mode of the log posterior. Our annealing algorithms involve essentially trivial changes to existing optimization algorithms built on block relaxation or the EM or MM principle. Our examples include estimation with the multivariate t distribution, Gaussian mixture models, latent class analysis, factor analysis, multidimensional scaling and a one‐way random effects model. In the numerical examples explored, the proposed annealing strategies significantly improve the chances for locating the global maximum.

Suggested Citation

  • Hua Zhou & Kenneth L. Lange, 2010. "On the Bumpy Road to the Dominant Mode," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 37(4), pages 612-631, December.
    • Handle: RePEc:bla:scjsta:v:37:y:2010:i:4:p:612-631
    DOI: 10.1111/j.1467-9469.2009.00681.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-9469.2009.00681.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1467-9469.2009.00681.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Patrick Groenen & Willem Heiser, 1996. "The tunneling method for global optimization in multidimensional scaling," Psychometrika, Springer;The Psychometric Society, vol. 61(3), pages 529-550, September.
    2. Jiannong Liu & James S. Hodges, 2003. "Posterior bimodality in the balanced one‐way random‐effects model," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(1), pages 247-255, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kenneth Lange & Eric C. Chi & Hua Zhou, 2014. "Rejoinder," International Statistical Review, International Statistical Institute, vol. 82(1), pages 81-89, April.
    2. Utkarsh J. Dang & Antonio Punzo & Paul D. McNicholas & Salvatore Ingrassia & Ryan P. Browne, 2017. "Multivariate Response and Parsimony for Gaussian Cluster-Weighted Models," Journal of Classification, Springer;The Classification Society, vol. 34(1), pages 4-34, April.
    3. O’Hagan, Adrian & Murphy, Thomas Brendan & Gormley, Isobel Claire, 2012. "Computational aspects of fitting mixture models via the expectation–maximization algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 3843-3864.
    4. Adrian O’Hagan & Arthur White, 2019. "Improved model-based clustering performance using Bayesian initialization averaging," Computational Statistics, Springer, vol. 34(1), pages 201-231, March.
    5. Vrbik, I. & McNicholas, P.D., 2012. "Analytic calculations for the EM algorithm for multivariate skew-t mixture models," Statistics & Probability Letters, Elsevier, vol. 82(6), pages 1169-1174.
    6. Paul D. McNicholas, 2016. "Model-Based Clustering," Journal of Classification, Springer;The Classification Society, vol. 33(3), pages 331-373, October.
    7. Hua Zhou & Kenneth Lange, 2015. "Path following in the exact penalty method of convex programming," Computational Optimization and Applications, Springer, vol. 61(3), pages 609-634, July.
    8. Pledger, Shirley & Arnold, Richard, 2014. "Multivariate methods using mixtures: Correspondence analysis, scaling and pattern-detection," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 241-261.
    9. Murray, Paula M. & Browne, Ryan P. & McNicholas, Paul D., 2017. "Hidden truncation hyperbolic distributions, finite mixtures thereof, and their application for clustering," Journal of Multivariate Analysis, Elsevier, vol. 161(C), pages 141-156.
    10. Tang, Yang & Browne, Ryan P. & McNicholas, Paul D., 2015. "Model based clustering of high-dimensional binary data," Computational Statistics & Data Analysis, Elsevier, vol. 87(C), pages 84-101.
    11. Murray, Paula M. & Browne, Ryan P. & McNicholas, Paul D., 2014. "Mixtures of skew-t factor analyzers," Computational Statistics & Data Analysis, Elsevier, vol. 77(C), pages 326-335.
    12. Luca Brusa & Francesco Bartolucci & Fulvia Pennoni, 2023. "Tempered expectation-maximization algorithm for the estimation of discrete latent variable models," Computational Statistics, Springer, vol. 38(3), pages 1391-1424, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. S. Hess & E. Suárez & J. Camacho & G. Ramírez & B. Hernández, 2001. "Reliability of Coordinates Obtained by MINISSA Concerning the Order of Presented Stimuli," Quality & Quantity: International Journal of Methodology, Springer, vol. 35(2), pages 117-128, May.
    2. Michael Brusco & Stephanie Stahl, 2005. "Optimal Least-Squares Unidimensional Scaling: Improved Branch-and-Bound Procedures and Comparison to Dynamic Programming," Psychometrika, Springer;The Psychometric Society, vol. 70(2), pages 253-270, June.
    3. Michael Brusco & Hans-Friedrich Köhn & Stephanie Stahl, 2008. "Heuristic Implementation of Dynamic Programming for Matrix Permutation Problems in Combinatorial Data Analysis," Psychometrika, Springer;The Psychometric Society, vol. 73(3), pages 503-522, September.
    4. de Leeuw, Jan & Mair, Patrick, 2009. "Multidimensional Scaling Using Majorization: SMACOF in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 31(i03).
    5. van den Burg, G.J.J. & Groenen, P.J.F., 2014. "GenSVM: A Generalized Multiclass Support Vector Machine," Econometric Institute Research Papers EI 2014-33, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    6. Groenen, Patrick J. F. & Franses, Philip Hans, 2000. "Visualizing time-varying correlations across stock markets," Journal of Empirical Finance, Elsevier, vol. 7(2), pages 155-172, August.
    7. Groenen, P.J.F. & Kaymak, U. & van Rosmalen, J.M., 2006. "Fuzzy clustering with Minkowski distance," Econometric Institute Research Papers EI 2006-24, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    8. Marc Robini & Pierre-Jean Reissman, 2013. "From simulated annealing to stochastic continuation: a new trend in combinatorial optimization," Journal of Global Optimization, Springer, vol. 56(1), pages 185-215, May.
    9. Michael Brusco & Patrick Doreian, 2015. "An Exact Algorithm for the Two-Mode KL-Means Partitioning Problem," Journal of Classification, Springer;The Classification Society, vol. 32(3), pages 481-515, October.
    10. He, Yi & Hodges, James S., 2008. "Point estimates for variance-structure parameters in Bayesian analysis of hierarchical models," Computational Statistics & Data Analysis, Elsevier, vol. 52(5), pages 2560-2577, January.
    11. Malone, Samuel W. & Tarazaga, Pablo & Trosset, Michael W., 2002. "Better initial configurations for metric multidimensional scaling," Computational Statistics & Data Analysis, Elsevier, vol. 41(1), pages 143-156, November.
    12. Michael Brusco & Hannah J Stolze & Michaela Hoffman & Douglas Steinley, 2017. "A simulated annealing heuristic for maximum correlation core/periphery partitioning of binary networks," PLOS ONE, Public Library of Science, vol. 12(5), pages 1-17, May.
    13. Leung, Pui Lam & Lau, Kin-nam, 2004. "Estimating the city-block two-dimensional scaling model with simulated annealing," European Journal of Operational Research, Elsevier, vol. 158(2), pages 518-524, October.
    14. Lisa Henn & James S. Hodges, 2014. "Multiple Local Maxima in Restricted Likelihoods and Posterior Distributions for Mixed Linear Models," International Statistical Review, International Statistical Institute, vol. 82(1), pages 90-105, April.
    15. Håvard Rue & Sara Martino & Nicolas Chopin, 2009. "Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(2), pages 319-392, April.
    16. Michael Brusco & Douglas Steinley, 2011. "A Tabu-Search Heuristic for Deterministic Two-Mode Blockmodeling of Binary Network Matrices," Psychometrika, Springer;The Psychometric Society, vol. 76(4), pages 612-633, October.
    17. Michael Brusco & Stephanie Stahl, 2001. "An interactive multiobjective programming approach to combinatorial data analysis," Psychometrika, Springer;The Psychometric Society, vol. 66(1), pages 5-24, March.
    18. Groenen, P.J.F. & Borg, I., 2013. "The Past, Present, and Future of Multidimensional Scaling," Econometric Institute Research Papers EI 2013-07, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    19. Groenen, P.J.F. & van de Velden, M., 2004. "Multidimensional scaling," Econometric Institute Research Papers EI 2004-15, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    20. repec:jss:jstsof:31:i03 is not listed on IDEAS

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:scjsta:v:37:y:2010:i:4:p:612-631. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0303-6898 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.