IDEAS home Printed from https://ideas.repec.org/a/spr/metcap/v18y2016i1d10.1007_s11009-014-9395-6.html
   My bibliography  Save this article

Adaptive Rejection Metropolis Simulated Annealing for Detecting Global Maximum Regions

Author

Listed:
  • Huaiye Zhang

    (Virginia Polytechnic Institute and State University)

  • Inyoung Kim

    (Virginia Polytechnic Institute and State University)

Abstract

A finite mixture model has been used to fit the data from heterogeneous populations to many applications. An Expectation Maximization (EM) algorithm is the most popular method to estimate parameters in a finite mixture model. A Bayesian approach is another method for fitting a mixture model. However, the EM algorithm often converges to the local maximum regions, and it is sensitive to the choice of starting points. In the Bayesian approach, the Markov Chain Monte Carlo (MCMC) sometimes converges to the local mode and is difficult to move to another mode. Hence, in this paper we propose a new method to improve the limitation of EM algorithm so that the EM can estimate the parameters at the global maximum region and to develop a more effective Bayesian approach so that the MCMC chain moves from one mode to another more easily in the mixture model. Our approach is developed by using both simulated annealing (SA) and adaptive rejection metropolis sampling (ARMS). Although SA is a well-known approach for detecting distinct modes, the limitation of SA is the difficulty in choosing sequences of proper proposal distributions for a target distribution. Since ARMS uses a piecewise linear envelope function for a proposal distribution, we incorporate ARMS into an SA approach so that we can start a more proper proposal distribution and detect separate modes. As a result, we can detect the maximum region and estimate parameters for this global region. We refer to this approach as ARMS annealing. By putting together ARMS annealing with the EM algorithm and with the Bayesian approach, respectively, we have proposed two approaches: an EM-ARMS annealing algorithm and a Bayesian-ARMS annealing approach. We compare our two approaches with traditional EM algorithm alone and Bayesian approach alone using simulation, showing that our two approaches are comparable to each other but perform better than EM algorithm alone and Bayesian approach alone. Our two approaches detect the global maximum region well and estimate the parameters in this region. We demonstrate the advantage of our approaches using an example of the mixture of two Poisson regression models. This mixture model is used to analyze a survey data on the number of charitable donations.

Suggested Citation

  • Huaiye Zhang & Inyoung Kim, 2016. "Adaptive Rejection Metropolis Simulated Annealing for Detecting Global Maximum Regions," Methodology and Computing in Applied Probability, Springer, vol. 18(1), pages 1-19, March.
    • Handle: RePEc:spr:metcap:v:18:y:2016:i:1:d:10.1007_s11009-014-9395-6
    DOI: 10.1007/s11009-014-9395-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11009-014-9395-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11009-014-9395-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. W. R. Gilks & N. G. Best & K. K. C. Tan, 1995. "Adaptive Rejection Metropolis Sampling Within Gibbs Sampling," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 44(4), pages 455-472, December.
    2. W. R. Gilks & P. Wild, 1992. "Adaptive Rejection Sampling for Gibbs Sampling," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 41(2), pages 337-348, June.
    3. Duncan, Brian, 1999. "Modeling charitable contributions of time and money," Journal of Public Economics, Elsevier, vol. 72(2), pages 213-242, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hazan, Alon & Landsman, Zinoviy & E Makov, Udi, 2003. "Robustness via a mixture of exponential power distributions," Computational Statistics & Data Analysis, Elsevier, vol. 42(1-2), pages 111-121, February.
    2. Xiao Li & Michele Guindani & Chaan S. Ng & Brian P. Hobbs, 2021. "A Bayesian nonparametric model for textural pattern heterogeneity," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(2), pages 459-480, March.
    3. Jelena Nikolić & Danijela Aleksić & Zoran Perić & Milan Dinčić, 2021. "Iterative Algorithm for Parameterization of Two-Region Piecewise Uniform Quantizer for the Laplacian Source," Mathematics, MDPI, vol. 9(23), pages 1-14, November.
    4. Chunling Wang & Xiaoyan Lin, 2022. "Bayesian Semiparametric Regression Analysis of Multivariate Panel Count Data," Stats, MDPI, vol. 5(2), pages 1-17, May.
    5. Meyer, Renate & Cai, Bo & Perron, François, 2008. "Adaptive rejection Metropolis sampling using Lagrange interpolation polynomials of degree 2," Computational Statistics & Data Analysis, Elsevier, vol. 52(7), pages 3408-3423, March.
    6. Cai, Bo & Lin, Xiaoyan & Wang, Lianming, 2011. "Bayesian proportional hazards model for current status data with monotone splines," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2644-2651, September.
    7. Mazucheli, Josmar & Louzada-Neto, Francisco & Achcar, Jorge A., 2001. "Bayesian inference for polyhazard models in the presence of covariates," Computational Statistics & Data Analysis, Elsevier, vol. 38(1), pages 1-14, November.
    8. Yi-Ping Chang & Chih-Tun Yu, 2014. "Bayesian confidence intervals for probability of default and asset correlation of portfolio credit risk," Computational Statistics, Springer, vol. 29(1), pages 331-361, February.
    9. Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(3), pages 361-393.
    10. Gareth W. Peters & Pavel V. Shevchenko & Mario V. Wuthrich, 2009. "Dynamic operational risk: modeling dependence and combining different sources of information," Papers 0904.4074, arXiv.org, revised Jul 2009.
    11. H. Abebe & F. Tan & G. Breukelen & M. Berger, 2014. "Robustness of Bayesian D-optimal design for the logistic mixed model against misspecification of autocorrelation," Computational Statistics, Springer, vol. 29(6), pages 1667-1690, December.
    12. Yu Yue & Paul Speckman & Dongchu Sun, 2012. "Priors for Bayesian adaptive spline smoothing," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 64(3), pages 577-613, June.
    13. Cappuccio Nunzio & Lubian Diego & Raggi Davide, 2004. "MCMC Bayesian Estimation of a Skew-GED Stochastic Volatility Model," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 8(2), pages 1-31, May.
    14. White, Gentry & Porter, Michael D., 2014. "GPU accelerated MCMC for modeling terrorist activity," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 643-651.
    15. Neville Francis & Laura E. Jackson & Michael T. Owyang, 2018. "Countercyclical Policy and the Speed of Recovery after Recessions," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 50(4), pages 675-704, June.
    16. Helio Migon & Alexandra Schmidt & Romy Ravines & João Pereira, 2013. "An efficient sampling scheme for dynamic generalized models," Computational Statistics, Springer, vol. 28(5), pages 2267-2293, October.
    17. Pan, Chun & Cai, Bo & Wang, Lianming & Lin, Xiaoyan, 2014. "Bayesian semiparametric model for spatially correlated interval-censored survival data," Computational Statistics & Data Analysis, Elsevier, vol. 74(C), pages 198-208.
    18. Abebe, Haftom T. & Tan, Frans E.S. & Van Breukelen, Gerard J.P. & Berger, Martijn P.F., 2014. "Bayesian D-optimal designs for the two parameter logistic mixed effects model," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 1066-1076.
    19. Pang, W. K. & Yang, Z. H. & Hou, S. H. & Leung, P. K., 2002. "Non-uniform random variate generation by the vertical strip method," European Journal of Operational Research, Elsevier, vol. 142(3), pages 595-609, November.
    20. Al-Ubaydli, Omar & Yeomans, Mike, 2017. "Do people donate more when they perceive a single beneficiary whom they know? A field experimental test of the identifiability effect," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 66(C), pages 96-103.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:metcap:v:18:y:2016:i:1:d:10.1007_s11009-014-9395-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.