IDEAS home Printed from https://ideas.repec.org/p/crs/wpaper/2010-44.html
   My bibliography  Save this paper

Using Parallel Computation to Improve Independent Metropolis-Hastings Based Estimation

Author

Listed:
  • Pierre Jacob

    (Crest)

  • Christian P. Robert

    (Crest)

  • Murray H. Smith

    (Crest)

Abstract

In this paper, we consider the implications of the fact that parallel raw-power can be exploited by a generic Metropolis--Hastings algorithm if the proposed values are independent. In particular, we present improvements to the independent Metropolis--Hastings algorithm that significantly decrease the variance of any estimator derived from the MCMC output, for a null computing cost since those improvements are based on a fixed number of target density evaluations. Furthermore, the techniques developed in this paper do not jeopardize the Markovian convergence properties of the algorithm, since they are based on the Rao--Blackwell principles of Gelfand and Smith (1990), already exploited in Casella and Robert (1996), Atchade and Perron (2005) and Douc and Robert (2010). We illustrate those improvements both on a toy normal example and on a classical probit regression model, but stress the fact that they are applicable in any case where the independent Metropolis-Hastings is applicable.

Suggested Citation

  • Pierre Jacob & Christian P. Robert & Murray H. Smith, 2010. "Using Parallel Computation to Improve Independent Metropolis-Hastings Based Estimation," Working Papers 2010-44, Center for Research in Economics and Statistics.
    • Handle: RePEc:crs:wpaper:2010-44
    as

    Download full text from publisher

    File URL: http://crest.science/RePEc/wpstorage/2010-44.pdf
    File Function: Crest working paper version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. James P. Hobert, 2002. "On the applicability of regenerative simulation in Markov chain Monte Carlo," Biometrika, Biometrika Trust, vol. 89(4), pages 731-743, December.
    2. Craiu, Radu V. & Rosenthal, Jeffrey & Yang, Chao, 2009. "Learn From Thy Neighbor: Parallel-Chain and Regional Adaptive MCMC," Journal of the American Statistical Association, American Statistical Association, vol. 104(488), pages 1454-1466.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Johnson, Alicia A. & Jones, Galin L., 2015. "Geometric ergodicity of random scan Gibbs samplers for hierarchical one-way random effects models," Journal of Multivariate Analysis, Elsevier, vol. 140(C), pages 325-342.
    2. Dimitris Korobilis & Davide Pettenuzzo, 2020. "Machine Learning Econometrics: Bayesian algorithms and methods," Working Papers 2020_09, Business School - Economics, University of Glasgow.
    3. Bertail, Patrice & Clemencon, Stephan, 2008. "Approximate regenerative-block bootstrap for Markov chains," Computational Statistics & Data Analysis, Elsevier, vol. 52(5), pages 2739-2756, January.
    4. Bhattacharya, Sourabh, 2008. "Consistent estimation of the accuracy of importance sampling using regenerative simulation," Statistics & Probability Letters, Elsevier, vol. 78(15), pages 2522-2527, October.
    5. Nalan Baştürk & Stefano Grassi & Lennart Hoogerheide & Herman K. Van Dijk, 2016. "Parallelization Experience with Four Canonical Econometric Models Using ParMitISEM," Econometrics, MDPI, vol. 4(1), pages 1-20, March.
    6. Takefumi Yamazaki, 2018. "Financial friction sources in emerging economies: Structural estimation of sovereign default models," Discussion papers ron303, Policy Research Institute, Ministry of Finance Japan.
    7. Emilio De Santis & Mauro Piccioni, 2008. "Exact Simulation for Discrete Time Spin Systems and Unilateral Fields," Methodology and Computing in Applied Probability, Springer, vol. 10(1), pages 105-120, March.
    8. Murray, Lawrence M., 2015. "Bayesian State-Space Modelling on High-Performance Hardware Using LibBi," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i10).
    9. Pierre E. Jacob & John O’Leary & Yves F. Atchadé, 2020. "Unbiased Markov chain Monte Carlo methods with couplings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(3), pages 543-600, July.
    10. Marco A. Gallegos-Herrada & David Ledvinka & Jeffrey S. Rosenthal, 2024. "Equivalences of Geometric Ergodicity of Markov Chains," Journal of Theoretical Probability, Springer, vol. 37(2), pages 1230-1256, June.
    11. Craiu, V. Radu & Sabeti, Avideh, 2012. "In mixed company: Bayesian inference for bivariate conditional copula models with discrete and continuous outcomes," Journal of Multivariate Analysis, Elsevier, vol. 110(C), pages 106-120.
    12. Ting, Samson & Lymburn, Thomas & Stemler, Thomas & Sun, Yuchao & Small, Michael, 2024. "Parameter estimation for Gipps’ car following model in a Bayesian framework," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 639(C).
    13. Yu Hang Jiang & Tong Liu & Zhiya Lou & Jeffrey S. Rosenthal & Shanshan Shangguan & Fei Wang & Zixuan Wu, 2022. "Markov Chain Confidence Intervals and Biases," International Journal of Statistics and Probability, Canadian Center of Science and Education, vol. 11(1), pages 1-29, March.
    14. M. A. Fleischer & S. H. Jacobson, 2002. "Scale Invariance Properties in the Simulated Annealing Algorithm," Methodology and Computing in Applied Probability, Springer, vol. 4(3), pages 219-241, September.
    15. Fabrizio Leisen & Roberto Casarin & David Luengo & Luca Martino, 2013. "Adaptive Sticky Generalized Metropolis," Working Papers 2013:19, Department of Economics, University of Venice "Ca' Foscari".
    16. Samuel Livingstone, 2021. "Geometric Ergodicity of the Random Walk Metropolis with Position-Dependent Proposal Covariance," Mathematics, MDPI, vol. 9(4), pages 1-14, February.
    17. Rigat, F. & Mira, A., 2012. "Parallel hierarchical sampling: A general-purpose interacting Markov chains Monte Carlo algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 56(6), pages 1450-1467.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:crs:wpaper:2010-44. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Secretariat General (email available below). General contact details of provider: https://edirc.repec.org/data/crestfr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.