IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v80y2018i4p749-767.html
   My bibliography  Save this article

Auxiliary gradient‐based sampling algorithms

Author

Listed:
  • Michalis K. Titsias
  • Omiros Papaspiliopoulos

Abstract

We introduce a new family of Markov chain Monte Carlo samplers that combine auxiliary variables, Gibbs sampling and Taylor expansions of the target density. Our approach permits the marginalization over the auxiliary variables, yielding marginal samplers, or the augmentation of the auxiliary variables, yielding auxiliary samplers. The well‐known Metropolis‐adjusted Langevin algorithm MALA and preconditioned Crank–Nicolson–Langevin algorithm pCNL are shown to be special cases. We prove that marginal samplers are superior in terms of asymptotic variance and demonstrate cases where they are slower in computing time compared with auxiliary samplers. In the context of latent Gaussian models we propose new auxiliary and marginal samplers whose implementation requires a single tuning parameter, which can be found automatically during the transient phase. Extensive experimentation shows that the increase in efficiency (measured as the effective sample size per unit of computing time) relative to (optimized implementations of) pCNL, elliptical slice sampling and MALA ranges from tenfold in binary classification problems to 25 fold in log‐Gaussian Cox processes to 100 fold in Gaussian process regression, and it is on a par with Riemann manifold Hamiltonian Monte Carlo sampling in an example where that algorithm has the same complexity as the aforementioned algorithms. We explain this remarkable improvement in terms of the way that alternative samplers try to approximate the eigenvalues of the target. We introduce a novel Markov chain Monte Carlo sampling scheme for hyperparameter learning that builds on the auxiliary samplers. The MATLAB code for reproducing the experiments in the paper is publicly available and an on‐line supplement to this paper contains additional experiments and implementation details.

Suggested Citation

  • Michalis K. Titsias & Omiros Papaspiliopoulos, 2018. "Auxiliary gradient‐based sampling algorithms," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(4), pages 749-767, September.
  • Handle: RePEc:bla:jorssb:v:80:y:2018:i:4:p:749-767
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssb.12269
    Download Restriction: no

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Angelos Alexopoulos & Petros Dellaportas & Omiros Papaspiliopoulos, 2019. "Bayesian prediction of jumps in large panels of time series data," Papers 1904.05312, arXiv.org.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:80:y:2018:i:4:p:749-767. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley Content Delivery). General contact details of provider: http://edirc.repec.org/data/rssssea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.