IDEAS home Printed from https://ideas.repec.org/p/crs/wpaper/2017-39.html
   My bibliography  Save this paper

Concentration of tempered posteriors and of their variational approximations

Author

Listed:
  • Pierre Alquier

    (CREST-ENSAE, CNRS)

  • James Ridgway

    (INRIA)

Abstract

While Bayesian methods are extremely popular in statistics and machine learning, their application to massive datasets is often challenging, when possible at all. Indeed, the classical MCMC algorithms are prohibitively slow when both the model dimension and the sample size are large. Variational Bayesian methods aim at approximating the posterior by a distribution in a tractable family. Thus, MCMC are replaced by an optimization algorithm which is orders of magnitude faster. VB methods have been applied in such computationally demanding applications as including collaborative filtering, image and video processing, NLP and text processing... However, despite very nice results in practice, the theoretical properties of these approximations are usually not known. In this paper, we propose a general approach to prove the concentration of variational approximations of fractional posteriors. We apply our theory to two examples: matrix completion, and Gaussian VB.

Suggested Citation

  • Pierre Alquier & James Ridgway, 2017. "Concentration of tempered posteriors and of their variational approximations," Working Papers 2017-39, Center for Research in Economics and Statistics.
  • Handle: RePEc:crs:wpaper:2017-39
    as

    Download full text from publisher

    File URL: http://crest.science/RePEc/wpstorage/2017-39.pdf
    File Function: CREST working paper version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David M. Blei & Alp Kucukelbir & Jon D. McAuliffe, 2017. "Variational Inference: A Review for Statisticians," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 859-877, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Allassonnière, Stéphanie & Chevallier, Juliette, 2021. "A new class of stochastic EM algorithms. Escaping local maxima and handling intractable sampling," Computational Statistics & Data Analysis, Elsevier, vol. 159(C).
    2. Riccardo Rastelli & Michael Fop, 2020. "A stochastic block model for interaction lengths," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 14(2), pages 485-512, June.
    3. Deborah Gefang & Gary Koop & Aubrey Poon, 2019. "Variational Bayesian Inference in Large Vector Autoregressions with Hierarchical Shrinkage," Discussion Papers in Economics 19/05, Division of Economics, School of Business, University of Leicester.
    4. Korobilis, Dimitris & Koop, Gary, 2018. "Variational Bayes inference in high-dimensional time-varying parameter models," Essex Finance Centre Working Papers 22665, University of Essex, Essex Business School.
    5. Krueger, Rico & Rashidi, Taha H. & Vij, Akshay, 2020. "A Dirichlet process mixture model of discrete choice: Comparisons and a case study on preferences for shared automated vehicles," Journal of choice modelling, Elsevier, vol. 36(C).
    6. Ying C. MacNab, 2018. "Rejoinder on: Some recent work on multivariate Gaussian Markov random fields," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(3), pages 554-569, September.
    7. Chirag Nagpal & Robert E. Tillman & Prashant Reddy & Manuela Veloso, 2020. "Bayesian Consensus: Consensus Estimates from Miscalibrated Instruments under Heteroscedastic Noise," Papers 2004.06565, arXiv.org, revised Jan 2021.
    8. Nathaniel Tomasetti & Catherine Forbes & Anastasios Panagiotelis, 2020. "Updating Variational Bayes: Fast Sequential Posterior Inference," Monash Econometrics and Business Statistics Working Papers 27/20, Monash University, Department of Econometrics and Business Statistics.
    9. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    10. Dimitris Korobilis & Davide Pettenuzzo, 2020. "Machine Learning Econometrics: Bayesian algorithms and methods," Working Papers 2020_09, Business School - Economics, University of Glasgow.
    11. Thayer Alshaabi & David R Dewhurst & James P Bagrow & Peter S Dodds & Christopher M Danforth, 2021. "The sociospatial factors of death: Analyzing effects of geospatially-distributed variables in a Bayesian mortality model for Hong Kong," PLOS ONE, Public Library of Science, vol. 16(3), pages 1-20, March.
    12. Bansal, Prateek & Krueger, Rico & Graham, Daniel J., 2021. "Fast Bayesian estimation of spatial count data models," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    13. Shota Gugushvili & Frank van der Meulen & Moritz Schauer & Peter Spreij, 2018. "Nonparametric Bayesian volatility estimation," Papers 1801.09956, arXiv.org, revised Mar 2019.
    14. Kazuhiro Yamaguchi & Kensuke Okada, 2020. "Variational Bayes Inference for the DINA Model," Journal of Educational and Behavioral Statistics, , vol. 45(5), pages 569-597, October.
    15. Xu, Hang & Alvo, Mayer & Yu, Philip L.H., 2018. "Angle-based models for ranking data," Computational Statistics & Data Analysis, Elsevier, vol. 121(C), pages 113-136.
    16. Gefang, Deborah & Koop, Gary & Poon, Aubrey, 2020. "Computationally efficient inference in large Bayesian mixed frequency VARs," Economics Letters, Elsevier, vol. 191(C).
    17. Bonneville, Christophe & Jenquin, Maxwell & Londono, Juan & Kelly, Alex & Cipolla, Jeffrey & Earls, Christopher, 2021. "Gaussian processes for shock test emulation," Reliability Engineering and System Safety, Elsevier, vol. 212(C).
    18. Melvin Wong & Bilal Farooq, 2019. "Information processing constraints in travel behaviour modelling: A generative learning approach," Papers 1907.07036, arXiv.org, revised Jul 2019.
    19. Alex Burnap & John R. Hauser & Artem Timoshenko, 2019. "Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach," Papers 1907.07786, arXiv.org.
    20. Ning Zhong & David A. Schweidel, 2020. "Capturing Changes in Social Media Content: A Multiple Latent Changepoint Topic Model," Marketing Science, INFORMS, vol. 39(4), pages 827-846, July.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:crs:wpaper:2017-39. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/crestfr.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Secretariat General (email available below). General contact details of provider: https://edirc.repec.org/data/crestfr.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.