IDEAS home Printed from https://ideas.repec.org/a/spr/alstar/v108y2024i2d10.1007_s10182-024-00497-z.html
   My bibliography  Save this article

Bernstein flows for flexible posteriors in variational Bayes

Author

Listed:
  • Oliver Dürr

    (Konstanz University of Applied Sciences)

  • Stefan Hörtling

    (Konstanz University of Applied Sciences)

  • Danil Dold

    (Konstanz University of Applied Sciences)

  • Ivonne Kovylov

    (Konstanz University of Applied Sciences)

  • Beate Sick

    (Zurich University of Applied Sciences
    University of Zurich)

Abstract

Black-box variational inference (BBVI) is a technique to approximate the posterior of Bayesian models by optimization. Similar to MCMC, the user only needs to specify the model; then, the inference procedure is done automatically. In contrast to MCMC, BBVI scales to many observations, is faster for some applications, and can take advantage of highly optimized deep learning frameworks since it can be formulated as a minimization task. In the case of complex posteriors, however, other state-of-the-art BBVI approaches often yield unsatisfactory posterior approximations. This paper presents Bernstein flow variational inference (BF-VI), a robust and easy-to-use method flexible enough to approximate complex multivariate posteriors. BF-VI combines ideas from normalizing flows and Bernstein polynomial-based transformation models. In benchmark experiments, we compare BF-VI solutions with exact posteriors, MCMC solutions, and state-of-the-art BBVI methods, including normalizing flow-based BBVI. We show for low-dimensional models that BF-VI accurately approximates the true posterior; in higher-dimensional models, BF-VI compares favorably against other BBVI methods. Further, using BF-VI, we develop a Bayesian model for the semi-structured melanoma challenge data, combining a CNN model part for image data with an interpretable model part for tabular data, and demonstrate, for the first time, the use of BBVI in semi-structured models.

Suggested Citation

  • Oliver Dürr & Stefan Hörtling & Danil Dold & Ivonne Kovylov & Beate Sick, 2024. "Bernstein flows for flexible posteriors in variational Bayes," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 108(2), pages 375-394, June.
  • Handle: RePEc:spr:alstar:v:108:y:2024:i:2:d:10.1007_s10182-024-00497-z
    DOI: 10.1007/s10182-024-00497-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10182-024-00497-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10182-024-00497-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Torsten Hothorn & Thomas Kneib & Peter Bühlmann, 2014. "Conditional transformation models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(1), pages 3-27, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Benjamin Säfken & David Rügamer, 2024. "Editorial special issue: Bridging the gap between AI and Statistics," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 108(2), pages 225-229, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuanhua Feng & Wolfgang Karl Härdle, 2021. "Uni- and multivariate extensions of the sinh-arcsinh normal distribution applied to distributional regression," Working Papers CIE 142, Paderborn University, CIE Center for International Economics.
    2. Alexander Silbersdorff & Julia Lynch & Stephan Klasen & Thomas Kneib, 2017. "Reconsidering the Income-Illness Relationship using Distributional Regression: An Application to Germany," Courant Research Centre: Poverty, Equity and Growth - Discussion Papers 231, Courant Research Centre PEG.
    3. Alexander Sohn, 2015. "Beyond Conventional Wage Discrimination Analysis: Assessing Comprehensive Wage Distributions of Males and Females Using Structured Additive Distributional Regression," SOEPpapers on Multidisciplinary Panel Data Research 802, DIW Berlin, The German Socio-Economic Panel (SOEP).
    4. Alexander Henzi & Johanna F. Ziegel & Tilmann Gneiting, 2021. "Isotonic distributional regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(5), pages 963-993, November.
    5. Souhaib Ben Taieb & James W. Taylor & Rob J. Hyndman, 2017. "Coherent Probabilistic Forecasts for Hierarchical Time Series," Monash Econometrics and Business Statistics Working Papers 3/17, Monash University, Department of Econometrics and Business Statistics.
    6. Alina Schenk & Moritz Berger & Matthias Schmid, 2024. "Pseudo-value regression trees," Lifetime Data Analysis: An International Journal Devoted to Statistical Methods and Applications for Time-to-Event Data, Springer, vol. 30(2), pages 439-471, April.
    7. Cheng Peng & Stanislav Uryasev, 2023. "Factor Model of Mixtures," Papers 2301.13843, arXiv.org, revised Mar 2023.
    8. Silius M. Vandeskog & Thordis L. Thorarinsdottir & Ingelin Steinsland & Finn Lindgren, 2022. "Quantile based modeling of diurnal temperature range with the five‐parameter lambda distribution," Environmetrics, John Wiley & Sons, Ltd., vol. 33(4), June.
    9. Wiemann, Paul F.V. & Klein, Nadja & Kneib, Thomas, 2022. "Correcting for sample selection bias in Bayesian distributional regression models," Computational Statistics & Data Analysis, Elsevier, vol. 168(C).
    10. Nadja Klein & Torsten Hothorn & Luisa Barbanti & Thomas Kneib, 2022. "Multivariate conditional transformation models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(1), pages 116-142, March.
    11. Jiaan Han & Junxiao Chen & Yanzhe Fu, 2024. "CatNet: Controlling the False Discovery Rate in LSTM with SHAP Feature Importance and Gaussian Mirrors," Papers 2411.16666, arXiv.org, revised Jun 2025.
    12. Alexander Silbersdorff & Julia Lynch & Stephan Klasen & Thomas Kneib, 2018. "Reconsidering the income‐health relationship using distributional regression," Health Economics, John Wiley & Sons, Ltd., vol. 27(7), pages 1074-1088, July.
    13. Möst Lisa & Hothorn Torsten, 2015. "Conditional Transformation Models for Survivor Function Estimation," The International Journal of Biostatistics, De Gruyter, vol. 11(1), pages 23-50, May.
    14. Anatolyev, Stanislav & Baruník, Jozef, 2019. "Forecasting dynamic return distributions based on ordered binary choice," International Journal of Forecasting, Elsevier, vol. 35(3), pages 823-835.
    15. Alexander Silbersdorff & Kai Sebastian Schneider, 2019. "Distributional Regression Techniques in Socioeconomic Research on the Inequality of Health with an Application on the Relationship between Mental Health and Income," IJERPH, MDPI, vol. 16(20), pages 1-28, October.
    16. Julien Hambuckers & Marie Kratz & Antoine Usseglio-Carleve, 2023. "Efficient Estimation In Extreme Value Regression Models Of Hedge Fund Tail Risks," Working Papers hal-04090916, HAL.
    17. Miguel A Delgado & Andrés García-Suaza & Pedro H C Sant’Anna, 2022. "Distribution regression in duration analysis: an application to unemployment spells [Lecture notes in statistics: Proceedings]," The Econometrics Journal, Royal Economic Society, vol. 25(3), pages 675-698.
    18. Kneib, Thomas & Silbersdorff, Alexander & Säfken, Benjamin, 2023. "Rage Against the Mean – A Review of Distributional Regression Approaches," Econometrics and Statistics, Elsevier, vol. 26(C), pages 99-123.
    19. Julien Hambuckers & Marie Kratz & Antoine Usseglio-Carleve, 2023. "Efficient Estimation in Extreme Value Regression Models of Hedge Fund Tail Risks," Papers 2304.06950, arXiv.org.
    20. Samantha Leorato & Franco Peracchi, 2015. "Comparing Distribution and Quantile Regression," EIEF Working Papers Series 1511, Einaudi Institute for Economics and Finance (EIEF), revised Oct 2015.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:alstar:v:108:y:2024:i:2:d:10.1007_s10182-024-00497-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.