IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v71y2009i2p319-392.html
   My bibliography  Save this article

Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations

Author

Listed:
  • Håvard Rue
  • Sara Martino
  • Nicolas Chopin

Abstract

Structured additive regression models are perhaps the most commonly used class of models in statistical applications. It includes, among others, (generalized) linear models, (generalized) additive models, smoothing spline models, state space models, semiparametric regression, spatial and spatiotemporal models, log-Gaussian Cox processes and geostatistical and geoadditive models. We consider approximate Bayesian inference in a popular subset of structured additive regression models, "latent Gaussian models", where the latent field is Gaussian, controlled by a few hyperparameters and with non-Gaussian response variables. The posterior marginals are not available in closed form owing to the non-Gaussian response variables. For such models, Markov chain Monte Carlo methods can be implemented, but they are not without problems, in terms of both convergence and computational time. In some practical applications, the extent of these problems is such that Markov chain Monte Carlo sampling is simply not an appropriate tool for routine analysis. We show that, by using an integrated nested Laplace approximation and its simplified version, we can directly compute very accurate approximations to the posterior marginals. The main benefit of these approximations is computational: where Markov chain Monte Carlo algorithms need hours or days to run, our approximations provide more precise estimates in seconds or minutes. Another advantage with our approach is its generality, which makes it possible to perform Bayesian analysis in an automatic, streamlined way, and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged. Copyright (c) 2009 Royal Statistical Society.

Suggested Citation

  • Håvard Rue & Sara Martino & Nicolas Chopin, 2009. "Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(2), pages 319-392.
  • Handle: RePEc:bla:jorssb:v:71:y:2009:i:2:p:319-392
    as

    Download full text from publisher

    File URL: http://www.blackwell-synergy.com/doi/abs/10.1111/j.1467-9868.2008.00700.x
    File Function: link to full text
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521785167, March.
    2. J. Durbin & S. J. Koopman, 2000. "Time series analysis of non-Gaussian observations based on state space models from both classical and Bayesian perspectives," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 62(1), pages 3-56.
    3. Thomas Kneib & Ludwig Fahrmeir, 2006. "Structured Additive Regression for Categorical Space–Time Data: A Mixed Model Approach," Biometrics, The International Biometric Society, vol. 62(1), pages 109-118, March.
    4. Alexandros Beskos & Omiros Papaspiliopoulos & Gareth O. Roberts & Paul Fearnhead, 2006. "Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion)," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(3), pages 333-382.
    5. Ludwig Fahrmeir & Stefan Lang, 2001. "Bayesian inference for generalized additive mixed models based on Markov random field priors," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 50(2), pages 201-220.
    6. Jo Eidsvik & Sara Martino & Håvard Rue, 2009. "Approximate Bayesian Inference in Spatial Generalized Linear Mixed Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 36(1), pages 1-22.
    7. I. S. Weir & A. N. Pettitt, 2000. "Binary probability maps using a hidden conditional autoregressive Gaussian process with an application to Finnish common toad data," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 49(4), pages 473-484.
    8. Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," Review of Economic Studies, Oxford University Press, vol. 65(3), pages 361-393.
    9. Reich, Brian J. & Hodges, James S. & Carlin, Bradley P., 2007. "Spatial Analyses of Periodontal Data Using Conditionally Autoregressive Priors Having Two Classes of Neighbor Relations," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 44-55, March.
    10. Noel Cressie & Gardar Johannesson, 2008. "Fixed rank kriging for very large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(1), pages 209-226.
    11. Anders Brix & Peter J. Diggle, 2001. "Spatiotemporal prediction for log-Gaussian Cox processes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(4), pages 823-841.
    12. Thomas Kneib & Ludwig Fahrmeir, 2007. "A Mixed Model Approach for Geoadditive Hazard Regression," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 34(1), pages 207-228.
    13. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    14. Sudipto Banerjee & Alan E. Gelfand & Andrew O. Finley & Huiyan Sang, 2008. "Gaussian predictive process models for large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(4), pages 825-848.
    15. Ole F. Christensen & Gareth O. Roberts & Jeffrey S. Rosenthal, 2005. "Scaling limits for the transient phase of local Metropolis-Hastings algorithms," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 253-268.
    16. Julian Besag & Jeremy York & Annie Mollié, 1991. "Bayesian image restoration, with two applications in spatial statistics," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 43(1), pages 1-20, March.
    17. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    18. Anders Brix, 2001. "Space-time Multi Type Log Gaussian Cox Processes with a View to Modelling Weeds," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 28(3), pages 471-488.
    19. Fuentes, Montserrat, 2007. "Approximate Likelihood for Large Irregularly Spaced Spatial Data," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 321-331, March.
    20. Harvey, Andrew C & Shephard, Neil, 1996. "Estimation of an Asymmetric Stochastic Volatility Model for Asset Returns," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(4), pages 429-434, October.
    21. McCAUSLAND, William, 2008. "The Hessian Method (Highly Efficient State Smoothing, In a Nutshell)," Cahiers de recherche 2008-03, Universite de Montreal, Departement de sciences economiques.
    22. Hannig, J. & Marron, J.S., 2006. "Advanced Distribution Theory for SiZer," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 484-499, June.
    23. Skaug, Hans J. & Fournier, David A., 2006. "Automatic approximation of the marginal likelihood in non-Gaussian hierarchical models," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 699-709, November.
    24. Youngjo Lee & John A. Nelder, 2006. "Double hierarchical generalized linear models (with discussion)," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 55(2), pages 139-185.
    25. Rasmus Plenge Waagepetersen, 2007. "An Estimating Function Approach to Inference for Inhomogeneous Neyman–Scott Processes," Biometrics, The International Biometric Society, vol. 63(1), pages 252-258, March.
    26. Chib, Siddhartha & Nardari, Federico & Shephard, Neil, 2002. "Markov chain Monte Carlo methods for stochastic volatility models," Journal of Econometrics, Elsevier, vol. 108(2), pages 281-316, June.
    27. Nicholas G. Polson & Jonathan R. Stroud & Peter Müller, 2008. "Practical filtering with sequential parameter learning," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(2), pages 413-428.
    28. Borus Jungbacker & Siem Jan Koopman, 2007. "Monte Carlo Estimation for Nonlinear Non-Gaussian State Space Models," Biometrika, Biometrika Trust, vol. 94(4), pages 827-839.
    29. David J. Spiegelhalter & Nicola G. Best & Bradley P. Carlin & Angelika van der Linde, 2002. "Bayesian measures of model complexity and fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 583-639.
    30. Leonhard Knorr-Held, 2002. "On Block Updating in Markov Random Field Models for Disease Mapping," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 29(4), pages 597-614.
    31. Jiannong Liu & James S. Hodges, 2003. "Posterior bimodality in the balanced one-way random-effects model," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(1), pages 247-255.
    32. Tilmann Gneiting & Larissa Stanberry & Eric Grimit & Leonhard Held & Nicholas Johnson, 2008. "Rejoinder on: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 17(2), pages 256-264, August.
    33. Leonhard Knorr-Held & Günter Raßer & Nikolaus Becker, 2002. "Disease Mapping of Stage-Specific Cancer Incidence Data," Biometrics, The International Biometric Society, vol. 58(3), pages 492-501, September.
    34. Ainsworth, L.M. & Dean, C.B., 2006. "Approximate inference for disease mapping," Computational Statistics & Data Analysis, Elsevier, vol. 50(10), pages 2552-2570, June.
    35. R. A. Rigby & D. M. Stasinopoulos, 2005. "Generalized additive models for location, scale and shape," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 54(3), pages 507-554.
    36. Michael L. Stein & Zhiyi Chi & Leah J. Welty, 2004. "Approximating likelihoods for large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(2), pages 275-296.
    37. Noh, Maengseok & Lee, Youngjo, 2007. "Robust Modeling for Inference From Generalized Linear Model Classes," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1059-1072, September.
    38. Ruppert,David & Wand,M. P. & Carroll,R. J., 2003. "Semiparametric Regression," Cambridge Books, Cambridge University Press, number 9780521780506, March.
    39. Noh, Maengseok & Lee, Youngjo, 2007. "REML estimation for binary data in GLMMs," Journal of Multivariate Analysis, Elsevier, vol. 98(5), pages 896-915, May.
    40. Sahu, Sujit K. & Gelfand, Alan E. & Holland, David M., 2007. "High-Resolution SpaceTime Ozone Modeling for Assessing Trends," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 1221-1234, December.
    41. A. Azzalini & A. Capitanio, 1999. "Statistical applications of the multivariate skew normal distribution," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(3), pages 579-602.
    42. A. Brezger & L. Fahrmeir & A. Hennerfeind, 2007. "Adaptive Gaussian Markov random fields with applications in human brain mapping," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 56(3), pages 327-345.
    43. Tilmann Gneiting & Larissa Stanberry & Eric Grimit & Leonhard Held & Nicholas Johnson, 2008. "Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 17(2), pages 211-235, August.
    44. Kauermann, Goran, 2005. "Penalized spline smoothing in multivariable survival models with varying coefficients," Computational Statistics & Data Analysis, Elsevier, vol. 49(1), pages 169-186, April.
    45. Leonhard Knorr-Held, 1999. "Conditional Prior Proposals in Dynamic Models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 26(1), pages 129-144.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:71:y:2009:i:2:p:319-392. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley-Blackwell Digital Licensing) or (Christopher F. Baum). General contact details of provider: http://edirc.repec.org/data/rssssea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.