Advanced Search
MyIDEAS: Login

A Class of Adaptive EM-based Importance Sampling Algorithms for Efficient and Robust Posterior and Predictive Simulation

Contents:

Author Info

  • Lennart Hoogerheide

    (Erasmus University Rotterdam)

  • Anne Opschoor

    (Erasmus University Rotterdam)

  • Herman K. van Dijk

    (Erasmus University Rotterdam)

Abstract

A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student- t densities that approximates accurately the target distribution -typically a posterior distribution, of which we only require a kernel - in the sense that the Kullback-Leibler divergence between target and mixture is minimized. We label this approach Mixture of t by Importance Sampling and Expectation Maximization (MitISEM). We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner, so that the candidate distribution for posterior simulation is cleverly updated when new data become available. Our results show that the computational effort reduces enormously. This sequential approach can be combined with a tempering approach, which facilitates the simulation from densities with multiple modes that are far apart. Second, we introduce a permutation-augmented MitISEM approach, for importance sampling from posterior distributions in mixture models without the requirement of imposing identification restrictions on the model's mixture regimes' parameters. Third, we propose a partial MitISEM approach, which aims at approximating the marginal and conditional posterior distributions of subsets of model parameters, rather than the joint. This division can substantially reduce the dimension of the approximation problem.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL: http://papers.tinbergen.nl/11004.pdf
Download Restriction: no

Bibliographic Info

Paper provided by Tinbergen Institute in its series Tinbergen Institute Discussion Papers with number 11-004/4.

as in new window
Length:
Date of creation: 06 Jan 2011
Date of revision:
Handle: RePEc:dgr:uvatin:20110004

Contact details of provider:
Web page: http://www.tinbergen.nl

Related research

Keywords: mixture of Student-t distributions; importance sampling; Kullback-Leibler divergence; Expectation Maximization; Metropolis-Hastings algorithm; predictive likelihoods; mixture GARCH models; Value at Risk;

Find related papers by JEL classification:

This paper has been announced in the following NEP Reports:

References

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.
  2. HOOGERHEIDE, Lennart F. & VAN DIJK, Herman K. & VAN OEST, Rutger D., 2007. "Simulation based Bayesian econometric inference: principles and some recent computational advances," CORE Discussion Papers 2007015, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
  3. María Concepcion Ausin & Pedro Galeano, 2005. "Bayesian Estimation Of The Gaussian Mixture Garch Model," Statistics and Econometrics Working Papers ws053605, Universidad Carlos III, Departamento de Estadística y Econometría.
  4. Tim Bollerslev, 1986. "Generalized autoregressive conditional heteroskedasticity," EERI Research Paper Series EERI RP 1986/01, Economics and Econometrics Research Institute (EERI), Brussels.
  5. Eklund, Jana & Karlsson, Sune, 2005. "Forecast Combination and Model Averaging using Predictive Measures," Working Paper Series 191, Sveriges Riksbank (Central Bank of Sweden).
  6. Luc Bauwens & Michel Lubrano, 1998. "Bayesian inference on GARCH models using the Gibbs sampler," Econometrics Journal, Royal Economic Society, vol. 1(Conferenc), pages C23-C46.
  7. Lennart Hoogerheide & Herman K. van Dijk, 2008. "Bayesian Forecasting of Value at Risk and Expected Shortfall using Adaptive Importance Sampling," Tinbergen Institute Discussion Papers 08-092/4, Tinbergen Institute.
  8. van Dijk, H. K. & Kloek, T., 1980. "Further experience in Bayesian analysis using Monte Carlo integration," Journal of Econometrics, Elsevier, vol. 14(3), pages 307-328, December.
  9. Kloek, Tuen & van Dijk, Herman K, 1978. "Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo," Econometrica, Econometric Society, vol. 46(1), pages 1-19, January.
Full references (including those not matched with items on IDEAS)

Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Koop, Gary & Leon-Gonzalez, Roberto & Strachan, Rodney, 2011. "Bayesian Model Averaging in the Instrumental Variable Regression Model," SIRE Discussion Papers 2011-23, Scottish Institute for Research in Economics (SIRE).
  2. Hoogerheide, Lennart & Opschoor, Anne & van Dijk, Herman K., 2012. "A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation," Journal of Econometrics, Elsevier, vol. 171(2), pages 101-120.
  3. Hoogerheide, Lennart & Block, Joern H. & Thurik, Roy, 2012. "Family background variables as instruments for education in income regressions: A Bayesian analysis," Economics of Education Review, Elsevier, vol. 31(5), pages 515-523.
  4. Lennart F. Hoogerheide & Francesco Ravazzolo & Herman K. van Dijk, 2011. "Backtesting Value-at-Risk using Forecasts for Multiple Horizons, a Comment on the Forecast Rationality Tests of A.J. Patton and A. Timmermann," Tinbergen Institute Discussion Papers 11-131/4, Tinbergen Institute.
  5. Arnold Zellner & Tomohiro Ando & Nalan Basturk & Lennart Hoogerheide & Herman K. van Dijk, 2011. "Instrumental Variables, Errors in Variables, and Simultaneous Equations Models: Applicability and Limitations of Direct Monte Carlo," Tinbergen Institute Discussion Papers 11-137/4, Tinbergen Institute.

Lists

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Statistics

Access and download statistics

Corrections

When requesting a correction, please mention this item's handle: RePEc:dgr:uvatin:20110004. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Antoine Maartens (+31 626 - 160 892)).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.