IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Login to save this article or follow this journal

A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation

  • Hoogerheide, Lennart
  • Opschoor, Anne
  • van Dijk, Herman K.

A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution–typically a posterior distribution, of which we only require a kernel–in the sense that the Kullback–Leibler divergence between target and mixture is minimized. We label this approach Mixture oftby Importance Sampling weighted Expectation Maximization (MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis–Hastings (MH) method. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner, so that the candidate distribution for posterior simulation is cleverly updated when new data become available. Our results show that the computational effort reduces enormously, while the quality of the approximation remains almost unchanged. This sequential approach can be combined with a tempering approach, which facilitates the simulation from densities with multiple modes that are far apart. Second, we introduce a permutation-augmented MitISEM approach. This is useful for importance or Metropolis–Hastings sampling from posterior distributions in mixture models without the requirement of imposing identification restrictions on the model’s mixture regimes’ parameters. Third, we propose a partial MitISEM approach, which aims at approximating the joint distribution by estimating a product of marginal and conditional distributions. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like DCC or mixture GARCH models and a mixture instrumental variables model.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.sciencedirect.com/science/article/pii/S0304407612001583
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Article provided by Elsevier in its journal Journal of Econometrics.

Volume (Year): 171 (2012)
Issue (Month): 2 ()
Pages: 101-120

as
in new window

Handle: RePEc:eee:econom:v:171:y:2012:i:2:p:101-120
Contact details of provider: Web page: http://www.elsevier.com/locate/jeconom

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Frühwirth-Schnatter, Sylvia & Wagner, Helga, 2008. "Marginal likelihoods for non-Gaussian models using auxiliary mixture sampling," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4608-4624, June.
  2. Hoogerheide, Lennart & van Dijk, Herman K., 2010. "Bayesian forecasting of Value at Risk and Expected Shortfall using adaptive importance sampling," International Journal of Forecasting, Elsevier, vol. 26(2), pages 231-247, April.
  3. Joshua Angrist & Alan Krueger, 1990. "Does Compulsory School Attendance Affect Schooling and Earnings?," Working Papers 653, Princeton University, Department of Economics, Industrial Relations Section..
  4. Kloek, Tuen & van Dijk, Herman K, 1978. "Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo," Econometrica, Econometric Society, vol. 46(1), pages 1-19, January.
  5. Eklund, Jana & Karlsson, Sune, 2005. "Forecast Combination and Model Averaging using Predictive Measures," Working Paper Series 191, Sveriges Riksbank (Central Bank of Sweden).
  6. Geweke, John, 2007. "Interpretation and inference in mixture models: Simple MCMC works," Computational Statistics & Data Analysis, Elsevier, vol. 51(7), pages 3529-3550, April.
  7. Geweke, John, 1989. "Bayesian Inference in Econometric Models Using Monte Carlo Integration," Econometrica, Econometric Society, vol. 57(6), pages 1317-39, November.
  8. Fruhwirth-Schnatter S., 2001. "Markov Chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 194-209, March.
  9. Nicolas Chopin, 2000. "A Sequential Particle Filter Method for Static Models," Working Papers 2000-45, Centre de Recherche en Economie et Statistique.
  10. HOOGERHEIDE, Lennart F. & KAASHOEK, Johan F. & VAN DIJK, Herman K., 2005. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," CORE Discussion Papers 2005029, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
  11. Nelson, Daniel B, 1991. "Conditional Heteroskedasticity in Asset Returns: A New Approach," Econometrica, Econometric Society, vol. 59(2), pages 347-70, March.
  12. Lennart Hoogerheide & Anne Opschoor & Herman K. van Dijk, 2011. "A Class of Adaptive EM-based Importance Sampling Algorithms for Efficient and Robust Posterior and Predictive Simulation," Tinbergen Institute Discussion Papers 11-004/4, Tinbergen Institute.
  13. Engle, Robert, 2002. "Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 339-50, July.
  14. van Dijk, H. K. & Kloek, T., 1980. "Further experience in Bayesian analysis using Monte Carlo integration," Journal of Econometrics, Elsevier, vol. 14(3), pages 307-328, December.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:171:y:2012:i:2:p:101-120. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.