A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation
A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution–typically a posterior distribution, of which we only require a kernel–in the sense that the Kullback–Leibler divergence between target and mixture is minimized. We label this approach Mixture oftby Importance Sampling weighted Expectation Maximization (MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis–Hastings (MH) method. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner, so that the candidate distribution for posterior simulation is cleverly updated when new data become available. Our results show that the computational effort reduces enormously, while the quality of the approximation remains almost unchanged. This sequential approach can be combined with a tempering approach, which facilitates the simulation from densities with multiple modes that are far apart. Second, we introduce a permutation-augmented MitISEM approach. This is useful for importance or Metropolis–Hastings sampling from posterior distributions in mixture models without the requirement of imposing identification restrictions on the model’s mixture regimes’ parameters. Third, we propose a partial MitISEM approach, which aims at approximating the joint distribution by estimating a product of marginal and conditional distributions. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like DCC or mixture GARCH models and a mixture instrumental variables model.
If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Geweke, John, 1989. "Bayesian Inference in Econometric Models Using Monte Carlo Integration," Econometrica, Econometric Society, vol. 57(6), pages 1317-39, November.
- van Dijk, H. K. & Kloek, T., 1980. "Further experience in Bayesian analysis using Monte Carlo integration," Journal of Econometrics, Elsevier, vol. 14(3), pages 307-328, December.
- Nelson, Daniel B, 1991. "Conditional Heteroskedasticity in Asset Returns: A New Approach," Econometrica, Econometric Society, vol. 59(2), pages 347-70, March.
- Fruhwirth-Schnatter S., 2001. "Markov Chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 194-209, March.
- Angrist, Joshua D & Krueger, Alan B, 1991.
"Does Compulsory School Attendance Affect Schooling and Earnings?,"
The Quarterly Journal of Economics,
MIT Press, vol. 106(4), pages 979-1014, November.
- Joshua Angrist & Alan Krueger, 1990. "Does Compulsory School Attendance Affect Schooling and Earnings?," Working Papers 653, Princeton University, Department of Economics, Industrial Relations Section..
- Joshua D. Angrist & Alan B. Krueger, 1990. "Does Compulsory School Attendance Affect Schooling and Earnings?," NBER Working Papers 3572, National Bureau of Economic Research, Inc.
- Nicolas Chopin, 2000.
"A Sequential Particle Filter Method for Static Models,"
2000-45, Centre de Recherche en Economie et Statistique.
- Nicolas Chopin, 2002. "A sequential particle filter method for static models," Biometrika, Biometrika Trust, vol. 89(3), pages 539-552, August.
- Engle, Robert, 2002. "Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 339-50, July.
- Geweke, John, 2007. "Interpretation and inference in mixture models: Simple MCMC works," Computational Statistics & Data Analysis, Elsevier, vol. 51(7), pages 3529-3550, April.
- Kloek, Tuen & van Dijk, Herman K, 1978. "Bayesian Estimates of Equation System Parameters: An Application of Integration by Monte Carlo," Econometrica, Econometric Society, vol. 46(1), pages 1-19, January.
- HOOGERHEIDE, Lennart F. & KAASHOEK, Johan F. & VAN DIJK, Herman K., 2005.
"On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks,"
CORE Discussion Papers
2005029, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
- Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.
- Hoogerheide, L.F. & Kaashoek, J.F. & van Dijk, H.K., 2005. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: an application of flexible sampling methods using neural networks," Econometric Institute Research Papers EI 2005-12, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
- HOOGERHEIDE, Lennart F. & KAASHOEK, Johan F. & van DIJK, Herman K., . "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: an application of flexible sampling methods using neural networks," CORE Discussion Papers RP -1922, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
- Hoogerheide, Lennart & van Dijk, Herman K., 2010. "Bayesian forecasting of Value at Risk and Expected Shortfall using adaptive importance sampling," International Journal of Forecasting, Elsevier, vol. 26(2), pages 231-247, April.
- Frühwirth-Schnatter, Sylvia & Wagner, Helga, 2008. "Marginal likelihoods for non-Gaussian models using auxiliary mixture sampling," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4608-4624, June.
- Lennart Hoogerheide & Anne Opschoor & Herman K. van Dijk, 2011. "A Class of Adaptive EM-based Importance Sampling Algorithms for Efficient and Robust Posterior and Predictive Simulation," Tinbergen Institute Discussion Papers 11-004/4, Tinbergen Institute.
- Eklund, Jana & Karlsson, Sune, 2005.
"Forecast Combination and Model Averaging Using Predictive Measures,"
CEPR Discussion Papers
5268, C.E.P.R. Discussion Papers.
- Jana Eklund & Sune Karlsson, 2007. "Forecast Combination and Model Averaging Using Predictive Measures," Econometric Reviews, Taylor & Francis Journals, vol. 26(2-4), pages 329-363.
- Eklund, Jana & Karlsson, Sune, 2005. "Forecast Combination and Model Averaging using Predictive Measures," Working Paper Series 191, Sveriges Riksbank (Central Bank of Sweden).
When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:171:y:2012:i:2:p:101-120. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei)
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.