IDEAS home Printed from
MyIDEAS: Login to save this paper or follow this series

Marginal Likelihood Estimation with the Cross-Entropy Method

  • Chan, Joshua
  • Eisenstat, Eric

We consider an adaptive importance sampling approach to estimating the marginal likelihood, a quantity that is fundamental in Bayesian model comparison and Bayesian model averaging. This approach is motivated by the difficulty of obtaining an accurate estimate through existing algorithms that use Markov chain Monte Carlo (MCMC) draws, where the draws are typically costly to obtain and highly correlated in high-dimensional settings. In contrast, we use the cross-entropy (CE) method, a versatile adaptive Monte Carlo algorithm originally developed for rare-event simulation. The main advantage of the importance sampling approach is that random samples can be obtained from some convenient density with little additional costs. As we are generating independent draws instead of correlated MCMC draws, the increase in simulation effort is much smaller should one wish to reduce the numerical standard error of the estimator. Moreover, the importance density derived via the CE method is in a well-defined sense optimal. We demonstrate the utility of the proposed approach by two empirical applications involving women's labor market participation and U.S. macroeconomic time series. In both applications the proposed CE method compares favorably to existing estimators.

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL:
File Function: original version
Download Restriction: no

Paper provided by University Library of Munich, Germany in its series MPRA Paper with number 40051.

in new window

Date of creation: 2012
Date of revision:
Handle: RePEc:pra:mprapa:40051
Contact details of provider: Postal: Schackstr. 4, D-80539 Munich, Germany
Phone: +49-(0)89-2180-2219
Fax: +49-(0)89-2180-3900
Web page:

More information through EDIRC

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as in new window
  1. Thomas Mroz, . "The Sensitivity of an Empirical Model of Married Women's Hours of Work to Economic and Statistical Assumptions," University of Chicago - Population Research Center 84-8, Chicago - Population Research Center.
  2. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
  3. HOOGERHEIDE, Lennart F. & KAASHOEK, Johan F. & VAN DIJK, Herman K., 2005. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," CORE Discussion Papers 2005029, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
  4. Kim, Sangjoon & Shephard, Neil & Chib, Siddhartha, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," Review of Economic Studies, Wiley Blackwell, vol. 65(3), pages 361-93, July.
  5. N. Friel & A. N. Pettitt, 2008. "Marginal likelihood estimation via power posteriors," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(3), pages 589-607.
  6. Ben S. Bernanke & Jean Boivin & Piotr Eliasz, 2004. "Measuring the Effects of Monetary Policy: A Factor-Augmented Vector Autoregressive (FAVAR) Approach," NBER Working Papers 10220, National Bureau of Economic Research, Inc.
  7. Koop, Gary & Korobilis, Dimitris, 2009. "Bayesian Multivariate Time Series Methods for Empirical Macroeconomics," MPRA Paper 20125, University Library of Munich, Germany.
  8. David Ardia & Nalan Basturk & Lennart Hoogerheide & Herman K. van Dijk, 2010. "A Comparative Study of Monte Carlo Methods for Efficient Evaluation of Marginal Likelihood," Tinbergen Institute Discussion Papers 10-059/4, Tinbergen Institute.
  9. Golan, Amos & Judge, George G. & Miller, Douglas, 1996. "Maximum Entropy Econometrics," Staff General Research Papers 1488, Iowa State University, Department of Economics.
  10. John Geweke, 1999. "Using simulation methods for bayesian econometric models: inference, development,and communication," Econometric Reviews, Taylor & Francis Journals, vol. 18(1), pages 1-73.
  11. Ivan Jeliazkov & Rui Liu, 2010. "A model-based ranking of U.S. recessions," Economics Bulletin, AccessEcon, vol. 30(3), pages 2289-2296.
  12. P. T. de Boer & D. P. Kroese & R. Y. Rubinstein, 2004. "A Fast Cross-Entropy Method for Estimating Buffer Overflows in Queueing Networks," Management Science, INFORMS, vol. 50(7), pages 883-895, July.
  13. Geweke, John, 1989. "Exact predictive densities for linear models with arch disturbances," Journal of Econometrics, Elsevier, vol. 40(1), pages 63-86, January.
  14. Rubinstein, Reuven Y., 1997. "Optimization of computer simulation models with rare events," European Journal of Operational Research, Elsevier, vol. 99(1), pages 89-112, May.
  15. Chan, Joshua C.C. & Kroese, Dirk P., 2010. "Efficient estimation of large portfolio loss probabilities in t-copula models," European Journal of Operational Research, Elsevier, vol. 205(2), pages 361-367, September.
  16. Frühwirth-Schnatter, Sylvia & Wagner, Helga, 2008. "Marginal likelihoods for non-Gaussian models using auxiliary mixture sampling," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4608-4624, June.
  17. Golan, Amos, 2007. "Information and entropy econometrics - volume overview and synthesis," Journal of Econometrics, Elsevier, vol. 138(2), pages 379-387, June.
  18. Chib S. & Jeliazkov I., 2001. "Marginal Likelihood From the Metropolis-Hastings Output," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 270-281, March.
Full references (including those not matched with items on IDEAS)

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:40051. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Ekkehart Schlicht)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.