IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/40051.html
   My bibliography  Save this paper

Marginal Likelihood Estimation with the Cross-Entropy Method

Author

Listed:
  • Chan, Joshua
  • Eisenstat, Eric

Abstract

We consider an adaptive importance sampling approach to estimating the marginal likelihood, a quantity that is fundamental in Bayesian model comparison and Bayesian model averaging. This approach is motivated by the difficulty of obtaining an accurate estimate through existing algorithms that use Markov chain Monte Carlo (MCMC) draws, where the draws are typically costly to obtain and highly correlated in high-dimensional settings. In contrast, we use the cross-entropy (CE) method, a versatile adaptive Monte Carlo algorithm originally developed for rare-event simulation. The main advantage of the importance sampling approach is that random samples can be obtained from some convenient density with little additional costs. As we are generating independent draws instead of correlated MCMC draws, the increase in simulation effort is much smaller should one wish to reduce the numerical standard error of the estimator. Moreover, the importance density derived via the CE method is in a well-defined sense optimal. We demonstrate the utility of the proposed approach by two empirical applications involving women's labor market participation and U.S. macroeconomic time series. In both applications the proposed CE method compares favorably to existing estimators.

Suggested Citation

  • Chan, Joshua & Eisenstat, Eric, 2012. "Marginal Likelihood Estimation with the Cross-Entropy Method," MPRA Paper 40051, University Library of Munich, Germany.
  • Handle: RePEc:pra:mprapa:40051
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/40051/1/MPRA_paper_40051.pdf
    File Function: original version
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.
    2. Ivan Jeliazkov & Rui Liu, 2010. "A model-based ranking of U.S. recessions," Economics Bulletin, AccessEcon, vol. 30(3), pages 2289-2296.
    3. Rubinstein, Reuven Y., 1997. "Optimization of computer simulation models with rare events," European Journal of Operational Research, Elsevier, vol. 99(1), pages 89-112, May.
    4. Koop, Gary & Korobilis, Dimitris, 2010. "Bayesian Multivariate Time Series Methods for Empirical Macroeconomics," Foundations and Trends(R) in Econometrics, now publishers, vol. 3(4), pages 267-358, July.
    5. John Geweke, 1999. "Using simulation methods for bayesian econometric models: inference, development,and communication," Econometric Reviews, Taylor & Francis Journals, vol. 18(1), pages 1-73.
    6. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    7. Geweke, John, 1989. "Exact predictive densities for linear models with arch disturbances," Journal of Econometrics, Elsevier, vol. 40(1), pages 63-86, January.
    8. Golan, Amos & Judge, George G. & Miller, Douglas, 1996. "Maximum Entropy Econometrics," Staff General Research Papers Archive 1488, Iowa State University, Department of Economics.
    9. Ardia, David & Baştürk, Nalan & Hoogerheide, Lennart & van Dijk, Herman K., 2012. "A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3398-3414.
    10. P. T. de Boer & D. P. Kroese & R. Y. Rubinstein, 2004. "A Fast Cross-Entropy Method for Estimating Buffer Overflows in Queueing Networks," Management Science, INFORMS, vol. 50(7), pages 883-895, July.
    11. Ben S. Bernanke & Jean Boivin & Piotr Eliasz, 2005. "Measuring the Effects of Monetary Policy: A Factor-Augmented Vector Autoregressive (FAVAR) Approach," The Quarterly Journal of Economics, Oxford University Press, vol. 120(1), pages 387-422.
    12. Chan, Joshua C.C. & Kroese, Dirk P., 2010. "Efficient estimation of large portfolio loss probabilities in t-copula models," European Journal of Operational Research, Elsevier, vol. 205(2), pages 361-367, September.
    13. Chib S. & Jeliazkov I., 2001. "Marginal Likelihood From the Metropolis-Hastings Output," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 270-281, March.
    14. Golan, Amos, 2007. "Information and entropy econometrics - volume overview and synthesis," Journal of Econometrics, Elsevier, vol. 138(2), pages 379-387, June.
    15. Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," Review of Economic Studies, Oxford University Press, vol. 65(3), pages 361-393.
    16. N. Friel & A. N. Pettitt, 2008. "Marginal likelihood estimation via power posteriors," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(3), pages 589-607.
    17. Frühwirth-Schnatter, Sylvia & Wagner, Helga, 2008. "Marginal likelihoods for non-Gaussian models using auxiliary mixture sampling," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4608-4624, June.
    18. Mroz, Thomas A, 1987. "The Sensitivity of an Empirical Model of Married Women's Hours of Work to Economic and Statistical Assumptions," Econometrica, Econometric Society, vol. 55(4), pages 765-799, July.
    19. K.-P. Hui & N. Bean & M. Kraetzl & Dirk Kroese, 2005. "The Cross-Entropy Method for Network Reliability Estimation," Annals of Operations Research, Springer, vol. 134(1), pages 101-118, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Grant, Angelia L. & Chan, Joshua C.C., 2017. "Reconciling output gaps: Unobserved components model and Hodrick–Prescott filter," Journal of Economic Dynamics and Control, Elsevier, vol. 75(C), pages 114-121.
    2. Cross, Jamie & Nguyen, Bao H., 2017. "The relationship between global oil price shocks and China's output: A time-varying analysis," Energy Economics, Elsevier, vol. 62(C), pages 79-91.
    3. repec:wly:japmet:v:32:y:2017:i:1:p:103-119 is not listed on IDEAS
    4. Anders Warne & Günter Coenen & Kai Christoffel, 2017. "Marginalized Predictive Likelihood Comparisons of Linear Gaussian State‐Space Models with Applications to DSGE, DSGE‐VAR, and VAR Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(1), pages 103-119, January.
    5. Karlsson, Sune & Österholm, Pär, 2018. "Is the US Phillips Curve Stable? Evidence from Bayesian VARs," Working Papers 2018:5, Örebro University, School of Business.
    6. repec:wly:jmoncb:v:49:y:2017:i:2-3:p:525-552 is not listed on IDEAS
    7. Chan, Joshua C.C. & Grant, Angelia L., 2016. "Modeling energy price dynamics: GARCH versus stochastic volatility," Energy Economics, Elsevier, vol. 54(C), pages 182-189.
    8. Chan, Joshua C.C. & Grant, Angelia L., 2016. "Fast computation of the deviance information criterion for latent variable models," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 847-859.
    9. repec:wly:japmet:v:32:y:2017:i:1:p:224-232 is not listed on IDEAS
    10. Angelia L. Grant & Joshua C.C. Chan, 2017. "A Bayesian Model Comparison for Trend‐Cycle Decompositions of Output," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 49(2-3), pages 525-552, March.
    11. Joshua C.C. Chan, 2015. "Specification tests for time-varying parameter models with stochastic volatility," CAMA Working Papers 2015-42, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    12. repec:eee:reveco:v:51:y:2017:i:c:p:36-59 is not listed on IDEAS
    13. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2013. "Predictive likelihood comparisons with DSGE and DSGE-VAR models," Working Paper Series 1536, European Central Bank.
    14. repec:taf:emetrv:v:37:y:2018:i:8:p:807-823 is not listed on IDEAS
    15. Joshua C.C. Chan & Eric Eisenstat, 2015. "Bayesian model comparison for time-varying parameter VARs with stochastic volatility," CAMA Working Papers 2015-32, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.

    More about this item

    Keywords

    importance sampling; model selection; probit; logit; time-varying parameter vector autoregressive model; dynamic factor model;

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:40051. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Joachim Winter) or (Rebekah McClure). General contact details of provider: http://edirc.repec.org/data/vfmunde.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.