IDEAS home Printed from https://ideas.repec.org/a/taf/emetrv/v34y2015i3p256-285.html
   My bibliography  Save this article

Marginal Likelihood Estimation with the Cross-Entropy Method

Author

Listed:
  • Joshua C. C. Chan
  • Eric Eisenstat

Abstract

We consider an adaptive importance sampling approach to estimating the marginal likelihood, a quantity that is fundamental in Bayesian model comparison and Bayesian model averaging. This approach is motivated by the difficulty of obtaining an accurate estimate through existing algorithms that use Markov chain Monte Carlo (MCMC) draws, where the draws are typically costly to obtain and highly correlated in high-dimensional settings. In contrast, we use the cross-entropy (CE) method, a versatile adaptive Monte Carlo algorithm originally developed for rare-event simulation. The main advantage of the importance sampling approach is that random samples can be obtained from some convenient density with little additional costs. As we are generating independent draws instead of correlated MCMC draws, the increase in simulation effort is much smaller should one wish to reduce the numerical standard error of the estimator. Moreover, the importance density derived via the CE method is grounded in information theory, and therefore, is in a well-defined sense optimal. We demonstrate the utility of the proposed approach by two empirical applications involving women's labor market participation and U.S. macroeconomic time series. In both applications, the proposed CE method compares favorably to existing estimators.

Suggested Citation

  • Joshua C. C. Chan & Eric Eisenstat, 2015. "Marginal Likelihood Estimation with the Cross-Entropy Method," Econometric Reviews, Taylor & Francis Journals, vol. 34(3), pages 256-285, March.
  • Handle: RePEc:taf:emetrv:v:34:y:2015:i:3:p:256-285
    DOI: 10.1080/07474938.2014.944474
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/07474938.2014.944474
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hoogerheide, Lennart F. & Kaashoek, Johan F. & van Dijk, Herman K., 2007. "On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of flexible sampling methods using neural networks," Journal of Econometrics, Elsevier, vol. 139(1), pages 154-180, July.
    2. Sangjoon Kim & Neil Shephard & Siddhartha Chib, 1998. "Stochastic Volatility: Likelihood Inference and Comparison with ARCH Models," Review of Economic Studies, Oxford University Press, vol. 65(3), pages 361-393.
    3. Ardia, David & Baştürk, Nalan & Hoogerheide, Lennart & van Dijk, Herman K., 2012. "A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3398-3414.
    4. Golan, Amos, 2007. "Information and entropy econometrics - volume overview and synthesis," Journal of Econometrics, Elsevier, vol. 138(2), pages 379-387, June.
    5. Ivan Jeliazkov & Rui Liu, 2010. "A model-based ranking of U.S. recessions," Economics Bulletin, AccessEcon, vol. 30(3), pages 2289-2296.
    6. Rubinstein, Reuven Y., 1997. "Optimization of computer simulation models with rare events," European Journal of Operational Research, Elsevier, vol. 99(1), pages 89-112, May.
    7. Koop, Gary & Korobilis, Dimitris, 2010. "Bayesian Multivariate Time Series Methods for Empirical Macroeconomics," Foundations and Trends(R) in Econometrics, now publishers, vol. 3(4), pages 267-358, July.
    8. N. Friel & A. N. Pettitt, 2008. "Marginal likelihood estimation via power posteriors," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(3), pages 589-607, July.
    9. Frühwirth-Schnatter, Sylvia & Wagner, Helga, 2008. "Marginal likelihoods for non-Gaussian models using auxiliary mixture sampling," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4608-4624, June.
    10. Mroz, Thomas A, 1987. "The Sensitivity of an Empirical Model of Married Women's Hours of Work to Economic and Statistical Assumptions," Econometrica, Econometric Society, vol. 55(4), pages 765-799, July.
    11. Canova, Fabio, 1993. "Modelling and forecasting exchange rates with a Bayesian time-varying coefficient model," Journal of Economic Dynamics and Control, Elsevier, vol. 17(1-2), pages 233-261.
    12. John Geweke, 1999. "Using simulation methods for bayesian econometric models: inference, development,and communication," Econometric Reviews, Taylor & Francis Journals, vol. 18(1), pages 1-73.
    13. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    14. Geweke, John, 1989. "Exact predictive densities for linear models with arch disturbances," Journal of Econometrics, Elsevier, vol. 40(1), pages 63-86, January.
    15. Golan, Amos & Judge, George G. & Miller, Douglas, 1996. "Maximum Entropy Econometrics," Staff General Research Papers Archive 1488, Iowa State University, Department of Economics.
    16. P. T. de Boer & D. P. Kroese & R. Y. Rubinstein, 2004. "A Fast Cross-Entropy Method for Estimating Buffer Overflows in Queueing Networks," Management Science, INFORMS, vol. 50(7), pages 883-895, July.
    17. K.-P. Hui & N. Bean & M. Kraetzl & Dirk Kroese, 2005. "The Cross-Entropy Method for Network Reliability Estimation," Annals of Operations Research, Springer, vol. 134(1), pages 101-118, February.
    18. Ben S. Bernanke & Jean Boivin & Piotr Eliasz, 2005. "Measuring the Effects of Monetary Policy: A Factor-Augmented Vector Autoregressive (FAVAR) Approach," The Quarterly Journal of Economics, Oxford University Press, vol. 120(1), pages 387-422.
    19. Chan, Joshua C.C. & Kroese, Dirk P., 2010. "Efficient estimation of large portfolio loss probabilities in t-copula models," European Journal of Operational Research, Elsevier, vol. 205(2), pages 361-367, September.
    20. Chib S. & Jeliazkov I., 2001. "Marginal Likelihood From the Metropolis-Hastings Output," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 270-281, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Grant, Angelia L. & Chan, Joshua C.C., 2017. "Reconciling output gaps: Unobserved components model and Hodrick–Prescott filter," Journal of Economic Dynamics and Control, Elsevier, vol. 75(C), pages 114-121.
    2. Cross, Jamie & Nguyen, Bao H., 2017. "The relationship between global oil price shocks and China's output: A time-varying analysis," Energy Economics, Elsevier, vol. 62(C), pages 79-91.
    3. Anders Warne & Günter Coenen & Kai Christoffel, 2017. "Marginalized Predictive Likelihood Comparisons of Linear Gaussian State‐Space Models with Applications to DSGE, DSGE‐VAR, and VAR Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(1), pages 103-119, January.
    4. Töppel, Jannick & Tränkler, Timm, 2019. "Modeling energy efficiency insurances and energy performance contracts for a quantitative comparison of risk mitigation potential," Energy Economics, Elsevier, vol. 80(C), pages 842-859.
    5. Hajargasht, Gholamreza & Rao, D.S. Prasada, 2019. "Multilateral index number systems for international price comparisons: Properties, existence and uniqueness," Journal of Mathematical Economics, Elsevier, vol. 83(C), pages 36-47.
    6. Karlsson, Sune & Österholm, Pär, 2018. "Is the US Phillips Curve Stable? Evidence from Bayesian VARs," Working Papers 2018:5, Örebro University, School of Business.
    7. Angelia L. Grant & Joshua C.C. Chan, 2017. "A Bayesian Model Comparison for Trend‐Cycle Decompositions of Output," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 49(2-3), pages 525-552, March.
    8. Chan, Joshua C.C. & Grant, Angelia L., 2016. "Modeling energy price dynamics: GARCH versus stochastic volatility," Energy Economics, Elsevier, vol. 54(C), pages 182-189.
    9. Yong Jiang & Yi-Shuai Ren & Chao-Qun Ma & Jiang-Long Liu & Basil Sharp, 2018. "Does the price of strategic commodities respond to U.S. Partisan Conflict?," Papers 1810.08396, arXiv.org, revised Feb 2020.
    10. Liangliang Zhou & Yishao Shi & Xiangyang Cao, 2019. "Evaluation of Land Intensive Use in Shanghai Pilot Free Trade Zone," Land, MDPI, Open Access Journal, vol. 8(6), pages 1-16, May.
    11. Joshua C. C. Chan & Liana Jacobi & Dan Zhu, 2019. "An automated prior robustness analysis in Bayesian model comparison," CAMA Working Papers 2019-45, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    12. Lu Yang & Shigeyuki Hamori, 2018. "Modeling The Dynamics Of International Agricultural Commodity Prices: A Comparison Of Garch And Stochastic Volatility Models," Annals of Financial Economics (AFE), World Scientific Publishing Co. Pte. Ltd., vol. 13(03), pages 1-20, September.
    13. Tomáš Jeřábek & Radka Šperková, 2015. "A Predictive Likelihood Approach to Bayesian Averaging," Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis, Mendel University Press, vol. 63(4), pages 1269-1276.
    14. Chan, Joshua C.C. & Grant, Angelia L., 2016. "Fast computation of the deviance information criterion for latent variable models," Computational Statistics & Data Analysis, Elsevier, vol. 100(C), pages 847-859.
    15. Angelia L. Grant, 2017. "The Early Millennium Slowdown: Replicating the Peersman (2005) Results," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(1), pages 224-232, January.
    16. Chan, Joshua C.C. & Eisenstat, Eric, 2018. "Comparing hybrid time-varying parameter VARs," Economics Letters, Elsevier, vol. 171(C), pages 1-5.
    17. Joshua C. C. Chan, 2018. "Specification tests for time-varying parameter models with stochastic volatility," Econometric Reviews, Taylor & Francis Journals, vol. 37(8), pages 807-823, September.
    18. Bowen Fu, 2019. "Bubbles and crises: Replicating the Anundsen et al. (2016) results," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(5), pages 822-826, August.
    19. Joshua C. C. Chan & Eric Eisenstat, 2018. "Bayesian model comparison for time‐varying parameter VARs with stochastic volatility," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 33(4), pages 509-532, June.
    20. Afees A. Salisu & Ahamuefula Ephraim Ogbonna, 2018. "Does time-variation matter in the stochastic volatility components for G7 stock returns," Working Papers 062, Centre for Econometric and Allied Research, University of Ibadan.
    21. Sahar Abbas & Fahimeh Moosavi, 2012. "Finding shortest path in static networks: using a modified algorithm," International Journal of Finance & Banking Studies, Center for the Strategic Studies in Business and Finance, vol. 1(1), pages 29-34, January.
    22. Dima, Bogdan & Dima, Ştefana Maria, 2017. "Mutual information and persistence in the stochastic volatility of market returns: An emergent market example," International Review of Economics & Finance, Elsevier, vol. 51(C), pages 36-59.
    23. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2013. "Predictive likelihood comparisons with DSGE and DSGE-VAR models," Working Paper Series 1536, European Central Bank.

    More about this item

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:emetrv:v:34:y:2015:i:3:p:256-285. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://www.tandfonline.com/LECR20 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.