IDEAS home Printed from https://ideas.repec.org/a/bla/istatr/v90y2022i3p525-550.html
   My bibliography  Save this article

Rethinking the Effective Sample Size

Author

Listed:
  • Víctor Elvira
  • Luca Martino
  • Christian P. Robert

Abstract

The effective sample size (ESS) is widely used in sample‐based simulation methods for assessing the quality of a Monte Carlo approximation of a given distribution and of related integrals. In this paper, we revisit the approximation of the ESS in the specific context of importance sampling. The derivation of this approximation, that we will denote as ESS^, is partially available in a 1992 foundational technical report of Augustine Kong. This approximation has been widely used in the last 25 years due to its simplicity as a practical rule of thumb in a wide variety of importance sampling methods. However, we show that the multiple assumptions and approximations in the derivation of ESS^ make it difficult to be considered even as a reasonable approximation of the ESS. We extend the discussion of the ESS^ in the multiple importance sampling setting, we display numerical examples and we discuss several avenues for developing alternative metrics. This paper does not cover the use of ESS for Markov chain Monte Carlo algorithms.

Suggested Citation

  • Víctor Elvira & Luca Martino & Christian P. Robert, 2022. "Rethinking the Effective Sample Size," International Statistical Review, International Statistical Institute, vol. 90(3), pages 525-550, December.
    • Handle: RePEc:bla:istatr:v:90:y:2022:i:3:p:525-550
    DOI: 10.1111/insr.12500
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/insr.12500
    Download Restriction: no
    File URL: https://libkey.io/10.1111/insr.12500?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. H. Kahn & A. W. Marshall, 1953. "Methods of Reducing Sample Size in Monte Carlo Computations," Operations Research, INFORMS, vol. 1(5), pages 263-278, November.
    2. repec:dau:papers:123456789/6072 is not listed on IDEAS
    3. Chen, Yuguo, 2005. "Another look at rejection sampling through importance sampling," Statistics & Probability Letters, Elsevier, vol. 72(4), pages 277-283, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hatem Çoban & İpek Deveci Kocakoç & Şemsettin Erken & Mehmet Akif Aksoy, 2019. "Reducing Variation of Risk Estimation by Using Importance Sampling," Alphanumeric Journal, Bahadir Fatih Yildirim, vol. 7(2), pages 173-184, December.
    2. Koopman, Siem Jan & Shephard, Neil & Creal, Drew, 2009. "Testing the assumptions behind importance sampling," Journal of Econometrics, Elsevier, vol. 149(1), pages 2-11, April.
    3. Rahimiyan, Morteza, 2014. "A statistical cognitive model to assess impact of spatially correlated wind production on market behaviors," Applied Energy, Elsevier, vol. 122(C), pages 62-72.
    4. Hopmans, A.C.M. & Kleijnen, J.P.C., 1978. "Importance sampling in systems simulation : A practical failure?," Research Memorandum FEW 73, Tilburg University, School of Economics and Management.
    5. Cao, Quoc Dung & Choe, Youngjun, 2019. "Cross-entropy based importance sampling for stochastic simulation models," Reliability Engineering and System Safety, Elsevier, vol. 191(C).
    6. Youngjun Choe & Henry Lam & Eunshin Byon, 2018. "Uncertainty Quantification of Stochastic Simulation for Black-box Computer Experiments," Methodology and Computing in Applied Probability, Springer, vol. 20(4), pages 1155-1172, December.
    7. Rossi, Roberto & Chen, Zhen & Tarim, S. Armagan, 2024. "On the stochastic inventory problem under order capacity constraints," European Journal of Operational Research, Elsevier, vol. 312(2), pages 541-555.
    8. Geweke, John, 1996. "Monte carlo simulation and numerical integration," Handbook of Computational Economics, in: H. M. Amman & D. A. Kendrick & J. Rust (ed.), Handbook of Computational Economics, edition 1, volume 1, chapter 15, pages 731-800, Elsevier.
    9. Tito Homem-de-Mello, 2007. "A Study on the Cross-Entropy Method for Rare-Event Probability Estimation," INFORMS Journal on Computing, INFORMS, vol. 19(3), pages 381-394, August.
    10. Mie Augier & James G. March & Andrew W. Marshall, 2015. "Perspective—The Flaring of Intellectual Outliers: An Organizational Interpretation of the Generation of Novelty in the RAND Corporation," Organization Science, INFORMS, vol. 26(4), pages 1140-1161, August.
    11. Hopmans, A.C.M. & Kleijnen, J.P.C., 1979. "Importance sampling in systems simulation: a practical failure?," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 21(2), pages 209-220.
    12. Agnieszka Borowska & Lennart Hoogerheide & Siem Jan Koopman, 2019. "Bayesian Risk Forecasting for Long Horizons," Tinbergen Institute Discussion Papers 19-018/III, Tinbergen Institute.
    13. Bodendorf, Frank & Xie, Qiao & Merkl, Philipp & Franke, Jörg, 2022. "A multi-perspective approach to support collaborative cost management in supplier-buyer dyads," International Journal of Production Economics, Elsevier, vol. 245(C).
    14. Jean-Francois Richard, 2007. "Efficient High-Dimensional Importance Sampling," Working Paper 321, Department of Economics, University of Pittsburgh, revised Jan 2007.
    15. Geweke, John, 2001. "Bayesian econometrics and forecasting," Journal of Econometrics, Elsevier, vol. 100(1), pages 11-15, January.
    16. Li, Yong & Zhang, Mingzhi & Zhang, Yonghui, 2022. "Sequential Bayesian bandwidth selection for multivariate kernel regression with applications," Economic Modelling, Elsevier, vol. 112(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:istatr:v:90:y:2022:i:3:p:525-550. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/isiiinl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.