IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2002.09578.html
   My bibliography  Save this paper

Scoring Functions for Multivariate Distributions and Level Sets

Author

Listed:
  • Xiaochun Meng
  • James W. Taylor
  • Souhaib Ben Taieb
  • Siran Li

Abstract

Interest in predicting multivariate probability distributions is growing due to the increasing availability of rich datasets and computational developments. Scoring functions enable the comparison of forecast accuracy, and can potentially be used for estimation. A scoring function for multivariate distributions that has gained some popularity is the energy score. This is a generalization of the continuous ranked probability score (CRPS), which is widely used for univariate distributions. A little-known, alternative generalization is the multivariate CRPS (MCRPS). We propose a theoretical framework for scoring functions for multivariate distributions, which encompasses the energy score and MCRPS, as well as the quadratic score, which has also received little attention. We demonstrate how this framework can be used to generate new scores. For univariate distributions, it is well-established that the CRPS can be expressed as the integral over a quantile score. We show that, in a similar way, scoring functions for multivariate distributions can be "disintegrated" to obtain scoring functions for level sets. Using this, we present scoring functions for different types of level set, including those for densities and cumulative distributions. To compute the scoring functions, we propose a simple numerical algorithm. We illustrate our proposals using simulated and stock returns data.

Suggested Citation

  • Xiaochun Meng & James W. Taylor & Souhaib Ben Taieb & Siran Li, 2020. "Scoring Functions for Multivariate Distributions and Level Sets," Papers 2002.09578, arXiv.org, revised Jun 2020.
  • Handle: RePEc:arx:papers:2002.09578
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2002.09578
    File Function: Latest version
    Download Restriction: no

    References listed on IDEAS

    as
    1. Komunjer, Ivana, 2005. "Quasi-maximum likelihood estimation for conditional quantiles," Journal of Econometrics, Elsevier, vol. 128(1), pages 137-164, September.
    2. Diks, Cees & Panchenko, Valentyn & Sokolinskiy, Oleg & van Dijk, Dick, 2014. "Comparing the accuracy of multivariate density forecasts in selected regions of the copula support," Journal of Economic Dynamics and Control, Elsevier, vol. 48(C), pages 79-94.
    3. Yen-Chi Chen & Christopher R. Genovese & Larry Wasserman, 2017. "Density Level Sets: Asymptotics, Inference, and Visualization," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1684-1696, October.
    4. Natalia Nolde & Johanna F. Ziegel, 2016. "Elicitability and backtesting: Perspectives for banking regulation," Papers 1608.05498, arXiv.org, revised Feb 2017.
    5. Anthonisz, Sean A., 2012. "Asset pricing with partial-moments," Journal of Banking & Finance, Elsevier, vol. 36(7), pages 2122-2135.
    6. Jooyoung Jeon & James W. Taylor, 2012. "Using Conditional Kernel Density Estimation for Wind Power Density Forecasting," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(497), pages 66-79, March.
    7. Elena Di Bernardino & Thomas Laloë & Véronique Maume-Deschamps & Clémentine Prieur, 2013. "Plug-in estimation of level sets in a non-compact setting with applications in multivariate risk theory," Post-Print hal-00580624, HAL.
    8. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    9. Marc Hallin & Davy Paindaveine & Miroslav Siman, 2008. "Multivariate quantiles and multiple-output regression quantiles: from L1 optimization to halfspace depth," Working Papers ECARES 2008_042, ULB -- Universite Libre de Bruxelles.
    10. Ali E. Abbas & David V. Budescu & Yuhong (Rola) Gu, 2010. "Assessing Joint Distributions with Isoprobability Contours," Management Science, INFORMS, vol. 56(6), pages 997-1011, June.
    11. Koenker, Roger W & Bassett, Gilbert, Jr, 1978. "Regression Quantiles," Econometrica, Econometric Society, vol. 46(1), pages 33-50, January.
    12. Cousin, Areski & Di Bernardino, Elena, 2013. "On multivariate extensions of Value-at-Risk," Journal of Multivariate Analysis, Elsevier, vol. 119(C), pages 32-46.
    13. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    14. Gneiting, Tilmann & Ranjan, Roopesh, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(3), pages 411-422.
    15. Diks, Cees & Panchenko, Valentyn & van Dijk, Dick, 2011. "Likelihood-based scoring rules for comparing density forecasts in tails," Journal of Econometrics, Elsevier, vol. 163(2), pages 215-230, August.
    16. Patton, Andrew J., 2012. "A review of copula models for economic time series," Journal of Multivariate Analysis, Elsevier, vol. 110(C), pages 4-18.
    17. Cadre, BenoI^t, 2006. "Kernel estimation of density level sets," Journal of Multivariate Analysis, Elsevier, vol. 97(4), pages 999-1023, April.
    18. Briec, Walter & Kerstens, Kristiaan, 2010. "Portfolio selection in multidimensional general and partial moment space," Journal of Economic Dynamics and Control, Elsevier, vol. 34(4), pages 636-656, April.
    19. Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
    20. M.‐Y. Cheng & P. Hall, 1998. "Calibrating the excess mass and dip tests of modality," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 60(3), pages 579-589.
    21. Tilmann Gneiting & Roopesh Ranjan, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(3), pages 411-422, July.
    22. Price, Kelly & Price, Barbara & Nantell, Timothy J, 1982. "Variance and Lower Partial Moment Measures of Systematic Risk: Some Analytical and Empirical Results," Journal of Finance, American Finance Association, vol. 37(3), pages 843-855, June.
    23. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    24. Areski Cousin & Elena Di Bernadino, 2013. "On Multivariate Extensions of Value-at-Risk," Working Papers hal-00638382, HAL.
    25. Areski Cousin & Elena Di Bernadino, 2011. "On Multivariate Extensions of Value-at-Risk," Papers 1111.1349, arXiv.org, revised Apr 2013.
    26. repec:hal:journl:peer-00834423 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2002.09578. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (arXiv administrators). General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.