IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1705.04537.html
   My bibliography  Save this paper

Murphy Diagrams: Forecast Evaluation of Expected Shortfall

Author

Listed:
  • Johanna F. Ziegel
  • Fabian Kruger
  • Alexander Jordan
  • Fernando Fasciati

Abstract

Motivated by the Basel 3 regulations, recent studies have considered joint forecasts of Value-at-Risk and Expected Shortfall. A large family of scoring functions can be used to evaluate forecast performance in this context. However, little intuitive or empirical guidance is currently available, which renders the choice of scoring function awkward in practice. We therefore develop graphical checks (Murphy diagrams) of whether one forecast method dominates another under a relevant class of scoring functions, and propose an associated hypothesis test. We illustrate these tools with simulation examples and an empirical analysis of S&P 500 and DAX returns.

Suggested Citation

  • Johanna F. Ziegel & Fabian Kruger & Alexander Jordan & Fernando Fasciati, 2017. "Murphy Diagrams: Forecast Evaluation of Expected Shortfall," Papers 1705.04537, arXiv.org.
    • Handle: RePEc:arx:papers:1705.04537
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1705.04537
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    2. Neil Shephard & Kevin Sheppard, 2010. "Realising the future: forecasting with high-frequency-based volatility (HEAVY) models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(2), pages 197-231.
    3. Dennis D. Cox & Jong Soo Lee, 2008. "Pointwise testing with functional data using the Westfall--Young randomization method," Biometrika, Biometrika Trust, vol. 95(3), pages 621-634.
    4. Alexander J. McNeil & Rüdiger Frey & Paul Embrechts, 2015. "Quantitative Risk Management: Concepts, Techniques and Tools Revised edition," Economics Books, Princeton University Press, edition 2, number 10496.
    5. Zeileis, Achim, 2004. "Econometric Computing with HC and HAC Covariance Matrix Estimators," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 11(i10).
    6. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    7. Giovanni Barone Adesi, 2016. "VaR and CVaR Implied in Option Prices," JRFM, MDPI, vol. 9(1), pages 1-6, February.
    8. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    9. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    10. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    11. Black, Fischer & Scholes, Myron S, 1973. "The Pricing of Options and Corporate Liabilities," Journal of Political Economy, University of Chicago Press, vol. 81(3), pages 637-654, May-June.
    12. Ole E. Barndorff-Nielsen & Peter Reinhard Hansen & Asger Lunde & Neil Shephard, 2008. "Designing Realized Kernels to Measure the ex post Variation of Equity Prices in the Presence of Noise," Econometrica, Econometric Society, vol. 76(6), pages 1481-1536, November.
    13. O. E. Barndorff-Nielsen & P. Reinhard Hansen & A. Lunde & N. Shephard, 2009. "Realized kernels in practice: trades and quotes," Econometrics Journal, Royal Economic Society, vol. 12(3), pages 1-32, November.
    14. Andrew Harvey & Esther Ruiz & Neil Shephard, 1994. "Multivariate Stochastic Variance Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 61(2), pages 247-264.
    15. Tsyplakov, Alexander, 2014. "Theoretical guidelines for a partially informed forecast examiner," MPRA Paper 55017, University Library of Munich, Germany.
    16. G. Elliott & C. Granger & A. Timmermann (ed.), 2013. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 2, number 2.
    17. Hajo Holzmann & Matthias Eulert, 2014. "The role of the information set for forecasting - with applications to risk management," Papers 1404.7653, arXiv.org.
    18. Sovan Mitra, 2015. "The relationship between conditional value at risk and option prices with a closed-form solution," The European Journal of Finance, Taylor & Francis Journals, vol. 21(5), pages 400-425, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Taylor, James W., 2020. "Forecast combinations for value at risk and expected shortfall," International Journal of Forecasting, Elsevier, vol. 36(2), pages 428-441.
    2. Sander Barendse & Andrew J. Patton, 2022. "Comparing Predictive Accuracy in the Presence of a Loss Function Shape Parameter," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(3), pages 1057-1069, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ziegel, Johanna F. & Krueger, Fabian & Jordan, Alexander & Fasciati, Fernando, 2017. "Murphy Diagrams: Forecast Evaluation of Expected Shortfall," Working Papers 0632, University of Heidelberg, Department of Economics.
    2. Catania, Leopoldo & Proietti, Tommaso, 2020. "Forecasting volatility with time-varying leverage and volatility of volatility effects," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1301-1317.
    3. Giovanni Bonaccolto & Massimiliano Caporin, 2016. "The Determinants of Equity Risk and Their Forecasting Implications: A Quantile Regression Perspective," JRFM, MDPI, vol. 9(3), pages 1-25, July.
    4. Werner Ehm & Tilmann Gneiting & Alexander Jordan & Fabian Krüger, 2016. "Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 505-562, June.
    5. Fabian Krüger & Todd E. Clark & Francesco Ravazzolo, 2017. "Using Entropic Tilting to Combine BVAR Forecasts With External Nowcasts," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 35(3), pages 470-485, July.
    6. Malte Knüppel & Fabian Krüger, 2022. "Forecast uncertainty, disagreement, and the linear pool," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(1), pages 23-41, January.
    7. Andrea BUCCI, 2017. "Forecasting Realized Volatility A Review," Journal of Advanced Studies in Finance, ASERS Publishing, vol. 8(2), pages 94-138.
    8. Heejoon Han & Shen Zhang, 2012. "Non‐stationary non‐parametric volatility model," Econometrics Journal, Royal Economic Society, vol. 15(2), pages 204-225, June.
    9. Giampiero M. Gallo & Edoardo Otranto, 2014. "Forecasting Realized Volatility with Changes of Regimes," Econometrics Working Papers Archive 2014_03, Universita' degli Studi di Firenze, Dipartimento di Statistica, Informatica, Applicazioni "G. Parenti", revised Feb 2014.
    10. Kevin Sheppard & Wen Xu, 2014. "Factor High-Frequency Based Volatility (HEAVY) Models," Economics Series Working Papers 710, University of Oxford, Department of Economics.
    11. Gallo, Giampiero M. & Otranto, Edoardo, 2015. "Forecasting realized volatility with changing average levels," International Journal of Forecasting, Elsevier, vol. 31(3), pages 620-634.
    12. Sharma, Prateek & Vipul,, 2016. "Forecasting stock market volatility using Realized GARCH model: International evidence," The Quarterly Review of Economics and Finance, Elsevier, vol. 59(C), pages 222-230.
    13. Rossi, Alessandro & Gallo, Giampiero M., 2006. "Volatility estimation via hidden Markov models," Journal of Empirical Finance, Elsevier, vol. 13(2), pages 203-230, March.
    14. Heejoon Han & Myung D. Park, 2013. "Comparison of Realized Measure and Implied Volatility in Forecasting Volatility," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 32(6), pages 522-533, September.
    15. Diego Amaya & Jean-François Bégin & Geneviève Gauthier, 2022. "The Informational Content of High-Frequency Option Prices," Management Science, INFORMS, vol. 68(3), pages 2166-2201, March.
    16. Natalia Nolde & Johanna F. Ziegel, 2016. "Elicitability and backtesting: Perspectives for banking regulation," Papers 1608.05498, arXiv.org, revised Feb 2017.
    17. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    18. Diaa Noureldin & Neil Shephard & Kevin Sheppard, 2012. "Multivariate high‐frequency‐based volatility (HEAVY) models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 907-933, September.
    19. Mandalinci, Zeyyad, 2017. "Forecasting inflation in emerging markets: An evaluation of alternative models," International Journal of Forecasting, Elsevier, vol. 33(4), pages 1082-1104.
    20. Patton, Andrew J. & Ziegel, Johanna F. & Chen, Rui, 2019. "Dynamic semiparametric models for expected shortfall (and Value-at-Risk)," Journal of Econometrics, Elsevier, vol. 211(2), pages 388-413.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1705.04537. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.