IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v37y2021i2p800-811.html
   My bibliography  Save this article

Evaluating quantile-bounded and expectile-bounded interval forecasts

Author

Listed:
  • Taylor, James W.

Abstract

In many different contexts, decision-making is improved by the availability of probabilistic predictions. The accuracy of probabilistic forecasting methods can be compared using scoring functions and insight provided by calibration tests. These tests evaluate the consistency of predictions with the observations. Our main agenda in this paper is interval forecasts and their evaluation. Such forecasts are usually bounded by two quantile forecasts. However, a limitation of quantiles is that they convey no information regarding the size of potential exceedances. By contrast, the location of an expectile is dictated by the whole distribution. This prompts us to propose expectile-bounded intervals. We provide interpretation, a consistent scoring function and a calibration test. Before doing this, we reflect on the evaluation of forecasts of quantile-bounded intervals and expectiles, and suggest extensions of previously proposed calibration tests in order to guard against strategic forecasting. We illustrate ideas using day-ahead electricity price forecasting.

Suggested Citation

  • Taylor, James W., 2021. "Evaluating quantile-bounded and expectile-bounded interval forecasts," International Journal of Forecasting, Elsevier, vol. 37(2), pages 800-811.
    • Handle: RePEc:eee:intfor:v:37:y:2021:i:2:p:800-811
    DOI: 10.1016/j.ijforecast.2020.09.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207020301485
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijforecast.2020.09.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Natalia Nolde & Johanna F. Ziegel, 2016. "Elicitability and backtesting: Perspectives for banking regulation," Papers 1608.05498, arXiv.org, revised Feb 2017.
    2. Kuan, Chung-Ming & Yeh, Jin-Huei & Hsu, Yu-Chin, 2009. "Assessing value at risk with CARE, the Conditional Autoregressive Expectile models," Journal of Econometrics, Elsevier, vol. 150(2), pages 261-270, June.
    3. Gerlach, Richard & Wang, Chao, 2020. "Semi-parametric dynamic asymmetric Laplace models for tail risk forecasting, incorporating realized measures," International Journal of Forecasting, Elsevier, vol. 36(2), pages 489-506.
    4. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    5. Raffaella Giacomini & Halbert White, 2006. "Tests of Conditional Predictive Ability," Econometrica, Econometric Society, vol. 74(6), pages 1545-1578, November.
    6. Knittel, Christopher R. & Roberts, Michael R., 2005. "An empirical examination of restructured electricity prices," Energy Economics, Elsevier, vol. 27(5), pages 791-817, September.
    7. Johanna F. Ziegel, 2016. "Coherence And Elicitability," Mathematical Finance, Wiley Blackwell, vol. 26(4), pages 901-918, October.
    8. Rafal Weron & Florian Ziel, 2018. "Electricity price forecasting," HSC Research Reports HSC/18/08, Hugo Steinhaus Center, Wroclaw University of Technology.
    9. Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
    10. Koenker,Roger, 2005. "Quantile Regression," Cambridge Books, Cambridge University Press, number 9780521845731, January.
    11. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    12. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    13. Olszewski, Wojciech, 2015. "Calibration and Expert Testing," Handbook of Game Theory with Economic Applications,, Elsevier.
    14. Glosten, Lawrence R & Jagannathan, Ravi & Runkle, David E, 1993. "On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks," Journal of Finance, American Finance Association, vol. 48(5), pages 1779-1801, December.
    15. Yao, Qiwei & Tong, Howell, 1996. "Asymmetric least squares regression estimation: a nonparametric approach," LSE Research Online Documents on Economics 19423, London School of Economics and Political Science, LSE Library.
    16. Bartosz Uniejewski & Rafal Weron & Florian Ziel, 2017. "Variance stabilizing transformations for electricity spot price forecasting," HSC Research Reports HSC/17/01, Hugo Steinhaus Center, Wroclaw University of Technology.
    17. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2018. "The M4 Competition: Results, findings, conclusion and way forward," International Journal of Forecasting, Elsevier, vol. 34(4), pages 802-808.
    18. Weron, Rafal & Misiorek, Adam, 2008. "Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models," International Journal of Forecasting, Elsevier, vol. 24(4), pages 744-763.
    19. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    20. Philippe Artzner & Freddy Delbaen & Jean‐Marc Eber & David Heath, 1999. "Coherent Measures of Risk," Mathematical Finance, Wiley Blackwell, vol. 9(3), pages 203-228, July.
    21. Kenneth C. Lichtendahl & Yael Grushka-Cockayne & Phillip E. Pfeifer, 2013. "The Wisdom of Competitive Crowds," Operations Research, INFORMS, vol. 61(6), pages 1383-1398, December.
    22. Taylor, James W., 2020. "A strategic predictive distribution for tests of probabilistic calibration," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1380-1388.
    23. Newey, Whitney K & Powell, James L, 1987. "Asymmetric Least Squares Estimation and Testing," Econometrica, Econometric Society, vol. 55(4), pages 819-847, July.
    24. James W. Taylor, 2008. "Estimating Value at Risk and Expected Shortfall Using Expectiles," Journal of Financial Econometrics, Oxford University Press, vol. 6(2), pages 231-252, Spring.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Zhu, Bangzhu & Wan, Chunzhuo & Wang, Ping, 2022. "Interval forecasting of carbon price: A novel multiscale ensemble forecasting approach," Energy Economics, Elsevier, vol. 115(C).
    2. Janczura, Joanna & Wójcik, Edyta, 2022. "Dynamic short-term risk management strategies for the choice of electricity market based on probabilistic forecasts of profit and risk measures. The German and the Polish market case study," Energy Economics, Elsevier, vol. 110(C).
    3. James Mitchell & Martin Weale, 2023. "Censored density forecasts: Production and evaluation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 714-734, August.
    4. Stephen Haben & Julien Caudron & Jake Verma, 2021. "Probabilistic Day-Ahead Wholesale Price Forecast: A Case Study in Great Britain," Forecasting, MDPI, vol. 3(3), pages 1-37, August.
    5. Norman Maswanganyi & Caston Sigauke & Edmore Ranganai, 2021. "Prediction of Extreme Conditional Quantiles of Electricity Demand: An Application Using South African Data," Energies, MDPI, vol. 14(20), pages 1-21, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hoga, Yannick, 2021. "The uncertainty in extreme risk forecasts from covariate-augmented volatility models," International Journal of Forecasting, Elsevier, vol. 37(2), pages 675-686.
    2. Antonio Rubia Serrano & Lidia Sanchis-Marco, 2015. "Measuring Tail-Risk Cross-Country Exposures in the Banking Industry," Working Papers. Serie AD 2015-01, Instituto Valenciano de Investigaciones Económicas, S.A. (Ivie).
    3. Hamidi, Benjamin & Maillet, Bertrand & Prigent, Jean-Luc, 2014. "A dynamic autoregressive expectile for time-invariant portfolio protection strategies," Journal of Economic Dynamics and Control, Elsevier, vol. 46(C), pages 1-29.
    4. Taylor, James W., 2020. "Forecast combinations for value at risk and expected shortfall," International Journal of Forecasting, Elsevier, vol. 36(2), pages 428-441.
    5. Patton, Andrew J. & Ziegel, Johanna F. & Chen, Rui, 2019. "Dynamic semiparametric models for expected shortfall (and Value-at-Risk)," Journal of Econometrics, Elsevier, vol. 211(2), pages 388-413.
    6. Mohammedi, Mustapha & Bouzebda, Salim & Laksaci, Ali, 2021. "The consistency and asymptotic normality of the kernel type expectile regression estimator for functional data," Journal of Multivariate Analysis, Elsevier, vol. 181(C).
    7. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.
    8. Kratz, Marie & Lok, Yen H. & McNeil, Alexander J., 2018. "Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall," Journal of Banking & Finance, Elsevier, vol. 88(C), pages 393-407.
    9. Qifa Xu & Lu Chen & Cuixia Jiang & Yezheng Liu, 2022. "Forecasting expected shortfall and value at risk with a joint elicitable mixed data sampling model," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(3), pages 407-421, April.
    10. Timo Dimitriadis & Xiaochun Liu & Julie Schnaitmann, 2020. "Encompassing Tests for Value at Risk and Expected Shortfall Multi-Step Forecasts based on Inference on the Boundary," Papers 2009.07341, arXiv.org.
    11. Fissler Tobias & Ziegel Johanna F., 2021. "On the elicitability of range value at risk," Statistics & Risk Modeling, De Gruyter, vol. 38(1-2), pages 25-46, January.
    12. James Ming Chen, 2018. "On Exactitude in Financial Regulation: Value-at-Risk, Expected Shortfall, and Expectiles," Risks, MDPI, vol. 6(2), pages 1-28, June.
    13. David Happersberger & Harald Lohre & Ingmar Nolte, 2020. "Estimating portfolio risk for tail risk protection strategies," European Financial Management, European Financial Management Association, vol. 26(4), pages 1107-1146, September.
    14. Natalia Nolde & Johanna F. Ziegel, 2016. "Elicitability and backtesting: Perspectives for banking regulation," Papers 1608.05498, arXiv.org, revised Feb 2017.
    15. Vica Tendenan & Richard Gerlach & Chao Wang, 2020. "Tail risk forecasting using Bayesian realized EGARCH models," Papers 2008.05147, arXiv.org, revised Aug 2020.
    16. Taylor, James W., 2022. "Forecasting Value at Risk and expected shortfall using a model with a dynamic omega ratio," Journal of Banking & Finance, Elsevier, vol. 140(C).
    17. Dimitriadis, Timo & Schnaitmann, Julie, 2021. "Forecast encompassing tests for the expected shortfall," International Journal of Forecasting, Elsevier, vol. 37(2), pages 604-621.
    18. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    19. Santos, Douglas G. & Candido, Osvaldo & Tófoli, Paula V., 2022. "Forecasting risk measures using intraday and overnight information," The North American Journal of Economics and Finance, Elsevier, vol. 60(C).
    20. Bonaccolto, Giovanni & Caporin, Massimiliano & Maillet, Bertrand B., 2022. "Dynamic large financial networks via conditional expected shortfalls," European Journal of Operational Research, Elsevier, vol. 298(1), pages 322-336.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:37:y:2021:i:2:p:800-811. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.