IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0174202.html
   My bibliography  Save this article

A new accuracy measure based on bounded relative error for time series forecasting

Author

Listed:
  • Chao Chen
  • Jamie Twycross
  • Jonathan M Garibaldi

Abstract

Many accuracy measures have been proposed in the past for time series forecasting comparisons. However, many of these measures suffer from one or more issues such as poor resistance to outliers and scale dependence. In this paper, while summarising commonly used accuracy measures, a special review is made on the symmetric mean absolute percentage error. Moreover, a new accuracy measure called the Unscaled Mean Bounded Relative Absolute Error (UMBRAE), which combines the best features of various alternative measures, is proposed to address the common issues of existing measures. A comparative evaluation on the proposed and related measures has been made with both synthetic and real-world data. The results indicate that the proposed measure, with user selectable benchmark, performs as well as or better than other measures on selected criteria. Though it has been commonly accepted that there is no single best accuracy measure, we suggest that UMBRAE could be a good choice to evaluate forecasting methods, especially for cases where measures based on geometric mean of relative errors, such as the geometric mean relative absolute error, are preferred.

Suggested Citation

  • Chao Chen & Jamie Twycross & Jonathan M Garibaldi, 2017. "A new accuracy measure based on bounded relative error for time series forecasting," PLOS ONE, Public Library of Science, vol. 12(3), pages 1-23, March.
    • Handle: RePEc:plo:pone00:0174202
    DOI: 10.1371/journal.pone.0174202
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0174202
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0174202&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0174202?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nahapetyan Yervand, 2019. "The benefits of the Velvet Revolution in Armenia: Estimation of the short-term economic gains using deep neural networks," Central European Economic Journal, Sciendo, vol. 53(6), pages 286-303, January.
    2. Hayashi, Masayoshi, 2014. "Forecasting welfare caseloads: The case of the Japanese public assistance program," Socio-Economic Planning Sciences, Elsevier, vol. 48(2), pages 105-114.
    3. OlaOluwa S. Yaya & Ahamuefula E. Ogbonna & Fumitaka Furuoka & Luis A. Gil‐Alana, 2021. "A New Unit Root Test for Unemployment Hysteresis Based on the Autoregressive Neural Network," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 83(4), pages 960-981, August.
    4. Kourentzes, Nikolaos & Petropoulos, Fotios & Trapero, Juan R., 2014. "Improving forecasting by estimating time series structural components across multiple frequencies," International Journal of Forecasting, Elsevier, vol. 30(2), pages 291-302.
    5. Man Li & Tao Ye & Peijun Shi & Jian Fang, 2015. "Impacts of the global economic crisis and Tohoku earthquake on Sino–Japan trade: a comparative perspective," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 75(1), pages 541-556, January.
    6. Maghsoodi, Abtin Ijadi, 2023. "Cryptocurrency portfolio allocation using a novel hybrid and predictive big data decision support system," Omega, Elsevier, vol. 115(C).
    7. Anna Staszewska-Bystrova & Peter Winker, 2016. "Improved bootstrap prediction intervals for SETAR models," Statistical Papers, Springer, vol. 57(1), pages 89-98, March.
    8. Döpke, Jörg & Fritsche, Ulrich & Müller, Karsten, 2019. "Has macroeconomic forecasting changed after the Great Recession? Panel-based evidence on forecast accuracy and forecaster behavior from Germany," Journal of Macroeconomics, Elsevier, vol. 62(C).
    9. Goodwin, Paul & Önkal, Dilek & Thomson, Mary, 2010. "Do forecasts expressed as prediction intervals improve production planning decisions?," European Journal of Operational Research, Elsevier, vol. 205(1), pages 195-201, August.
    10. Nowotarski, Jakub & Weron, Rafał, 2018. "Recent advances in electricity price forecasting: A review of probabilistic forecasting," Renewable and Sustainable Energy Reviews, Elsevier, vol. 81(P1), pages 1548-1568.
    11. Giuseppe Cavaliere & Dimitris N. Politis & Anders Rahbek & Paul Doukhan & Gabriel Lang & Anne Leucht & Michael H. Neumann, 2015. "Recent developments in bootstrap methods for dependent data," Journal of Time Series Analysis, Wiley Blackwell, vol. 36(3), pages 290-314, May.
    12. Charles, Amelie & Darne, Olivier & Kim, Jae, 2016. "Stock Return Predictability: Evaluation based on Prediction Intervals," MPRA Paper 70143, University Library of Munich, Germany.
    13. Syntetos, Aris A. & Nikolopoulos, Konstantinos & Boylan, John E. & Fildes, Robert & Goodwin, Paul, 2009. "The effects of integrating management judgement into intermittent demand forecasts," International Journal of Production Economics, Elsevier, vol. 118(1), pages 72-81, March.
    14. Schneider, Matthew J. & Gupta, Sachin, 2016. "Forecasting sales of new and existing products using consumer reviews: A random projections approach," International Journal of Forecasting, Elsevier, vol. 32(2), pages 243-256.
    15. Vidhi Vig & Anmol Kaur, 2022. "Time series forecasting and mathematical modeling of COVID-19 pandemic in India: a developing country struggling to cope up," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 13(6), pages 2920-2933, December.
    16. Voyant, Cyril & Notton, Gilles & Duchaud, Jean-Laurent & Gutiérrez, Luis Antonio García & Bright, Jamie M. & Yang, Dazhi, 2022. "Benchmarks for solar radiation time series forecasting," Renewable Energy, Elsevier, vol. 191(C), pages 747-762.
    17. Abhishek Singh & G. C. Mishra, 2015. "Application Of Box-Jenkins Method And Artificial Neural Network Procedure For Time Series Forecasting Of Prices," Statistics in Transition New Series, Polish Statistical Association, vol. 16(1), pages 83-96, March.
    18. Nieto, María Rosa & Carmona-Benítez, Rafael Bernardo, 2018. "ARIMA + GARCH + Bootstrap forecasting method applied to the airline industry," Journal of Air Transport Management, Elsevier, vol. 71(C), pages 1-8.
    19. Mergani A. Khairalla & Xu Ning & Nashat T. AL-Jallad & Musaab O. El-Faroug, 2018. "Short-Term Forecasting for Energy Consumption through Stacking Heterogeneous Ensemble Learning Model," Energies, MDPI, vol. 11(6), pages 1-21, June.
    20. Marzouq, Manal & El Fadili, Hakim & Zenkouar, Khalid & Lakhliai, Zakia & Amouzg, Mohammed, 2020. "Short term solar irradiance forecasting via a novel evolutionary multi-model framework and performance assessment for sites with no solar irradiance data," Renewable Energy, Elsevier, vol. 157(C), pages 214-231.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0174202. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.