IDEAS home Printed from https://ideas.repec.org/a/jof/jforec/v24y2005i6p433-451.html
   My bibliography  Save this article

Evaluating forecasts: a look at aggregate bias and accuracy measures

Author

Listed:
  • Dean W. Wichern

    (Texas A&M University, USA)

  • Benito E. Flores

    (Texas A&M University, USA)

Abstract

In this paper an investigation is made of the properties and use of two aggregate measures of forecast bias and accuracy. These are metrics used in business to calculate aggregate forecasting performance for a family (group) of products. We find that the aggregate measures are not particularly informative if some of the one-step-ahead forecasts are biased. This is likely to be the case in practice if frequently employed forecasting methods are used to generate a large number of individual forecasts. In the paper, examples are constructed to illustrate some potential problems in the use of the metrics. We propose a simple graphical display of forecast bias and accuracy to supplement the information yielded by the accuracy measures. This support includes relevant boxplots of measures of individual forecasting success. This tool is simple but helpful as the graphic display has the potential to indicate forecast deterioration that can be masked by one or both of the aggregate metrics. The procedures are illustrated with data representing sales of food items. Copyright © 2005 John Wiley & Sons, Ltd.

Suggested Citation

  • Dean W. Wichern & Benito E. Flores, 2005. "Evaluating forecasts: a look at aggregate bias and accuracy measures," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 24(6), pages 433-451.
    • Handle: RePEc:jof:jforec:v:24:y:2005:i:6:p:433-451
    DOI: 10.1002/for.962
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1002/for.962
    File Function: Link to full text; subscription required
    Download Restriction: no
    File URL: https://libkey.io/10.1002/for.962?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Assimakopoulos, V. & Nikolopoulos, K., 2000. "The theta model: a decomposition approach to forecasting," International Journal of Forecasting, Elsevier, vol. 16(4), pages 521-530.
    2. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    3. Hyndman, Rob J. & Billah, Baki, 2003. "Unmasking the Theta method," International Journal of Forecasting, Elsevier, vol. 19(2), pages 287-290.
    4. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
    5. Makridakis, Spyros, 1993. "Accuracy measures: theoretical and practical concerns," International Journal of Forecasting, Elsevier, vol. 9(4), pages 527-529, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wan, Xiang & Sanders, Nadia R., 2017. "The negative impact of product variety: Forecast bias, inventory levels, and the role of vertical integration," International Journal of Production Economics, Elsevier, vol. 186(C), pages 123-131.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
    2. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    3. Jan G. De Gooijer & Rob J. Hyndman, 2005. "25 Years of IIF Time Series Forecasting: A Selective Review," Monash Econometrics and Business Statistics Working Papers 12/05, Monash University, Department of Econometrics and Business Statistics.
    4. Sbrana, Giacomo & Silvestrini, Andrea, 2023. "The RWDAR model: A novel state-space approach to forecasting," International Journal of Forecasting, Elsevier, vol. 39(2), pages 922-937.
    5. Makridakis, Spyros & Spiliotis, Evangelos & Assimakopoulos, Vassilios, 2020. "The M4 Competition: 100,000 time series and 61 forecasting methods," International Journal of Forecasting, Elsevier, vol. 36(1), pages 54-74.
    6. Spiliotis, Evangelos & Nikolopoulos, Konstantinos & Assimakopoulos, Vassilios, 2019. "Tales from tails: On the empirical distributions of forecasting errors and their implication to risk," International Journal of Forecasting, Elsevier, vol. 35(2), pages 687-698.
    7. Theodosiou, Marina, 2011. "Forecasting monthly and quarterly time series using STL decomposition," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1178-1195, October.
    8. Sbrana, Giacomo & Silvestrini, Andrea, 2022. "Random coefficient state-space model: Estimation and performance in M3–M4 competitions," International Journal of Forecasting, Elsevier, vol. 38(1), pages 352-366.
    9. Hyndman, Rob J. & Khandakar, Yeasmin, 2008. "Automatic Time Series Forecasting: The forecast Package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i03).
    10. Fildes, Robert & Petropoulos, Fotios, 2015. "Simple versus complex selection rules for forecasting many time series," Journal of Business Research, Elsevier, vol. 68(8), pages 1692-1701.
    11. Bruzda, Joanna, 2019. "Quantile smoothing in supply chain and logistics forecasting," International Journal of Production Economics, Elsevier, vol. 208(C), pages 122-139.
    12. Athanasopoulos, George & Hyndman, Rob J. & Song, Haiyan & Wu, Doris C., 2011. "The tourism forecasting competition," International Journal of Forecasting, Elsevier, vol. 27(3), pages 822-844.
    13. Spiliotis, Evangelos & Assimakopoulos, Vassilios & Nikolopoulos, Konstantinos, 2019. "Forecasting with a hybrid method utilizing data smoothing, a variation of the Theta method and shrinkage of seasonal factors," International Journal of Production Economics, Elsevier, vol. 209(C), pages 92-102.
    14. João A. Bastos, 2019. "Forecasting the capacity of mobile networks," Telecommunication Systems: Modelling, Analysis, Design and Management, Springer, vol. 72(2), pages 231-242, October.
    15. Hyndman, Rob J., 2020. "A brief history of forecasting competitions," International Journal of Forecasting, Elsevier, vol. 36(1), pages 7-14.
    16. Weron, Rafał, 2014. "Electricity price forecasting: A review of the state-of-the-art with a look into the future," International Journal of Forecasting, Elsevier, vol. 30(4), pages 1030-1081.
    17. Semenoglou, Artemios-Anargyros & Spiliotis, Evangelos & Makridakis, Spyros & Assimakopoulos, Vassilios, 2021. "Investigating the accuracy of cross-learning time series forecasting methods," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1072-1084.
    18. Ji Wu & Xian Cheng & Stephen Shaoyi Liao, 2020. "Tourism forecast combination using the stochastic frontier analysis technique," Tourism Economics, , vol. 26(7), pages 1086-1107, November.
    19. Jennifer L. Castle & Jurgen A. Doornik & David F. Hendry, 2021. "Forecasting Principles from Experience with Forecasting Competitions," Forecasting, MDPI, vol. 3(1), pages 1-28, February.
    20. Wen, Xin & Jaxa-Rozen, Marc & Trutnevyte, Evelina, 2022. "Accuracy indicators for evaluating retrospective performance of energy system models," Applied Energy, Elsevier, vol. 325(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:jof:jforec:v:24:y:2005:i:6:p:433-451. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley-Blackwell Digital Licensing or Christopher F. Baum (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/2966 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.