IDEAS home Printed from https://ideas.repec.org/p/tin/wpaper/20110141.html
   My bibliography  Save this paper

Do Experts incorporate Statistical Model Forecasts and should they?

Author

Listed:
  • Rianne Legerstee

    (Erasmus University Rotterdam)

  • Philip Hans Franses

    (Erasmus University Rotterdam)

  • Richard Paap

    (Erasmus University Rotterdam)

Abstract

Experts can rely on statistical model forecasts when creating their own forecasts. Usually it is not known what experts actually do. In this paper we focus on three questions, which we try to answer given the availability of expert forecasts and model forecasts. First, is the expert forecast related to the model forecast and how? Second, how is this potential relation influenced by other factors? Third, how does this relation influence forecast accuracy? We propose a new and innovative two-level Hierarchical Bayes model to answer these questions. We apply our proposed methodology to a large data set of forecasts and realizations of SKU-level sales data from a pharmaceutical company. We find that expert forecasts can depend on model forecasts in a variety of ways. Average sales levels, sales volatility, and the forecast horizon influence this dependence. We also demonstrate that theoretical implications of expert behavior on forecast accuracy are reflected in the empirical data.

Suggested Citation

  • Rianne Legerstee & Philip Hans Franses & Richard Paap, 2011. "Do Experts incorporate Statistical Model Forecasts and should they?," Tinbergen Institute Discussion Papers 11-141/4, Tinbergen Institute.
  • Handle: RePEc:tin:wpaper:20110141
    as

    Download full text from publisher

    File URL: https://papers.tinbergen.nl/11141.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Heij, Christiaan & de Boer, Paul & Franses, Philip Hans & Kloek, Teun & van Dijk, Herman K., 2004. "Econometric Methods with Applications in Business and Economics," OUP Catalogue, Oxford University Press, number 9780199268016.
    2. Philip Hans Franses & Rianne Legerstee, 2010. "Do experts' adjustments on model-based SKU-level forecasts improve forecast quality?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(3), pages 331-340.
    3. Sanders, NR, 1992. "Accuracy of judgmental forecasts: A comparison," Omega, Elsevier, vol. 20(3), pages 353-364, May.
    4. McNees, Stephen K., 1990. "The role of judgment in macroeconomic forecasting accuracy," International Journal of Forecasting, Elsevier, vol. 6(3), pages 287-299, October.
    5. Robert Fildes & Paul Goodwin, 2007. "Good and Bad Judgment in Forecasting: Lessons from Four Companies," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 8, pages 5-10, Fall.
    6. Goodwin, Paul, 2000. "Improving the voluntary integration of statistical forecasts and judgment," International Journal of Forecasting, Elsevier, vol. 16(1), pages 85-99.
    7. Franses, Philip Hans & Legerstee, Rianne, 2009. "Properties of expert adjustments on model-based SKU-level forecasts," International Journal of Forecasting, Elsevier, vol. 25(1), pages 35-47.
    8. Robert C. Blattberg & Stephen J. Hoch, 1990. "Database Models and Managerial Intuition: 50% Model + 50% Manager," Management Science, INFORMS, vol. 36(8), pages 887-899, August.
    9. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    10. Bunn, Derek W. & Salo, Ahti A., 1996. "Adjustment of forecasts with model consistent expectations," International Journal of Forecasting, Elsevier, vol. 12(1), pages 163-170, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rianne Legerstee & Philip Hans Franses, 2014. "Do Experts’ SKU Forecasts Improve after Feedback?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 69-79, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rianne Legerstee & Philip Hans Franses, 2014. "Do Experts’ SKU Forecasts Improve after Feedback?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 69-79, January.
    2. Petropoulos, Fotios & Fildes, Robert & Goodwin, Paul, 2016. "Do ‘big losses’ in judgmental adjustments to statistical forecasts affect experts’ behaviour?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 842-852.
    3. Philip Hans Franses & Rianne Legerstee, 2010. "Do experts' adjustments on model-based SKU-level forecasts improve forecast quality?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(3), pages 331-340.
    4. Franses, Philip Hans & Kranendonk, Henk C. & Lanser, Debby, 2011. "One model and various experts: Evaluating Dutch macroeconomic forecasts," International Journal of Forecasting, Elsevier, vol. 27(2), pages 482-495.
    5. Perera, H. Niles & Hurley, Jason & Fahimnia, Behnam & Reisi, Mohsen, 2019. "The human factor in supply chain forecasting: A systematic review," European Journal of Operational Research, Elsevier, vol. 274(2), pages 574-600.
    6. Franses, Philip Hans & Legerstee, Rianne, 2013. "Do statistical forecasting models for SKU-level data benefit from including past expert knowledge?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 80-87.
    7. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    8. Philip Hans Franses, 2021. "Modeling Judgment in Macroeconomic Forecasts," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 19(1), pages 401-417, December.
    9. P H Franses & R Legerstee, 2011. "Experts' adjustment to model-based SKU-level forecasts: does the forecast horizon matter?," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(3), pages 537-543, March.
    10. Franses, Philip Hans, 2013. "Improving judgmental adjustment of model-based forecasts," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 93(C), pages 1-8.
    11. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    12. Bert de Bruijn & Philip Hans Franses, 2012. "Managing Sales Forecasters," Tinbergen Institute Discussion Papers 12-131/III, Tinbergen Institute.
    13. Lawrence, Michael & Goodwin, Paul & O'Connor, Marcus & Onkal, Dilek, 2006. "Judgmental forecasting: A review of progress over the last 25 years," International Journal of Forecasting, Elsevier, vol. 22(3), pages 493-518.
    14. Syntetos, Aris A. & Kholidasari, Inna & Naim, Mohamed M., 2016. "The effects of integrating management judgement into OUT levels: In or out of context?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 853-863.
    15. Philip Hans Franses & Michael McAleer & Rianne Legerstee, 2009. "Expert opinion versus expertise in forecasting," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 63(3), pages 334-346, August.
    16. Fildes, Robert & Goodwin, Paul, 2021. "Stability in the inefficient use of forecasting systems: A case study in a supply chain company," International Journal of Forecasting, Elsevier, vol. 37(2), pages 1031-1046.
    17. De Baets, Shari & Harvey, Nigel, 2018. "Forecasting from time series subject to sporadic perturbations: Effectiveness of different types of forecasting support," International Journal of Forecasting, Elsevier, vol. 34(2), pages 163-180.
    18. Sroginis, Anna & Fildes, Robert & Kourentzes, Nikolaos, 2023. "Use of contextual and model-based information in adjusting promotional forecasts," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1177-1191.
    19. Arvan, Meysam & Fahimnia, Behnam & Reisi, Mohsen & Siemsen, Enno, 2019. "Integrating human judgement into quantitative forecasting methods: A review," Omega, Elsevier, vol. 86(C), pages 237-252.
    20. Chang, Chia-Lin & Franses, Philip Hans & McAleer, Michael, 2011. "How accurate are government forecasts of economic fundamentals? The case of Taiwan," International Journal of Forecasting, Elsevier, vol. 27(4), pages 1066-1075, October.

    More about this item

    Keywords

    model forecasts; expert forecasts; forecast adjustment; Bayesian analysis; endogeneity;
    All these keywords.

    JEL classification:

    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tin:wpaper:20110141. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Tinbergen Office +31 (0)10-4088900 (email available below). General contact details of provider: https://edirc.repec.org/data/tinbenl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.