IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v33y2017i1p298-313.html
   My bibliography  Save this article

Expertise, credibility of system forecasts and integration methods in judgmental demand forecasting

Author

Listed:
  • Alvarado-Valencia, Jorge
  • Barrero, Lope H.
  • Önkal, Dilek
  • Dennerlein, Jack T.

Abstract

Expert knowledge elicitation lies at the core of judgmental forecasting—a domain that relies fully on the power of such knowledge and its integration into forecasting. Using experts in a demand forecasting framework, this work aims to compare the accuracy improvements and forecasting performances of three judgmental integration methods. To do this, a field study was conducted with 31 experts from four companies. The methods compared were the judgmental adjustment, the 50–50 combination, and the divide-and-conquer. Forecaster expertise, the credibility of system forecasts and the need to rectify system forecasts were also assessed, and mechanisms for performing this assessment were considered. When (a) a forecaster’s relative expertise was high, (b) the relative credibility of the system forecasts was low, and (c) the system forecasts had a strong need of correction, judgmental adjustment improved the accuracy relative to both the other integration methods and the system forecasts. Experts with higher levels of expertise showed higher adjustment frequencies. Our results suggest that judgmental adjustment promises to be valuable in the long term if adequate conditions of forecaster expertise and the credibility of system forecasts are met.

Suggested Citation

  • Alvarado-Valencia, Jorge & Barrero, Lope H. & Önkal, Dilek & Dennerlein, Jack T., 2017. "Expertise, credibility of system forecasts and integration methods in judgmental demand forecasting," International Journal of Forecasting, Elsevier, vol. 33(1), pages 298-313.
    • Handle: RePEc:eee:intfor:v:33:y:2017:i:1:p:298-313
    DOI: 10.1016/j.ijforecast.2015.12.010
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207016300140
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijforecast.2015.12.010?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Nada R. Sanders & Karl B. Manrodt, 1994. "Forecasting Practices in US Corporations: Survey Results," Interfaces, INFORMS, vol. 24(2), pages 92-100, April.
    2. Lee, Wing Yee & Goodwin, Paul & Fildes, Robert & Nikolopoulos, Konstantinos & Lawrence, Michael, 2007. "Providing support for the use of analogies in demand forecasting tasks," International Journal of Forecasting, Elsevier, vol. 23(3), pages 377-390.
    3. Sanders, Nada R. & Manrodt, Karl B., 2003. "The efficacy of using judgmental versus quantitative forecasting methods in practice," Omega, Elsevier, vol. 31(6), pages 511-522, December.
    4. Webby, Richard & O'Connor, Marcus, 1996. "Judgemental and statistical time series forecasting: a review of the literature," International Journal of Forecasting, Elsevier, vol. 12(1), pages 91-118, March.
    5. Benson, P. George & Onkal, Dilek, 1992. "The effects of feedback and training on the performance of probability forecasters," International Journal of Forecasting, Elsevier, vol. 8(4), pages 559-573, December.
    6. Flandoli, F. & Giorgi, E. & Aspinall, W.P. & Neri, A., 2011. "Comparison of a new expert elicitation model with the Classical Model, equal weights and single experts, using a cross-validation technique," Reliability Engineering and System Safety, Elsevier, vol. 96(10), pages 1292-1310.
    7. Goodwin, Paul & Sinan Gönül, M. & Önkal, Dilek, 2013. "Antecedents and effects of trust in forecasting advice," International Journal of Forecasting, Elsevier, vol. 29(2), pages 354-366.
    8. Willy Aspinall, 2010. "A route to more tractable expert advice," Nature, Nature, vol. 463(7279), pages 294-295, January.
    9. Sinan Gönül & Dilek Önkal & Paul Goodwin, 2009. "Expectations, use and judgmental adjustment of external financial and economic forecasts: an empirical investigation," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 28(1), pages 19-37.
    10. Lawrence, Michael & Goodwin, Paul & O'Connor, Marcus & Onkal, Dilek, 2006. "Judgmental forecasting: A review of progress over the last 25 years," International Journal of Forecasting, Elsevier, vol. 22(3), pages 493-518.
    11. Goodwin, Paul, 2005. "Providing support for decisions based on time series information under conditions of asymmetric loss," European Journal of Operational Research, Elsevier, vol. 163(2), pages 388-402, June.
    12. Goodwin, Paul & Fildes, Robert & Lawrence, Michael & Stephens, Greg, 2011. "Restrictiveness and guidance in support systems," Omega, Elsevier, vol. 39(3), pages 242-253, June.
    13. Hyndman, Rob J. & Koehler, Anne B., 2006. "Another look at measures of forecast accuracy," International Journal of Forecasting, Elsevier, vol. 22(4), pages 679-688.
    14. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    15. Webby, Richard & O'Connor, Marcus & Edmundson, Bob, 2005. "Forecasting support systems for the incorporation of event information: An empirical investigation," International Journal of Forecasting, Elsevier, vol. 21(3), pages 411-423.
    16. Lim, Joa Sang & O'Connor, Marcus, 1996. "Judgmental forecasting with time series and causal information," International Journal of Forecasting, Elsevier, vol. 12(1), pages 139-153, March.
    17. Trapero, Juan R. & Pedregal, Diego J. & Fildes, R. & Kourentzes, N., 2013. "Analysis of judgmental adjustments in the presence of promotions," International Journal of Forecasting, Elsevier, vol. 29(2), pages 234-243.
    18. Goodwin, Paul, 2000. "Correct or combine? Mechanically integrating judgmental forecasts with statistical methods," International Journal of Forecasting, Elsevier, vol. 16(2), pages 261-275.
    19. A A Syntetos & J E Boylan & S M Disney, 2009. "Forecasting for inventory planning: a 50-year review," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(1), pages 149-160, May.
    20. Franses, Philip Hans & Legerstee, Rianne, 2013. "Do statistical forecasting models for SKU-level data benefit from including past expert knowledge?," International Journal of Forecasting, Elsevier, vol. 29(1), pages 80-87.
    21. Wright, George & Lawrence, Michael J. & Collopy, Fred, 1996. "The role and validity of judgment in forecasting," International Journal of Forecasting, Elsevier, vol. 12(1), pages 1-8, March.
    22. Robert C. Blattberg & Stephen J. Hoch, 1990. "Database Models and Managerial Intuition: 50% Model + 50% Manager," Management Science, INFORMS, vol. 36(8), pages 887-899, August.
    23. R Fildes & K Nikolopoulos & S F Crone & A A Syntetos, 2008. "Forecasting and operational research: a review," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(9), pages 1150-1172, September.
    24. Bonaccio, Silvia & Dalal, Reeshad S., 2006. "Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences," Organizational Behavior and Human Decision Processes, Elsevier, vol. 101(2), pages 127-151, November.
    25. Goodwin, Paul, 2002. "Integrating management judgment and statistical methods to improve short-term forecasts," Omega, Elsevier, vol. 30(2), pages 127-135, April.
    26. Armstrong, J. Scott, 2006. "Findings from evidence-based forecasting: Methods for reducing forecast error," International Journal of Forecasting, Elsevier, vol. 22(3), pages 583-598.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bolger, Fergus & Wright, George, 2017. "Use of expert knowledge to anticipate the future: Issues, analysis and directions," International Journal of Forecasting, Elsevier, vol. 33(1), pages 230-243.
    2. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    3. Mario P. Brito & Ian G. J. Dawson, 2020. "Predicting the Validity of Expert Judgments in Assessing the Impact of Risk Mitigation Through Failure Prevention and Correction," Risk Analysis, John Wiley & Sons, vol. 40(10), pages 1928-1943, October.
    4. Hewage, Harsha Chamara & Perera, H. Niles & De Baets, Shari, 2022. "Forecast adjustments during post-promotional periods," European Journal of Operational Research, Elsevier, vol. 300(2), pages 461-472.
    5. Perera, H. Niles & Hurley, Jason & Fahimnia, Behnam & Reisi, Mohsen, 2019. "The human factor in supply chain forecasting: A systematic review," European Journal of Operational Research, Elsevier, vol. 274(2), pages 574-600.
    6. Sroginis, Anna & Fildes, Robert & Kourentzes, Nikolaos, 2023. "Use of contextual and model-based information in adjusting promotional forecasts," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1177-1191.
    7. Abolghasemi, Mahdi & Hurley, Jason & Eshragh, Ali & Fahimnia, Behnam, 2020. "Demand forecasting in the presence of systematic events: Cases in capturing sales promotions," International Journal of Production Economics, Elsevier, vol. 230(C).
    8. Bolger, Fergus & Rowe, Gene & Belton, Ian & Crawford, Megan M & Hamlin, Iain & Sissons, Aileen & Taylor Browne Lūka, Courtney & Vasilichi, Alexandrina & Wright, George, 2020. "The Simulated Group Response Paradigm: A new approach to the study of opinion change in Delphi and other structured-group techniques," OSF Preprints 4ufzg, Center for Open Science.
    9. Larissa Koupriouchina & Jean-Pierre van der Rest & Zvi Schwartz, 2023. "Judgmental Adjustments of Algorithmic Hotel Occupancy Forecasts: Does User Override Frequency Impact Accuracy at Different Time Horizons?," Tourism Economics, , vol. 29(8), pages 2143-2164, December.
    10. Mauksch, Stefanie & von der Gracht, Heiko A. & Gordon, Theodore J., 2020. "Who is an expert for foresight? A review of identification methods," Technological Forecasting and Social Change, Elsevier, vol. 154(C).
    11. Jože Rožanec & Elena Trajkova & Inna Novalija & Patrik Zajec & Klemen Kenda & Blaž Fortuna & Dunja Mladenić, 2022. "Enriching Artificial Intelligence Explanations with Knowledge Fragments," Future Internet, MDPI, vol. 14(5), pages 1-13, April.
    12. Arvan, Meysam & Fahimnia, Behnam & Reisi, Mohsen & Siemsen, Enno, 2019. "Integrating human judgement into quantitative forecasting methods: A review," Omega, Elsevier, vol. 86(C), pages 237-252.
    13. Fildes, Robert & Goodwin, Paul & Önkal, Dilek, 2019. "Use and misuse of information in supply chain forecasting of promotion effects," International Journal of Forecasting, Elsevier, vol. 35(1), pages 144-156.
    14. Christiane B. Haubitz & Cedric A. Lehmann & Andreas Fügener & Ulrich W. Thonemann, 2021. "The Risk of Algorithm Transparency: How Algorithm Complexity Drives the Effects on Use of Advice," ECONtribute Discussion Papers Series 078, University of Bonn and University of Cologne, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Perera, H. Niles & Hurley, Jason & Fahimnia, Behnam & Reisi, Mohsen, 2019. "The human factor in supply chain forecasting: A systematic review," European Journal of Operational Research, Elsevier, vol. 274(2), pages 574-600.
    2. Arvan, Meysam & Fahimnia, Behnam & Reisi, Mohsen & Siemsen, Enno, 2019. "Integrating human judgement into quantitative forecasting methods: A review," Omega, Elsevier, vol. 86(C), pages 237-252.
    3. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    4. Abolghasemi, Mahdi & Hurley, Jason & Eshragh, Ali & Fahimnia, Behnam, 2020. "Demand forecasting in the presence of systematic events: Cases in capturing sales promotions," International Journal of Production Economics, Elsevier, vol. 230(C).
    5. Leitner, Johannes & Leopold-Wildburger, Ulrike, 2011. "Experiments on forecasting behavior with several sources of information - A review of the literature," European Journal of Operational Research, Elsevier, vol. 213(3), pages 459-469, September.
    6. De Baets, Shari & Harvey, Nigel, 2018. "Forecasting from time series subject to sporadic perturbations: Effectiveness of different types of forecasting support," International Journal of Forecasting, Elsevier, vol. 34(2), pages 163-180.
    7. Baecke, Philippe & De Baets, Shari & Vanderheyden, Karlien, 2017. "Investigating the added value of integrating human judgement into statistical demand forecasting systems," International Journal of Production Economics, Elsevier, vol. 191(C), pages 85-96.
    8. Hewage, Harsha Chamara & Perera, H. Niles & De Baets, Shari, 2022. "Forecast adjustments during post-promotional periods," European Journal of Operational Research, Elsevier, vol. 300(2), pages 461-472.
    9. Van den Broeke, Maud & De Baets, Shari & Vereecke, Ann & Baecke, Philippe & Vanderheyden, Karlien, 2019. "Judgmental forecast adjustments over different time horizons," Omega, Elsevier, vol. 87(C), pages 34-45.
    10. Syntetos, Aris A. & Kholidasari, Inna & Naim, Mohamed M., 2016. "The effects of integrating management judgement into OUT levels: In or out of context?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 853-863.
    11. Song, Haiyan & Gao, Bastian Z. & Lin, Vera S., 2013. "Combining statistical and judgmental forecasts via a web-based tourism demand forecasting system," International Journal of Forecasting, Elsevier, vol. 29(2), pages 295-310.
    12. F Caniato & M Kalchschmidt & S Ronchi, 2011. "Integrating quantitative and qualitative forecasting approaches: organizational learning in an action research case," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 62(3), pages 413-424, March.
    13. Lawrence, Michael & Goodwin, Paul & O'Connor, Marcus & Onkal, Dilek, 2006. "Judgmental forecasting: A review of progress over the last 25 years," International Journal of Forecasting, Elsevier, vol. 22(3), pages 493-518.
    14. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    15. Eksoz, Can & Mansouri, S. Afshin & Bourlakis, Michael & Önkal, Dilek, 2019. "Judgmental adjustments through supply integration for strategic partnerships in food chains," Omega, Elsevier, vol. 87(C), pages 20-33.
    16. Eksoz, Can & Mansouri, S. Afshin & Bourlakis, Michael, 2014. "Collaborative forecasting in the food supply chain: A conceptual framework," International Journal of Production Economics, Elsevier, vol. 158(C), pages 120-135.
    17. Petropoulos, Fotios & Fildes, Robert & Goodwin, Paul, 2016. "Do ‘big losses’ in judgmental adjustments to statistical forecasts affect experts’ behaviour?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 842-852.
    18. Sroginis, Anna & Fildes, Robert & Kourentzes, Nikolaos, 2023. "Use of contextual and model-based information in adjusting promotional forecasts," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1177-1191.
    19. Fildes, Robert & Goodwin, Paul & Onkal, Dilek, 2015. "Information use in supply chain forecasting," MPRA Paper 66034, University Library of Munich, Germany.
    20. Fildes, Robert & Ma, Shaohui & Kolassa, Stephan, 2022. "Retail forecasting: Research and practice," International Journal of Forecasting, Elsevier, vol. 38(4), pages 1283-1318.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:33:y:2017:i:1:p:298-313. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.