IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/53579.html
   My bibliography  Save this paper

Golden Rule of Forecasting: Be conservative

Author

Listed:
  • Armstrong, J. Scott
  • Green, Kesten C.
  • Graefe, Andreas

Abstract

This paper proposes a unifying theory of forecasting in the form of a Golden Rule of Forecasting. The Golden Rule is to be conservative. A conservative forecast is consistent with cumulative knowledge about the present and the past. To be conservative, forecasters must seek all knowledge relevant to the problem, and use methods that have been validated for the situation. A checklist of 28 guidelines is provided to implement the Golden Rule. This article’s review of research found 150 experimental comparisons; all supported the guidelines. The average error reduction from following a single guideline (compared to common practice) was 28 percent. The Golden Rule Checklist helps forecasters to forecast more accurately, especially when the situation is uncertain and complex, and when bias is likely. Non-experts who know the Golden Rule can identify dubious forecasts quickly and inexpensively. To date, ignorance of research findings, bias, sophisticated statistical procedures, and the proliferation of big data have led forecasters to violate the Golden Rule. As a result, despite major advances in forecasting methods, evidence that forecasting practice has improved over the past half-century is lacking.

Suggested Citation

  • Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2014. "Golden Rule of Forecasting: Be conservative," MPRA Paper 53579, University Library of Munich, Germany.
  • Handle: RePEc:pra:mprapa:53579
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/53579/1/MPRA_paper_53579.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bunn, Derek W. & Vassilopoulos, Angelos I., 1999. "Comparison of seasonal estimation methods in multi-item short-term forecasting," International Journal of Forecasting, Elsevier, vol. 15(4), pages 431-443, October.
    2. Fildes, Robert & Hibon, Michele & Makridakis, Spyros & Meade, Nigel, 1998. "Generalising about univariate forecasting methods: further empirical evidence," International Journal of Forecasting, Elsevier, vol. 14(3), pages 339-358, September.
    3. Schnaars, Steven P. & Bavuso, R. Joseph, 1986. "Extrapolation models on very short-term forecasts," Journal of Business Research, Elsevier, vol. 14(1), pages 27-36, February.
    4. Graefe, Andreas & Armstrong, J. Scott, 2011. "Comparing face-to-face meetings, nominal groups, Delphi and prediction markets on an estimation task," International Journal of Forecasting, Elsevier, vol. 27(1), pages 183-195, January.
    5. Richard P. Larrick & Jack B. Soll, 2006. "Erratum--Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(2), pages 309-310, February.
    6. Wright, Malcolm J. & Stern, Philip, 2015. "Forecasting new product trial with analogous series," Journal of Business Research, Elsevier, vol. 68(8), pages 1732-1738.
    7. Vokurka, Robert J. & Flores, Benito E. & Pearce, Stephen L., 1996. "Automatic feature identification and graphical support in rule-based forecasting: a comparison," International Journal of Forecasting, Elsevier, vol. 12(4), pages 495-512, December.
    8. Green, Kesten C. & Armstrong, J. Scott, 2007. "Structured analogies for forecasting," International Journal of Forecasting, Elsevier, vol. 23(3), pages 365-376.
    9. Gene K. Groff, 1973. "Empirical Comparison of Models for Short Range Forecasting," Management Science, INFORMS, vol. 20(1), pages 22-31, September.
    10. Miller, Don M. & Williams, Dan, 2004. "Damping seasonal factors: Shrinkage estimators for the X-12-ARIMA program," International Journal of Forecasting, Elsevier, vol. 20(4), pages 529-549.
    11. Booth, Heather, 2006. "Demographic forecasting: 1980 to 2005 in review," International Journal of Forecasting, Elsevier, vol. 22(3), pages 547-581.
    12. Gardner, Everette S. & Anderson, Elizabeth A., 1997. "Focus forecasting reconsidered," International Journal of Forecasting, Elsevier, vol. 13(4), pages 501-508, December.
    13. Bent Flyvbjerg, 2013. "Quality Control and Due Diligence in Project Management: Getting Decisions Right by Taking the Outside View," Papers 1302.2544, arXiv.org.
    14. Graefe, Andreas & Armstrong, J. Scott, 2008. "Forecasting Elections from Voters’ Perceptions of Candidates’ Positions on Issues and Policies," MPRA Paper 9829, University Library of Munich, Germany.
    15. Dangerfield, Byron J. & Morris, John S., 1992. "Top-down or bottom-up: Aggregate versus disaggregate extrapolations," International Journal of Forecasting, Elsevier, vol. 8(2), pages 233-241, October.
    16. Makridakis, Spyros & Chatfield, Chris & Hibon, Michele & Lawrence, Michael & Mills, Terence & Ord, Keith & Simmons, LeRoy F., 1993. "The M2-competition: A real-time judgmentally based forecasting study," International Journal of Forecasting, Elsevier, vol. 9(1), pages 5-22, April.
    17. Armstrong, J. Scott, 1970. "An Application of Econometric Models to International Marketing," MPRA Paper 81698, University Library of Munich, Germany.
    18. Fildes, Robert & Goodwin, Paul & Lawrence, Michael & Nikolopoulos, Konstantinos, 2009. "Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning," International Journal of Forecasting, Elsevier, vol. 25(1), pages 3-23.
    19. Collopy, Fred & Armstrong, J. Scott, 1992. "Expert opinions about extrapolation and the mystery of the overlooked discontinuities," International Journal of Forecasting, Elsevier, vol. 8(4), pages 575-582, December.
    20. Armstrong, J Scott, 1978. "Forecasting with Econometric Methods: Folklore versus Fact," The Journal of Business, University of Chicago Press, vol. 51(4), pages 549-564, October.
    21. Armstrong, J. Scott & Green, Kesten C., 2013. "Effects of corporate social responsibility and irresponsibility policies," Journal of Business Research, Elsevier, vol. 66(10), pages 1922-1927.
    22. Rianne Legerstee & Philip Hans Franses, 2014. "Do Experts’ SKU Forecasts Improve after Feedback?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 33(1), pages 69-79, January.
    23. Ziliak, Stephen T. & McCloskey, Deirdre N., 2004. "Size matters: the standard error of regressions in the American Economic Review," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 33(5), pages 527-546, November.
    24. Victor Zarnowitz, 1967. "An Appraisal of Short-Term Economic Forecasts," NBER Books, National Bureau of Economic Research, Inc, number zarn67-1, March.
    25. Nico Keilman, 2008. "European Demographic Forecasts Have Not Become More Accurate Over the Past 25 Years," Population and Development Review, The Population Council, Inc., vol. 34(1), pages 137-153, March.
    26. Graefe, Andreas & Armstrong, J. Scott & Jones, Randall J. & Cuzán, Alfred G., 2014. "Combining forecasts: An application to elections," International Journal of Forecasting, Elsevier, vol. 30(1), pages 43-54.
    27. Armstrong, J. Scott & Collopy, Fred & Yokum, J. Thomas, 2005. "Decomposition by causal forces: a procedure for forecasting complex time series," International Journal of Forecasting, Elsevier, vol. 21(1), pages 25-36.
    28. Nelson, Charles R, 1972. "The Prediction Performance of the FRB-MIT-PENN Model of the U.S. Economy," American Economic Review, American Economic Association, vol. 62(5), pages 902-917, December.
    29. Kesten C. Green & J. Scott Armstrong, 2007. "Global Warming: Forecasts by Scientists Versus Scientific Forecasts," Energy & Environment, , vol. 18(7), pages 997-1021, December.
    30. Gorr, Wilpen & Olligschlaeger, Andreas & Thompson, Yvonne, 2003. "Short-term forecasting of crime," International Journal of Forecasting, Elsevier, vol. 19(4), pages 579-594.
    31. Allen, P. Geoffrey, 1994. "Economic forecasting in agriculture," International Journal of Forecasting, Elsevier, vol. 10(1), pages 81-135, June.
    32. Green, Kesten C. & Armstrong, J. Scott, 2015. "Simple versus complex forecasting: The evidence," Journal of Business Research, Elsevier, vol. 68(8), pages 1678-1685.
    33. Armstrong, J. Scott, 2006. "Findings from evidence-based forecasting: Methods for reducing forecast error," International Journal of Forecasting, Elsevier, vol. 22(3), pages 583-598.
    34. Tessier, Thomas H. & Armstrong, J. Scott, 2015. "Decomposition of time-series by level and change," Journal of Business Research, Elsevier, vol. 68(8), pages 1755-1758.
    35. Everette S. Gardner, Jr. & Ed. Mckenzie, 1985. "Forecasting Trends in Time Series," Management Science, INFORMS, vol. 31(10), pages 1237-1246, October.
    36. Schnaars, Steven P., 1986. "A comparison of extrapolation models on yearly sales forecasts," International Journal of Forecasting, Elsevier, vol. 2(1), pages 71-85.
    37. David E. Runkle, 1998. "Revisionist history: how data revisions distort economic policy research," Quarterly Review, Federal Reserve Bank of Minneapolis, vol. 22(Fall), pages 3-12.
    38. Nada R. Sanders & Karl B. Manrodt, 1994. "Forecasting Practices in US Corporations: Survey Results," Interfaces, INFORMS, vol. 24(2), pages 92-100, April.
    39. Withycombe, Richard, 1989. "Forecasting with combined seasonal indices," International Journal of Forecasting, Elsevier, vol. 5(4), pages 547-552.
    40. Fildes, Robert & Petropoulos, Fotios, 2015. "Simple versus complex selection rules for forecasting many time series," Journal of Business Research, Elsevier, vol. 68(8), pages 1692-1701.
    41. Philip Hans Franses & Rianne Legerstee, 2010. "Do experts' adjustments on model-based SKU-level forecasts improve forecast quality?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 29(3), pages 331-340.
    42. Carson, Richard T. & Cenesizoglu, Tolga & Parker, Roger, 2011. "Forecasting (aggregate) demand for US commercial air travel," International Journal of Forecasting, Elsevier, vol. 27(3), pages 923-941, July.
    43. Armstrong, J. Scott & Graefe, Andreas, 2011. "Predicting elections from biographical information about candidates: A test of the index method," Journal of Business Research, Elsevier, vol. 64(7), pages 699-706, July.
    44. J. Scott Armstrong & Kesten C. Green & Willie Soon, 2008. "Polar Bear Population Forecasts: A Public-Policy Forecasting Audit," Interfaces, INFORMS, vol. 38(5), pages 382-405, October.
    45. Soyer, Emre & Hogarth, Robin M., 2012. "The illusion of predictability: How regression statistics mislead experts," International Journal of Forecasting, Elsevier, vol. 28(3), pages 695-711.
    46. Kinney, Wr, 1971. "Predicting Earnings - Entity Versus Subentity Data," Journal of Accounting Research, Wiley Blackwell, vol. 9(1), pages 127-136.
    47. Fred Collopy & J. Scott Armstrong, 1992. "Rule-Based Forecasting: Development and Validation of an Expert Systems Approach to Combining Time Series Extrapolations," Management Science, INFORMS, vol. 38(10), pages 1394-1414, October.
    48. Harvey, Nigel, 1995. "Why Are Judgments Less Consistent in Less Predictable Task Situations?," Organizational Behavior and Human Decision Processes, Elsevier, vol. 63(3), pages 247-263, September.
    49. Goodwin, Paul, 2000. "Improving the voluntary integration of statistical forecasts and judgment," International Journal of Forecasting, Elsevier, vol. 16(1), pages 85-99.
    50. Nikolopoulos, Konstantinos & Litsa, Akrivi & Petropoulos, Fotios & Bougioukos, Vasileios & Khammash, Marwan, 2015. "Relative performance of methods for forecasting special events," Journal of Business Research, Elsevier, vol. 68(8), pages 1785-1791.
    51. Green, Kesten C., 2005. "Game theory, simulated interaction, and unaided judgement for forecasting decisions in conflicts: Further evidence," International Journal of Forecasting, Elsevier, vol. 21(3), pages 463-472.
    52. Robert Fildes & Paul Goodwin, 2007. "Against Your Better Judgment? How Organizations Can Improve Their Use of Management Judgment in Forecasting," Interfaces, INFORMS, vol. 37(6), pages 570-576, December.
    53. Graefe, Andreas, 2015. "Improving forecasts using equally weighted predictors," Journal of Business Research, Elsevier, vol. 68(8), pages 1792-1799.
    54. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    55. Gardner, Everette Jr. & Anderson-Fletcher, Elizabeth A. & Wicks, Angela M., 2001. "Further results on focus forecasting vs. exponential smoothing," International Journal of Forecasting, Elsevier, vol. 17(2), pages 287-293.
    56. Armstrong, J. Scott & Andress, James G., 1970. "Exploratory Analysis of Marketing Data: Trees vs. Regression," MPRA Paper 81668, University Library of Munich, Germany.
    57. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    58. Donna F. Davis & John T. Mentzer & Teresa M. Mccarthy & Susan L. Golicic, 2006. "The evolution of sales forecasting management: a 20-year longitudinal study of forecasting practices," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 25(5), pages 303-324.
    59. Goodwin, Paul, 2015. "When simple alternatives to Bayes formula work well: Reducing the cognitive load when updating probability forecasts," Journal of Business Research, Elsevier, vol. 68(8), pages 1686-1691.
    60. Miller, Tan & Liberatore, Matthew, 1993. "Seasonal exponential smoothing with damped trends : An application for production planning," International Journal of Forecasting, Elsevier, vol. 9(4), pages 509-515, December.
    61. Green, Kesten C. & Armstrong, J. Scott & Soon, Willie, 2009. "Validity of climate change forecasting for public policy decision making," International Journal of Forecasting, Elsevier, vol. 25(4), pages 826-832, October.
    62. Everette S. Gardner, 1984. "The Strange Case of the Lagging Forecasts," Interfaces, INFORMS, vol. 14(3), pages 47-50, June.
    63. J. Scott Armstrong, 2012. "Predicting Job Performance: The Moneyball Factor," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 25, pages 31-34, Spring.
    64. Chen, Huijing & Boylan, John E., 2008. "Empirical evidence on individual, group and shrinkage seasonal indices," International Journal of Forecasting, Elsevier, vol. 24(3), pages 525-534.
    65. Armstrong, J. Scott & Collopy, Fred, 1992. "Error measures for generalizing about forecasting methods: Empirical comparisons," International Journal of Forecasting, Elsevier, vol. 8(1), pages 69-80, June.
    66. Richard P. Larrick & Jack B. Soll, 2006. "Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(1), pages 111-127, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    2. Green, Kesten C. & Armstrong, J. Scott, 2015. "Simple versus complex forecasting: The evidence," Journal of Business Research, Elsevier, vol. 68(8), pages 1678-1685.
    3. Armstrong, J. Scott, 2006. "Findings from evidence-based forecasting: Methods for reducing forecast error," International Journal of Forecasting, Elsevier, vol. 22(3), pages 583-598.
    4. Perera, H. Niles & Hurley, Jason & Fahimnia, Behnam & Reisi, Mohsen, 2019. "The human factor in supply chain forecasting: A systematic review," European Journal of Operational Research, Elsevier, vol. 274(2), pages 574-600.
    5. Gardner, Everette Jr., 2006. "Exponential smoothing: The state of the art--Part II," International Journal of Forecasting, Elsevier, vol. 22(4), pages 637-666.
    6. R Fildes & K Nikolopoulos & S F Crone & A A Syntetos, 2008. "Forecasting and operational research: a review," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(9), pages 1150-1172, September.
    7. Arvan, Meysam & Fahimnia, Behnam & Reisi, Mohsen & Siemsen, Enno, 2019. "Integrating human judgement into quantitative forecasting methods: A review," Omega, Elsevier, vol. 86(C), pages 237-252.
    8. Leitner, Johannes & Leopold-Wildburger, Ulrike, 2011. "Experiments on forecasting behavior with several sources of information - A review of the literature," European Journal of Operational Research, Elsevier, vol. 213(3), pages 459-469, September.
    9. Makridakis, Spyros & Hyndman, Rob J. & Petropoulos, Fotios, 2020. "Forecasting in social settings: The state of the art," International Journal of Forecasting, Elsevier, vol. 36(1), pages 15-28.
    10. Song, Haiyan & Gao, Bastian Z. & Lin, Vera S., 2013. "Combining statistical and judgmental forecasts via a web-based tourism demand forecasting system," International Journal of Forecasting, Elsevier, vol. 29(2), pages 295-310.
    11. von der Gracht, Heiko A. & Hommel, Ulrich & Prokesch, Tobias & Wohlenberg, Holger, 2016. "Testing weighting approaches for forecasting in a Group Wisdom Support System environment," Journal of Business Research, Elsevier, vol. 69(10), pages 4081-4094.
    12. A A Syntetos & J E Boylan & S M Disney, 2009. "Forecasting for inventory planning: a 50-year review," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(1), pages 149-160, May.
    13. Eksoz, Can & Mansouri, S. Afshin & Bourlakis, Michael, 2014. "Collaborative forecasting in the food supply chain: A conceptual framework," International Journal of Production Economics, Elsevier, vol. 158(C), pages 120-135.
    14. De Gooijer, Jan G. & Hyndman, Rob J., 2006. "25 years of time series forecasting," International Journal of Forecasting, Elsevier, vol. 22(3), pages 443-473.
    15. Jan G. De Gooijer & Rob J. Hyndman, 2005. "25 Years of IIF Time Series Forecasting: A Selective Review," Monash Econometrics and Business Statistics Working Papers 12/05, Monash University, Department of Econometrics and Business Statistics.
    16. Petropoulos, Fotios & Fildes, Robert & Goodwin, Paul, 2016. "Do ‘big losses’ in judgmental adjustments to statistical forecasts affect experts’ behaviour?," European Journal of Operational Research, Elsevier, vol. 249(3), pages 842-852.
    17. Abolghasemi, Mahdi & Hurley, Jason & Eshragh, Ali & Fahimnia, Behnam, 2020. "Demand forecasting in the presence of systematic events: Cases in capturing sales promotions," International Journal of Production Economics, Elsevier, vol. 230(C).
    18. Deepa Mishra & Angappa Gunasekaran & Thanos Papadopoulos & Stephen J. Childe, 2018. "Big Data and supply chain management: a review and bibliometric analysis," Annals of Operations Research, Springer, vol. 270(1), pages 313-336, November.
    19. Webby, Richard & O'Connor, Marcus, 1996. "Judgemental and statistical time series forecasting: a review of the literature," International Journal of Forecasting, Elsevier, vol. 12(1), pages 91-118, March.
    20. Wang, Xiaoqian & Hyndman, Rob J. & Li, Feng & Kang, Yanfei, 2023. "Forecast combinations: An over 50-year review," International Journal of Forecasting, Elsevier, vol. 39(4), pages 1518-1547.

    More about this item

    Keywords

    accuracy; analytics; bias; big data; causal forces; causal models; combining; complexity; contrary series; damped trends; decision-making; decomposition; Delphi; ethics; extrapolation; inconsistent trends; index method; judgmental bootstrapping; judgmental forecasting; nowcasting; regression; risk; shrinkage; simplicity; stepwise regression; structured analogies;
    All these keywords.

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C2 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables
    • C3 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables
    • C4 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics
    • C5 - Mathematical and Quantitative Methods - - Econometric Modeling
    • C8 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • K2 - Law and Economics - - Regulation and Business Law

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:53579. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.