Accuracy versus Profitability

In a perfect world, forecast accuracy would be a reliable guide to the monetary value of the forecast; more accurate forecasts would be more profitable. But who lives in a perfect world? In real life, accuracy and monetary value do not necessarily correspond and, as Roy shows, can move in opposite directions. His examples reveal disconnections between accuracy and profitability among financial forecasters and how the same might occur among sales forecasters. To better explain this situation, Roy distinguishes between (a) the small movements in a target variable that occur ordinarily, and (b) the large changes that occur only occasionally. His point is similar to that made by Wil Gorr’s accompanying article in this issue on forecasting exceptional demands (p. 22): not only might we need different models to forecast exceptional vs. ordinary demand, but also different forecast-accuracy metrics. This clearly is a lesson worth attending to. Demand forecasters interested in devising appropriate forecasting performance metrics should also read Paul Goodwin’s Hot New Research column “Taking Stock: Assessing the True Cost of Forecast Errors,” in Foresight’s Fall 2009 issue. Copyright International Institute of Forecasters, 2011