IDEAS home Printed from https://ideas.repec.org/p/mil/wpdepa/2011-08.html
   My bibliography  Save this paper

Forecast evaluation in call centers: combined forecasts, flexible loss functions and economic criteria

Author

Listed:
  • Andrea BASTIANIN

    ()

  • Marzio GALEOTTI

    ()

  • Matteo MANERA

    ()

Abstract

Accurate forecasts of incoming calls are crucial to optimal staffing decisions in call centers. This paper evaluates a wide range of models and forecast combination techniques by means of statistical and economic criteria. Relative to the previous literature, this paper is novel in several respects. In particular, the statistical evaluation of competing models is carried out by using a flexible loss function as input to pairwise and joint forecast diagnostic checks. Informative rankings across alternative single models and different groups of models are obtained. Moreover, models are evaluated from the perspective of a manager, who needs reliable forecasts to dimension the call center. Money metrics of forecasting performance are computed, which are based on the economic value of information and the certainty equivalent.

Suggested Citation

  • Andrea BASTIANIN & Marzio GALEOTTI & Matteo MANERA, 2011. "Forecast evaluation in call centers: combined forecasts, flexible loss functions and economic criteria," Departmental Working Papers 2011-08, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
  • Handle: RePEc:mil:wpdepa:2011-08
    as

    Download full text from publisher

    File URL: http://wp.demm.unimi.it/files/wp/2011/DEMM-2011_008wp.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Tych, Wlodek & Pedregal, Diego J. & Young, Peter C. & Davies, John, 2002. "An unobserved component model for multi-rate forecasting of telephone call demand: the design of a forecasting support system," International Journal of Forecasting, Elsevier, vol. 18(4), pages 673-695.
    2. Garratt A. & Lee K. & Pesaran M.H. & Shin Y., 2003. "Forecast Uncertainties in Macroeconomic Modeling: An Application to the U.K. Economy," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 829-838, January.
    3. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Publishing House "SINERGIA PRESS", vol. 33(1), pages 125-132.
    4. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    5. Skouras, Spyros, 2007. "Decisionmetrics: A decision-based approach to econometric modelling," Journal of Econometrics, Elsevier, vol. 137(2), pages 414-440, April.
    6. Norman R. Swanson & Halbert White, 1997. "A Model Selection Approach To Real-Time Macroeconomic Forecasting Using Linear Models And Artificial Neural Networks," The Review of Economics and Statistics, MIT Press, vol. 79(4), pages 540-550, November.
    7. Franses, Philip Hans & van Dijk, Dick, 2005. "The forecasting performance of various models for seasonality and nonlinearity for quarterly industrial production," International Journal of Forecasting, Elsevier, vol. 21(1), pages 87-102.
    8. Taylor, James W., 2010. "Exponentially weighted methods for forecasting intraday time series with multiple seasonal cycles," International Journal of Forecasting, Elsevier, vol. 26(4), pages 627-646, October.
    9. Edward E. Gbur & Robert A. Collins, 1989. "A Small-Sample Comparison of Estimators in the EU-MGF Approach to Decision Making," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 71(1), pages 202-210.
    10. James W. Taylor, 2012. "Density Forecasting of Intraday Call Center Arrivals Using Models Based on Exponential Smoothing," Management Science, INFORMS, vol. 58(3), pages 534-549, March.
    11. Robert F. Engle & Jeffrey R. Russell, 1998. "Autoregressive Conditional Duration: A New Model for Irregularly Spaced Transaction Data," Econometrica, Econometric Society, vol. 66(5), pages 1127-1162, September.
    12. Dorfman, Jeffrey H. & McIntosh, Christopher S., 1997. "Economic Criteria For Evaluating Commodity Price Forecasts," Journal of Agricultural and Applied Economics, Southern Agricultural Economics Association, vol. 29(02), December.
    13. Taylor, James W., 2008. "Exponentially weighted information criteria for selecting among forecasting models," International Journal of Forecasting, Elsevier, vol. 24(3), pages 513-524.
    14. Ivana Komunjer & Michael T. Owyang, 2012. "Multivariate Forecast Evaluation and Rationality Testing," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1066-1080, November.
    15. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    16. Zeng Tian & Swanson Norman R., 1998. "Predictive Evaluation of Econometric Forecasting Models in Commodity Futures Markets," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 2(4), pages 1-21, January.
    17. Andrea BASTIANIN & Marzio GALEOTTI & Matteo MANERA, 2011. "Forecast evaluation in call centers: combined forecasts, flexible loss functions and economic criteria," Departmental Working Papers 2011-08, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    18. Fildes, Robert & Kumar, V., 2002. "Telecommunications demand forecasting--a review," International Journal of Forecasting, Elsevier, vol. 18(4), pages 489-522.
    19. Robert Neil Collender & James A. Chalfant, 1986. "An Alternative Approach to Decisions under Uncertainty Using the Empirical Moment-Generating Function," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 68(3), pages 727-731.
    20. Robert Jung & A. Tremayne, 2011. "Useful models for time series of counts or simply wrong ones?," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 95(1), pages 59-91, March.
    21. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
    22. Taylor, James W., 2010. "Reply to the discussion of: Exponentially weighted methods for forecasting intraday time series with multiple seasonal cycles," International Journal of Forecasting, Elsevier, vol. 26(4), pages 658-660, October.
    23. Clemen, Robert T., 1989. "Combining forecasts: A review and annotated bibliography," International Journal of Forecasting, Elsevier, vol. 5(4), pages 559-583.
    24. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    25. Robert Engle, 2002. "New frontiers for arch models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 17(5), pages 425-446.
    26. James W. Taylor, 2008. "A Comparison of Univariate Time Series Methods for Forecasting Intraday Arrivals at a Call Center," Management Science, INFORMS, vol. 54(2), pages 253-265, February.
    27. Lawrence Brown & Noah Gans & Avishai Mandelbaum & Anat Sakov & Haipeng Shen & Sergey Zeltyn & Linda Zhao, 2005. "Statistical Analysis of a Telephone Call Center: A Queueing-Science Perspective," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 36-50, March.
    28. Bruce H. Andrews & Shawn M. Cunningham, 1995. "L. L. Bean Improves Call-Center Forecasting," Interfaces, INFORMS, vol. 25(6), pages 1-13, December.
    29. Bianchi, Lisa & Jarrett, Jeffrey & Choudary Hanumara, R., 1998. "Improving forecasting for telemarketing centers by ARIMA modeling with intervention," International Journal of Forecasting, Elsevier, vol. 14(4), pages 497-504, December.
    30. G. Elliott & C. Granger & A. Timmermann (ed.), 2006. "Handbook of Economic Forecasting," Handbook of Economic Forecasting, Elsevier, edition 1, volume 1, number 1, May.
    31. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    32. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    33. repec:pal:jorsoc:v:53:y:2002:i:9:d:10.1057_palgrave.jors.2601415 is not listed on IDEAS
    34. Leitch, Gordon & Tanner, J Ernest, 1991. "Economic Forecast Evaluation: Profits versus the Conventional Error Measures," American Economic Review, American Economic Association, vol. 81(3), pages 580-590, June.
    35. Graham Elliott & Ivana Komunjer & Allan Timmermann, 2008. "Biases in Macroeconomic Forecasts: Irrationality or Asymmetric Loss?," Journal of the European Economic Association, MIT Press, vol. 6(1), pages 122-157, March.
    36. Ibrahim, Rouba & Ye, Han & L’Ecuyer, Pierre & Shen, Haipeng, 2016. "Modeling and forecasting call center arrivals: A literature survey and a case study," International Journal of Forecasting, Elsevier, vol. 32(3), pages 865-874.
    37. Haipeng Shen & Jianhua Z. Huang, 2008. "Interday Forecasting and Intraday Updating of Call Center Arrivals," Manufacturing & Service Operations Management, INFORMS, vol. 10(3), pages 391-410, July.
    38. Graham Elliott & Allan Timmermann & Ivana Komunjer, 2005. "Estimation and Testing of Forecast Rationality under Flexible Loss," Review of Economic Studies, Oxford University Press, vol. 72(4), pages 1107-1125.
    39. Howard E. Thompson & George C. Tiao, 1971. "Analysis of Telephone Data: A Case Study of Forecasting Seasonal Time Series," Bell Journal of Economics, The RAND Corporation, vol. 2(2), pages 515-541, Autumn.
    40. Gardner, Everette Jr., 2006. "Exponential smoothing: The state of the art--Part II," International Journal of Forecasting, Elsevier, vol. 22(4), pages 637-666.
    41. David E. Rapach & Jack K. Strauss, 2008. "Forecasting US employment growth using forecast combining methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 27(1), pages 75-93.
    42. Swanson, Norman R & White, Halbert, 1995. "A Model-Selection Approach to Assessing the Information in the Term Structure Using Linear Models and Artificial Neural Networks," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 265-275, July.
    43. Nabil Channouf & Pierre L’Ecuyer & Armann Ingolfsson & Athanassios Avramidis, 2007. "The application of forecasting techniques to modeling emergency medical system calls in Calgary, Alberta," Health Care Management Science, Springer, vol. 10(1), pages 25-45, February.
    44. Noah Gans & Ger Koole & Avishai Mandelbaum, 2003. "Telephone Call Centers: Tutorial, Review, and Research Prospects," Manufacturing & Service Operations Management, INFORMS, vol. 5(2), pages 79-141, September.
    45. Jeff Fleming, 2001. "The Economic Value of Volatility Timing," Journal of Finance, American Finance Association, vol. 56(1), pages 329-352, February.
    46. Elamin H. Elbasha, 2005. "Risk aversion and uncertainty in cost-effectiveness analysis: the expected-utility, moment-generating function approach," Health Economics, John Wiley & Sons, Ltd., vol. 14(5), pages 457-470.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrea BASTIANIN & Marzio GALEOTTI & Matteo MANERA, 2011. "Forecast evaluation in call centers: combined forecasts, flexible loss functions and economic criteria," Departmental Working Papers 2011-08, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    2. Andrea Bastianin & Marzio Galeotti & Matteo Manera, 2011. "Forecast Evaluation in Call Centers: Combined Forecasts, Flexible Loss Functions and Economic Criteria," Working Papers 20110301, Università degli Studi di Milano-Bicocca, Dipartimento di Statistica.

    More about this item

    Keywords

    Combining forecasts; Decision making; Loss function; Seasonality;

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • M15 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - IT Management

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mil:wpdepa:2011-08. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (DEMM Working Papers). General contact details of provider: http://edirc.repec.org/data/damilit.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.