IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v25y2009i1p62-73.html
   My bibliography  Save this article

On the importance of verifying forecasting results

Author

Listed:
  • Yalta, A. Talha
  • Jenal, Olaf

Abstract

We discuss the various sources of error in numerical computations with the use of examples from the literature relevant to time series analysis. We also submit a case where, by manual verification, we were able to discover a plausible forecast to be erroneous due to a number of software flaws in the XLSTAT addin for Microsoft Excel. Furthermore, after discussing the alternative techniques for implementing the ARIMA (AutoRegressive Integrated Moving Average) methodology on a computer, we show that different approaches can cause considerable discrepancies in the results across different programs, and even within a single software system.

Suggested Citation

  • Yalta, A. Talha & Jenal, Olaf, 2009. "On the importance of verifying forecasting results," International Journal of Forecasting, Elsevier, vol. 25(1), pages 62-73.
    • Handle: RePEc:eee:intfor:v:25:y:2009:i:1:p:62-73
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169-2070(08)00068-X
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Nada R. Sanders & Karl B. Manrodt, 2003. "Forecasting Software in Practice: Use, Satisfaction, and Performance," Interfaces, INFORMS, vol. 33(5), pages 90-93, October.
    2. Christian Kleiber & Achim Zeileis, 2005. "Validating multiple structural change models-a case study," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(5), pages 685-690.
    3. Sawitzki, Gunther, 1994. "Testing numerical reliability of data analysis systems," Computational Statistics & Data Analysis, Elsevier, vol. 18(2), pages 269-286, September.
    4. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.
    5. Newbold, Paul & Agiakloglou, Christos & Miller, John, 1994. "Adventures with ARIMA software," International Journal of Forecasting, Elsevier, vol. 10(4), pages 573-581, December.
    6. Ernst R. Berndt & Bronwyn H. Hall & Robert E. Hall & Jerry A. Hausman, 1974. "Estimation and Inference in Nonlinear Structural Models," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 3, number 4, pages 653-665, National Bureau of Economic Research, Inc.
    7. A. Talha Yalta, 2010. "The Accuracy of Statistical Distributions in Microsoft (R) Excel 2007," Working Papers 1006, TOBB University of Economics and Technology, Department of Economics.
    8. McCullough, B D, 1999. "Econometric Software Reliability: EViews, LIMDEP, SHAZAM and TSP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(2), pages 191-202, March-Apr.
    9. C. R. McKenzie & Sumiko Takaoka, 2007. "EViews 5.1," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(6), pages 1145-1152.
    10. McCullough, B.D. & Heiser, David A., 2008. "On the accuracy of statistical procedures in Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4570-4578, June.
    11. D. McCullough, B. & Wilson, Berry, 2002. "On the accuracy of statistical procedures in Microsoft Excel 2000 and Excel XP," Computational Statistics & Data Analysis, Elsevier, vol. 40(4), pages 713-721, October.
    12. McCullough, B. D., 2000. "Is it safe to assume that software is accurate?," International Journal of Forecasting, Elsevier, vol. 16(3), pages 349-357.
    13. McCullough, B.D., 2008. "Microsoft Excel's 'Not The Wichmann-Hill' random number generators," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4587-4593, June.
    14. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    15. McCullough, B. D. & Wilson, Berry, 1999. "On the accuracy of statistical procedures in Microsoft Excel 97," Computational Statistics & Data Analysis, Elsevier, vol. 31(1), pages 27-37, July.
    16. McCullough, B.D. & Wilson, Berry, 2005. "On the accuracy of statistical procedures in Microsoft Excel 2003," Computational Statistics & Data Analysis, Elsevier, vol. 49(4), pages 1244-1252, June.
    17. H. D. Vinod & B. D. McCullough, 1999. "The Numerical Reliability of Econometric Software," Journal of Economic Literature, American Economic Association, vol. 37(2), pages 633-665, June.
    18. Kusters, Ulrich & McCullough, B.D. & Bell, Michael, 2006. "Forecasting software: Past, present and future," International Journal of Forecasting, Elsevier, vol. 22(3), pages 599-615.
    19. H. D. Vinod & B. D. McCullough, 1999. "Corrigenda: The Numerical Reliability of Econometric Software," Journal of Economic Literature, American Economic Association, vol. 37(4), pages 1565-1565, December.
    20. Jushan Bai & Pierre Perron, 2003. "Computation and analysis of multiple structural change models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 1-22.
    21. Yalta, A. Talha, 2008. "The accuracy of statistical distributions in Microsoft® Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4579-4586, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aurelia Rybak & Aleksandra Rybak, 2021. "The Impact of the COVID-19 Pandemic on Gaseous and Solid Air Pollutants Concentrations and Emissions in the EU, with Particular Emphasis on Poland," Energies, MDPI, vol. 14(11), pages 1-25, June.
    2. A. Talha Yalta & A. Yasemin Yalta, 2009. "Wilkinson Tests and gretl," EHUCHAPS, in: Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), Econometrics with gretl. Proceedings of the gretl Conference 2009, edition 1, chapter 16, pages 243-251, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales.
    3. Guy Mélard, 2014. "On the accuracy of statistical procedures in Microsoft Excel 2010," Computational Statistics, Springer, vol. 29(5), pages 1095-1128, October.
    4. Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), 2009. "Econometrics with gretl. Proceedings of the gretl Conference 2009," UPV/EHU Books, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales, edition 1, number 01.
    5. Hargreaves, Bruce R. & McWilliams, Thomas P., 2010. "Polynomial Trendline function flaws in Microsoft Excel," Computational Statistics & Data Analysis, Elsevier, vol. 54(4), pages 1190-1196, April.
    6. A. Yalta & A. Yalta, 2010. "Should Economists Use Open Source Software for Doing Research?," Computational Economics, Springer;Society for Computational Economics, vol. 35(4), pages 371-394, April.
    7. Yalta, A. Talha & Schreiber, Sven, 2012. "Random Number Generation in gretl," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 50(c01).
    8. McCullough, Bruce D. & Yalta, A. Talha, 2013. "Spreadsheets in the Cloud - Not Ready Yet," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 52(i07).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A. Yalta & A. Yalta, 2010. "Should Economists Use Open Source Software for Doing Research?," Computational Economics, Springer;Society for Computational Economics, vol. 35(4), pages 371-394, April.
    2. Charles G. Renfro, 2009. "The Practice of Econometric Theory," Advanced Studies in Theoretical and Applied Econometrics, Springer, number 978-3-540-75571-5, July-Dece.
    3. Oluwarotimi O. Odeh & Allen M. Featherstone & Jason S. Bergtold, 2010. "Reliability of Statistical Software," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 92(5), pages 1472-1489.
    4. Hargreaves, Bruce R. & McWilliams, Thomas P., 2010. "Polynomial Trendline function flaws in Microsoft Excel," Computational Statistics & Data Analysis, Elsevier, vol. 54(4), pages 1190-1196, April.
    5. H.-J. Sun & Kaoru Fukuda & B. D. McCullough, 2017. "Inaccurate regression coefficients in Microsoft Excel 2003: an investigation of Volpi’s “zero bug”," Computational Statistics, Springer, vol. 32(4), pages 1411-1421, December.
    6. Yalta, A. Talha, 2008. "The accuracy of statistical distributions in Microsoft® Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4579-4586, June.
    7. Yalta, A. Talha, 2007. "The Numerical Reliability of GAUSS 8.0," The American Statistician, American Statistical Association, vol. 61, pages 262-268, August.
    8. Jason S. Bergtold & Krishna P. Pokharel & Allen M. Featherstone & Lijia Mo, 2018. "On the examination of the reliability of statistical software for estimating regression models with discrete dependent variables," Computational Statistics, Springer, vol. 33(2), pages 757-786, June.
    9. repec:jss:jstsof:34:i04 is not listed on IDEAS
    10. A. Talha Yalta, 2010. "The Accuracy of Statistical Distributions in Microsoft (R) Excel 2007," Working Papers 1006, TOBB University of Economics and Technology, Department of Economics.
    11. Guy Mélard, 2014. "On the accuracy of statistical procedures in Microsoft Excel 2010," Computational Statistics, Springer, vol. 29(5), pages 1095-1128, October.
    12. McCullough, B.D. & Heiser, David A., 2008. "On the accuracy of statistical procedures in Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4570-4578, June.
    13. McCullough, Bruce D. & Yalta, A. Talha, 2013. "Spreadsheets in the Cloud - Not Ready Yet," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 52(i07).
    14. McCullough, B. D., 2000. "Is it safe to assume that software is accurate?," International Journal of Forecasting, Elsevier, vol. 16(3), pages 349-357.
    15. Kusters, Ulrich & McCullough, B.D. & Bell, Michael, 2006. "Forecasting software: Past, present and future," International Journal of Forecasting, Elsevier, vol. 22(3), pages 599-615.
    16. A. M. Kitchen & R. Drachenberg & J. Symanzik, 2003. "Assessing the reliability of web-based statistical software," Computational Statistics, Springer, vol. 18(1), pages 107-122, March.
    17. McCullough, B.D., 2008. "Special section on Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4568-4569, June.
    18. Gilles Teyssière, 2005. "Structural time series modelling with STAMP 6.02," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(4), pages 571-577, May.
    19. Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), 2009. "Econometrics with gretl. Proceedings of the gretl Conference 2009," UPV/EHU Books, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales, edition 1, number 01.
    20. A. Talha Yalta & A. Yasemin Yalta, 2009. "Wilkinson Tests and gretl," EHUCHAPS, in: Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), Econometrics with gretl. Proceedings of the gretl Conference 2009, edition 1, chapter 16, pages 243-251, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales.
    21. Bergtold, Jason S. & Pokharel, Krishna & Featherstone, Allen, 2015. "On the Examination of the Reliability of Statistical Software for Estimating Logistic Regression Models," 2015 AAEA & WAEA Joint Annual Meeting, July 26-28, San Francisco, California 205643, Agricultural and Applied Economics Association.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:25:y:2009:i:1:p:62-73. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.