IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v25y2009i1p62-73.html
   My bibliography  Save this article

On the importance of verifying forecasting results

Author

Listed:
  • Yalta, A. Talha
  • Jenal, Olaf

Abstract

We discuss the various sources of error in numerical computations with the use of examples from the literature relevant to time series analysis. We also submit a case where, by manual verification, we were able to discover a plausible forecast to be erroneous due to a number of software flaws in the XLSTAT addin for Microsoft Excel. Furthermore, after discussing the alternative techniques for implementing the ARIMA (AutoRegressive Integrated Moving Average) methodology on a computer, we show that different approaches can cause considerable discrepancies in the results across different programs, and even within a single software system.

Suggested Citation

  • Yalta, A. Talha & Jenal, Olaf, 2009. "On the importance of verifying forecasting results," International Journal of Forecasting, Elsevier, vol. 25(1), pages 62-73.
  • Handle: RePEc:eee:intfor:v:25:y:2009:i:1:p:62-73
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169-2070(08)00068-X
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Nada R. Sanders & Karl B. Manrodt, 2003. "Forecasting Software in Practice: Use, Satisfaction, and Performance," Interfaces, INFORMS, vol. 33(5), pages 90-93, October.
    2. McCullough, B.D. & Heiser, David A., 2008. "On the accuracy of statistical procedures in Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4570-4578, June.
    3. D. McCullough, B. & Wilson, Berry, 2002. "On the accuracy of statistical procedures in Microsoft Excel 2000 and Excel XP," Computational Statistics & Data Analysis, Elsevier, vol. 40(4), pages 713-721, October.
    4. McCullough, B. D., 2000. "Is it safe to assume that software is accurate?," International Journal of Forecasting, Elsevier, vol. 16(3), pages 349-357.
    5. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    6. McCullough, B. D. & Wilson, Berry, 1999. "On the accuracy of statistical procedures in Microsoft Excel 97," Computational Statistics & Data Analysis, Elsevier, vol. 31(1), pages 27-37, July.
    7. McCullough, B.D. & Wilson, Berry, 2005. "On the accuracy of statistical procedures in Microsoft Excel 2003," Computational Statistics & Data Analysis, Elsevier, vol. 49(4), pages 1244-1252, June.
    8. Sawitzki, Gunther, 1994. "Testing numerical reliability of data analysis systems," Computational Statistics & Data Analysis, Elsevier, vol. 18(2), pages 269-286, September.
    9. H. D. Vinod & B. D. McCullough, 1999. "The Numerical Reliability of Econometric Software," Journal of Economic Literature, American Economic Association, vol. 37(2), pages 633-665, June.
    10. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.
    11. Kusters, Ulrich & McCullough, B.D. & Bell, Michael, 2006. "Forecasting software: Past, present and future," International Journal of Forecasting, Elsevier, vol. 22(3), pages 599-615.
    12. H. D. Vinod & B. D. McCullough, 1999. "Corrigenda: The Numerical Reliability of Econometric Software," Journal of Economic Literature, American Economic Association, vol. 37(4), pages 1565-1565, December.
    13. Newbold, Paul & Agiakloglou, Christos & Miller, John, 1994. "Adventures with ARIMA software," International Journal of Forecasting, Elsevier, vol. 10(4), pages 573-581, December.
    14. Ernst R. Berndt & Bronwyn H. Hall & Robert E. Hall & Jerry A. Hausman, 1974. "Estimation and Inference in Nonlinear Structural Models," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 3, number 4, pages 653-665, National Bureau of Economic Research, Inc.
    15. McCullough, B D, 1999. "Econometric Software Reliability: EViews, LIMDEP, SHAZAM and TSP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(2), pages 191-202, March-Apr.
    16. Jushan Bai & Pierre Perron, 2003. "Computation and analysis of multiple structural change models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 1-22.
    17. Yalta, A. Talha, 2008. "The accuracy of statistical distributions in Microsoft® Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4579-4586, June.
    18. C. R. McKenzie & Sumiko Takaoka, 2007. "EViews 5.1," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(6), pages 1145-1152.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. A. Talha Yalta & A. Yasemin Yalta, 2009. "Wilkinson Tests and gretl," EHUCHAPS, in: Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), Econometrics with gretl. Proceedings of the gretl Conference 2009, edition 1, chapter 16, pages 243-251, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales.
    2. Guy Mélard, 2014. "On the accuracy of statistical procedures in Microsoft Excel 2010," Computational Statistics, Springer, vol. 29(5), pages 1095-1128, October.
    3. Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), 2009. "Econometrics with gretl. Proceedings of the gretl Conference 2009," UPV/EHU Books, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales, edition 1, number 01, December.
    4. Hargreaves, Bruce R. & McWilliams, Thomas P., 2010. "Polynomial Trendline function flaws in Microsoft Excel," Computational Statistics & Data Analysis, Elsevier, vol. 54(4), pages 1190-1196, April.
    5. A. Yalta & A. Yalta, 2010. "Should Economists Use Open Source Software for Doing Research?," Computational Economics, Springer;Society for Computational Economics, vol. 35(4), pages 371-394, April.
    6. Yalta, A. Talha & Schreiber, Sven, 2012. "Random Number Generation in gretl," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 50(c01).
    7. McCullough, Bruce D. & Yalta, A. Talha, 2013. "Spreadsheets in the Cloud - Not Ready Yet," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 52(i07).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A. Yalta & A. Yalta, 2010. "Should Economists Use Open Source Software for Doing Research?," Computational Economics, Springer;Society for Computational Economics, vol. 35(4), pages 371-394, April.
    2. Charles G. Renfro, 2009. "The Practice of Econometric Theory," Advanced Studies in Theoretical and Applied Econometrics, Springer, number 978-3-540-75571-5, enero-jun.
    3. Jason S. Bergtold & Krishna P. Pokharel & Allen M. Featherstone & Lijia Mo, 2018. "On the examination of the reliability of statistical software for estimating regression models with discrete dependent variables," Computational Statistics, Springer, vol. 33(2), pages 757-786, June.
    4. Hargreaves, Bruce R. & McWilliams, Thomas P., 2010. "Polynomial Trendline function flaws in Microsoft Excel," Computational Statistics & Data Analysis, Elsevier, vol. 54(4), pages 1190-1196, April.
    5. McCullough, B. D., 2000. "Is it safe to assume that software is accurate?," International Journal of Forecasting, Elsevier, vol. 16(3), pages 349-357.
    6. Yalta, A. Talha, 2008. "The accuracy of statistical distributions in Microsoft® Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4579-4586, June.
    7. McCullough, B.D. & Heiser, David A., 2008. "On the accuracy of statistical procedures in Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4570-4578, June.
    8. A. M. Kitchen & R. Drachenberg & J. Symanzik, 2003. "Assessing the reliability of web-based statistical software," Computational Statistics, Springer, vol. 18(1), pages 107-122, March.
    9. Bergtold, Jason S. & Pokharel, Krishna & Featherstone, Allen, 2015. "On the Examination of the Reliability of Statistical Software for Estimating Logistic Regression Models," 2015 AAEA & WAEA Joint Annual Meeting, July 26-28, San Francisco, California 205643, Agricultural and Applied Economics Association.
    10. McCullough, Bruce D. & Yalta, A. Talha, 2013. "Spreadsheets in the Cloud - Not Ready Yet," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 52(i07).
    11. McCullough, B.D., 2008. "Special section on Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4568-4569, June.
    12. H.-J. Sun & Kaoru Fukuda & B. D. McCullough, 2017. "Inaccurate regression coefficients in Microsoft Excel 2003: an investigation of Volpi’s “zero bug”," Computational Statistics, Springer, vol. 32(4), pages 1411-1421, December.
    13. Oluwarotimi O. Odeh & Allen M. Featherstone & Jason S. Bergtold, 2010. "Reliability of Statistical Software," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 92(5), pages 1472-1489.
    14. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    15. David E. Allen & Michael McAleer, 2018. "Theoretical and Empirical Differences between Diagonal and Full BEKK for Risk Management," Energies, MDPI, Open Access Journal, vol. 11(7), pages 1-19, June.
    16. Kusters, Ulrich & McCullough, B.D. & Bell, Michael, 2006. "Forecasting software: Past, present and future," International Journal of Forecasting, Elsevier, vol. 22(3), pages 599-615.
    17. Gatfaoui, Hayette, 2013. "Translating financial integration into correlation risk: A weekly reporting's viewpoint for the volatility behavior of stock markets," Economic Modelling, Elsevier, vol. 30(C), pages 776-791.
    18. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.
    19. Ooms, M., 2008. "Trends in Applied Econometrics Software Development 1985-2008, an analysis of Journal of Applied Econometrics research articles, software reviews, data and code," Serie Research Memoranda 0021, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics.
    20. Nash, John C., 2008. "Teaching statistics with Excel 2007 and other spreadsheets," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4602-4606, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:25:y:2009:i:1:p:62-73. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Nithya Sathishkumar). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.