IDEAS home Printed from https://ideas.repec.org/a/lrk/eeaart/35_2_5.html
   My bibliography  Save this article

Forecasting Performance and Information Measures. Revisiting the M-Competition /Evaluación de Predicciones y Medidas de Información. Reexamen de la M-Competición

Author

Listed:
  • LÓPEZ MENÉNDEZ, ANA JESÚS

    () (Universidad de Oviedo, Facultad Economía y Empresa, Avda. del Cristo, s/n, 33006 Oviedo, España.)

  • PÉREZ SUÁREZ, RIGOBERTO

    () (Universidad de Oviedo, Facultad Economía y Empresa, Avda. del Cristo, s/n, 33006 Oviedo, España.)

Abstract

Economic and financial time series are widely considered as one of the most challenging applications of modeling and forecasting. The increasing in forecasting availability and the controversial debate about the advantages of alternative forecasting procedures suggest the need of further research on the forecasting evaluation metrics. In this context, this paper focuses on two information-based accuracy measures: Theil´s U Index and the Quadratic Information Accuracy Measure (QIAM), and aims to re-examine the empirical results of the M3-Competition by Makridakis and Hibon (2000), and specifically those referred to the subset of macroeconomic and financial series. The computation of the proposed accuracy indicators leads to new rankings of forecasting techniques, showing some similarities and disagreements with the main conclusions by Makridakis & Hibon (2000), found on five error based accuracy measures. The obtained results also allow a complexity?accuracy analysis. La obtención de predicciones para series temporales económicas y financieras es una tarea de gran dificultad. En un contexto de disponibilidad creciente de predicciones y debate sobre las alternativas metodológicas para su obtención, resulta recomendable dedicar nuevos esfuerzos a las medidas utilizadas para su evaluación. Este trabajo analiza dos indicadores de precisión basados en medidas de información: el índice U de Theil y la Medida de Información Cuadrática de Precisión (QIAM), cuya aplicación a la M-Competición de Makridakis and Hibon (2000) permite reexaminar los resultados empíricos obtenidos por estos autores para el conjunto de la base de datos y más concretamente para las series macroeconómicas y financieras. El cálculo de las medidas propuestas proporciona un nuevo ranking de técnicas predictivas, que muestra coincidencias y diferencias con las conclusiones obtenidas por Makridakis & Hibon a partir de cinco medidas de precisión basadas en errores. Los resultados obtenidos permiten también un análisis de complejidad versus precisión.

Suggested Citation

  • López Menéndez, Ana Jesús & Pérez Suárez, Rigoberto, 2017. "Forecasting Performance and Information Measures. Revisiting the M-Competition /Evaluación de Predicciones y Medidas de Información. Reexamen de la M-Competición," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 35, pages 299-314, Mayo.
    • Handle: RePEc:lrk:eeaart:35_2_5
    as

    Download full text from publisher

    File URL: http://www.revista-eea.net
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Fildes, Robert & Hibon, Michele & Makridakis, Spyros & Meade, Nigel, 1998. "Generalising about univariate forecasting methods: further empirical evidence," International Journal of Forecasting, Elsevier, vol. 14(3), pages 339-358, September.
    2. Mark J. Schervish & Teddy Seidenfeld & Joseph B. Kadane, 2009. "Proper Scoring Rules, Dominated Forecasts, and Coherence," Decision Analysis, INFORMS, vol. 6(4), pages 202-221, December.
    3. Clive Granger & Yongil Jeon, 2004. "Forecasting Performance of Information Criteria with Many Macro Series," Journal of Applied Statistics, Taylor & Francis Journals, vol. 31(10), pages 1227-1240.
    4. Fildes, Robert & Petropoulos, Fotios, 2015. "Simple versus complex selection rules for forecasting many time series," Journal of Business Research, Elsevier, vol. 68(8), pages 1692-1701.
    5. Thomas Wenzel, 2001. "Hits-and-misses for the evaluation and combination of forecasts," Journal of Applied Statistics, Taylor & Francis Journals, vol. 28(6), pages 759-773.
    6. Matteo Ciccarelli & Kirstin Hubrich, 2010. "Forecast uncertainty: sources, measurement and evaluation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(4), pages 509-513.
    7. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    8. Soyer, Emre & Hogarth, Robin M., 2015. "The golden rule of forecasting: Objections, refinements, and enhancements," Journal of Business Research, Elsevier, vol. 68(8), pages 1702-1704.
    9. Syntetos, Aris A. & Nikolopoulos, Konstantinos & Boylan, John E., 2010. "Judging the judges through accuracy-implication metrics: The case of inventory forecasting," International Journal of Forecasting, Elsevier, vol. 26(1), pages 134-143, January.
    10. Graefe, Andreas, 2015. "Improving forecasts using equally weighted predictors," Journal of Business Research, Elsevier, vol. 68(8), pages 1792-1799.
    11. Makridakis, Spyros & Hibon, Michele, 2000. "The M3-Competition: results, conclusions and implications," International Journal of Forecasting, Elsevier, vol. 16(4), pages 451-476.
    12. Granger, Clive W J, 1996. "Can We Improve the Perceived Quality of Economic Forecasts?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(5), pages 455-473, Sept.-Oct.
    13. Hibon, Michele & Evgeniou, Theodoros, 2005. "To combine or not to combine: selecting among forecasts and their combinations," International Journal of Forecasting, Elsevier, vol. 21(1), pages 15-24.
    14. Fildes, Robert & Petropoulos, Fotios, 2015. "Is there a Golden Rule?," Journal of Business Research, Elsevier, vol. 68(8), pages 1742-1745.
    15. Jane Binner & Rakesh Bissoondeeal & Thomas Elger & Alicia Gazely & Andrew Mullineux, 2005. "A comparison of linear forecasting models and neural networks: an application to Euro inflation and Euro Divisia," Applied Economics, Taylor & Francis Journals, vol. 37(6), pages 665-680.
    16. Mark Greer, 2005. "Combination forecasting for directional accuracy: An application to survey interest rate forecasts," Journal of Applied Statistics, Taylor & Francis Journals, vol. 32(6), pages 607-615.
    17. Petropoulos, Fotios & Makridakis, Spyros & Assimakopoulos, Vassilios & Nikolopoulos, Konstantinos, 2014. "‘Horses for Courses’ in demand forecasting," European Journal of Operational Research, Elsevier, vol. 237(1), pages 152-163.
    18. Haritini Tsangari, 2007. "An Alternative Methodology for Combining Different Forecasting Models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 34(4), pages 403-421.
    19. Maria Gil & Rigoberto Perez & Pedro Gil, 1989. "A family of measures of uncertainty involving utilities: Definition, properties, applications and statistical inferences," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 36(1), pages 129-147, December.
    20. Rob J. Hyndman, 2006. "Another Look at Forecast Accuracy Metrics for Intermittent Demand," Foresight: The International Journal of Applied Forecasting, International Institute of Forecasters, issue 4, pages 43-46, June.
    21. Clive W.J. Granger & Yongil Jeon, 2003. "Interactions between large macro models and time series analysis," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 8(1), pages 1-10.
    22. Makridakis, Spyros & Taleb, Nassim, 2009. "Living in a world of low levels of predictability," International Journal of Forecasting, Elsevier, vol. 25(4), pages 840-844, October.
    23. Blanca Moreno & Ana Jesús López, 2013. "Combining Economic Forecasts by Using a Maximum Entropy Econometric Approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 32(2), pages 124-136, March.
    24. Green, Kesten C. & Armstrong, J. Scott, 2015. "Simple versus complex forecasting: The evidence," Journal of Business Research, Elsevier, vol. 68(8), pages 1678-1685.
    25. Ramos Lobo, R. & Clar López, M. & Suriñach Caralt, J., 2000. "Comparación de la capacidad predictiva de los modelos de coeficientes fijos frente a variables en los modelos econométricos regionales: un análisis para Cataluña," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 15, pages 125-162, Agosto.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Predicción; Precisión; M-Competición; Indice de Theil; Medida de Información Cuadrática de Precisión (QIAM) ; Forecasting; Accuracy; M3-Competition; Theil´s U Index; Quadratic Information Accuracy Measure (QIAM). .;

    JEL classification:

    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:lrk:eeaart:35_2_5. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Miguel Angel Sanchez Granero). General contact details of provider: http://edirc.repec.org/data/fcvldes.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.