IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/42235.html
   My bibliography  Save this paper

Does BIC Estimate and Forecast Better than AIC?

Author

Listed:
  • Medel, Carlos A.
  • Salgado, Sergio C.

Abstract

We test two questions: (i) Is the Bayesian Information Criterion (BIC) more parsimonious than Akaike Information Criterion (AIC)?, and (ii) Is BIC better than AIC for forecasting purposes? By using simulated data, we provide statistical inference of both hypotheses individually and then jointly with a multiple hypotheses testing procedure to control better for type-I error. Both testing procedures deliver the same result: The BIC shows an in- and out-of-sample superiority over AIC only in a long-sample context.

Suggested Citation

  • Medel, Carlos A. & Salgado, Sergio C., 2012. "Does BIC Estimate and Forecast Better than AIC?," MPRA Paper 42235, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:42235
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/42235/1/MPRA_paper_42235.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lawrence Raffalovich & Glenn Deane & David Armstrong & Hui-Shien Tsao, 2008. "Model selection procedures in social research: Monte-Carlo simulation results," Journal of Applied Statistics, Taylor & Francis Journals, vol. 35(10), pages 1093-1114.
    2. Todd E. Clark, 2004. "Can out-of-sample forecast comparisons help prevent overfitting?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(2), pages 115-139.
    3. Clark, Todd E. & McCracken, Michael W., 2001. "Tests of equal forecast accuracy and encompassing for nested models," Journal of Econometrics, Elsevier, vol. 105(1), pages 85-110, November.
    4. Helmut Lütkepohl, 1985. "Comparison Of Criteria For Estimating The Order Of A Vector Autoregressive Process," Journal of Time Series Analysis, Wiley Blackwell, vol. 6(1), pages 35-52, January.
    5. Clive Granger & Yongil Jeon, 2004. "Forecasting Performance of Information Criteria with Many Macro Series," Journal of Applied Statistics, Taylor & Francis Journals, vol. 31(10), pages 1227-1240.
    6. West, Kenneth D, 1996. "Asymptotic Inference about Predictive Ability," Econometrica, Econometric Society, vol. 64(5), pages 1067-1084, September.
    7. Geweke, John & Meese, Richard, 1981. "Estimating regression models of finite but unknown order," Journal of Econometrics, Elsevier, vol. 16(1), pages 162-162, May.
    8. Hansen, Peter Reinhard, 2005. "A Test for Superior Predictive Ability," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 365-380, October.
    9. Ryan Sullivan & Allan Timmermann & Halbert White, 1999. "Data‐Snooping, Technical Trading Rule Performance, and the Bootstrap," Journal of Finance, American Finance Association, vol. 54(5), pages 1647-1691, October.
    10. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    11. Sims, Christopher A, 1980. "Macroeconomics and Reality," Econometrica, Econometric Society, vol. 48(1), pages 1-48, January.
    12. Hamparsum Bozdogan, 1987. "Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions," Psychometrika, Springer;The Psychometric Society, vol. 52(3), pages 345-370, September.
    13. Wolak, Frank A., 1989. "Testing inequality constraints in linear econometric models," Journal of Econometrics, Elsevier, vol. 41(2), pages 205-235, June.
    14. Anne B. Koehler & Emily S. Murphree, 1988. "A Comparison of the Akaike and Schwarz Criteria for Selecting Model Order," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 37(2), pages 187-195, June.
    15. Yi, Gang & Judge, George, 1988. "Statistical model selection criteria," Economics Letters, Elsevier, vol. 28(1), pages 47-51.
    16. Nickelsburg, Gerald, 1985. "Small-sample properties of dimensionality statistics for fitting VAR models to aggregate economic data : A Monte Carlo study," Journal of Econometrics, Elsevier, vol. 28(2), pages 183-192, May.
    17. Newey, Whitney & West, Kenneth, 2014. "A simple, positive semi-definite, heteroscedasticity and autocorrelation consistent covariance matrix," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 33(1), pages 125-132.
    18. Pablo Pincheira, 2012. "A Joint Test of Superior Predictive Ability for Chilean Inflation Forecasts," Journal Economía Chilena (The Chilean Economy), Central Bank of Chile, vol. 15(3), pages 04-39, December.
    19. Clifford M. Hurvich & Chih‐Ling Tsai, 1993. "A Corrected Akaike Information Criterion For Vector Autoregressive Model Selection," Journal of Time Series Analysis, Wiley Blackwell, vol. 14(3), pages 271-279, May.
    20. Halbert White, 2000. "A Reality Check for Data Snooping," Econometrica, Econometric Society, vol. 68(5), pages 1097-1126, September.
    21. Kilian, Lutz, 2001. "Impulse Response Analysis in Vector Autoregressions with Unknown Lag Order," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 20(3), pages 161-179, April.
    22. Sawa, Takamitsu, 1978. "Information Criteria for Discriminating among Alternative Regression Models," Econometrica, Econometric Society, vol. 46(6), pages 1273-1291, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carlos Medel, 2017. "Forecasting Chilean inflation with the hybrid new keynesian Phillips curve: globalisation, combination, and accuracy," Journal Economía Chilena (The Chilean Economy), Central Bank of Chile, vol. 20(3), pages 004-050, December.
    2. Cruz, Manuel David, 2023. "Labor Productivity, Real Wages, and Employment in OECD Economies," Structural Change and Economic Dynamics, Elsevier, vol. 66(C), pages 367-382.
    3. Carlos A. Medel, 2015. "Probabilidad Clásica de Sobreajuste con Criterios de Información: Estimaciones con Series Macroeconómicas Chilenas," Revista de Analisis Economico – Economic Analysis Review, Universidad Alberto Hurtado/School of Economics and Business, vol. 30(1), pages 57-72, Abril.
    4. Carlos A. Medel, 2018. "Forecasting Inflation with the Hybrid New Keynesian Phillips Curve: A Compact-Scale Global VAR Approach," International Economic Journal, Taylor & Francis Journals, vol. 32(3), pages 331-371, July.
    5. Manuel David Cruz, 2022. "Labor productivity, real wages, and employment: evidence from a panel of OECD economies over 1960-2019," Working Papers PKWP2203, Post Keynesian Economics Society (PKES).
    6. Shahrin Islam & Armana Sabiha Huq & Sabah Hossain Iqra & Raas Sarker Tomal, 2023. "Impacts of COVID-19 Pandemic Lockdown on Road Safety in Bangladesh," Sustainability, MDPI, vol. 15(3), pages 1-22, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carlos Medel, 2012. "¿Akaike o Schwarz? ¿Cuál elegir para Predecir el PIB Chileno?," Working Papers Central Bank of Chile 658, Central Bank of Chile.
    2. Daniel Fernández, 2011. "Suficiencia del capital y previsiones de la banca uruguaya por su exposición al sector industrial," Monetaria, CEMLA, vol. 0(4), pages 517-589, octubre-d.
    3. Javier Pereda, 2011. "Estimación de la tasa natural de interés para Perú: un enfoque financiero," Monetaria, CEMLA, vol. 0(4), pages 429-459, octubre-d.
    4. Tamara Burdisso & Eduardo Ariel Corso, 2011. "Incertidumbre y dolarización de cartera: el caso argentino en el último medio siglo," Monetaria, CEMLA, vol. 0(4), pages 461-515, octubre-d.
    5. Carlos A. Medel Vera, 2011. "¿Akaike o Schwarz? ¿Cuál utilizar para predecir el PIB chileno?," Monetaria, CEMLA, vol. 0(4), pages 591-615, octubre-d.
    6. Clark, Todd & McCracken, Michael, 2013. "Advances in Forecast Evaluation," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1107-1201, Elsevier.
    7. Carlos A. Medel, 2013. "How informative are in-sample information criteria to forecasting? The case of Chilean GDP," Latin American Journal of Economics-formerly Cuadernos de Economía, Instituto de Economía. Pontificia Universidad Católica de Chile., vol. 50(1), pages 133-161, May.
    8. McCracken, Michael W., 2007. "Asymptotics for out of sample tests of Granger causality," Journal of Econometrics, Elsevier, vol. 140(2), pages 719-752, October.
    9. Carlos A. Medel, 2015. "Probabilidad Clásica de Sobreajuste con Criterios de Información: Estimaciones con Series Macroeconómicas Chilenas," Revista de Analisis Economico – Economic Analysis Review, Universidad Alberto Hurtado/School of Economics and Business, vol. 30(1), pages 57-72, Abril.
    10. Christopher J. Neely & David E. Rapach & Jun Tu & Guofu Zhou, 2014. "Forecasting the Equity Risk Premium: The Role of Technical Indicators," Management Science, INFORMS, vol. 60(7), pages 1772-1791, July.
    11. Jin, Sainan & Corradi, Valentina & Swanson, Norman R., 2017. "Robust Forecast Comparison," Econometric Theory, Cambridge University Press, vol. 33(6), pages 1306-1351, December.
    12. Raffaella Giacomini & Barbara Rossi, 2013. "Forecasting in macroeconomics," Chapters, in: Nigar Hashimzade & Michael A. Thornton (ed.), Handbook of Research Methods and Applications in Empirical Macroeconomics, chapter 17, pages 381-408, Edward Elgar Publishing.
    13. Busetti, Fabio & Marcucci, Juri, 2013. "Comparing forecast accuracy: A Monte Carlo investigation," International Journal of Forecasting, Elsevier, vol. 29(1), pages 13-27.
    14. Granziera, Eleonora & Hubrich, Kirstin & Moon, Hyungsik Roger, 2014. "A predictability test for a small number of nested models," Journal of Econometrics, Elsevier, vol. 182(1), pages 174-185.
    15. Sébastien Laurent & Jeroen V. K. Rombouts & Francesco Violante, 2012. "On the forecasting accuracy of multivariate GARCH models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(6), pages 934-955, September.
    16. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    17. Valentina Corradi & Norman Swanson, 2013. "A Survey of Recent Advances in Forecast Accuracy Comparison Testing, with an Extension to Stochastic Dominance," Departmental Working Papers 201309, Rutgers University, Department of Economics.
    18. Drechsel, Katja & Scheufele, Rolf, 2012. "The performance of short-term forecasts of the German economy before and during the 2008/2009 recession," International Journal of Forecasting, Elsevier, vol. 28(2), pages 428-445.
    19. Sander Barendse & Andrew J. Patton, 2022. "Comparing Predictive Accuracy in the Presence of a Loss Function Shape Parameter," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(3), pages 1057-1069, June.
    20. Todd E. Clark & Michael W. McCracken, 2010. "Reality checks and nested forecast model comparisons," Working Papers 2010-032, Federal Reserve Bank of St. Louis.

    More about this item

    Keywords

    AIC; BIC; time-series models; overfitting; forecast comparison; joint hypothesis testing;
    All these keywords.

    JEL classification:

    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:42235. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.