IDEAS home Printed from https://ideas.repec.org/a/ila/anaeco/v30y2015i1p57-72.html
   My bibliography  Save this article

Probabilidad Clásica de Sobreajuste con Criterios de Información: Estimaciones con Series Macroeconómicas Chilenas

Author

Listed:
  • Carlos A. Medel

    (Universidad de Nottingham)

Abstract

This paper provides, via Monte Carlo simulations, estimates of the classical probability of overfitting under an autoregressive environment (AR), using the information criteria (IC) of Akaike, Schwarz and Hannan-Quinn (AIC, BIC and HQ), calibrated with Chilean data of total inflation, core inflation, IMACEC, and monthly return of the nominal exchange rate Chilean peso- American dollar. This probability corresponds to the number of times when a candidate model has a strictly greater number of coefficients than the true model, divided by the total number of searches. The results indicate that the increased risk of overfitting is obtained with the AIC, followed by HQ and finally the BIC. The highest probability of overfitting is achieved with the AIC, reaching 32 and 30% with the exchange rate and IMACEC, respectively, followed by 25 and 22% for total and core inflation. Considering the three IC, it is more likely to obtain an overfitted model by just one coefficient. Also, it is more likely that the overfitting does not exceed 10 coefficients. These results are important as quantifying the extent to which these variables are subject to overfitting risk when represented by AR models. The potential problems carried by overfitting includes: (i) the spurious regression, (ii) distor tthe estimation of impulse response function, and (iii) affect the predictive accuracy of the variable of interest. The latter problem is analyzed in detail.

Suggested Citation

  • Carlos A. Medel, 2015. "Probabilidad Clásica de Sobreajuste con Criterios de Información: Estimaciones con Series Macroeconómicas Chilenas," Revista de Analisis Economico – Economic Analysis Review, Universidad Alberto Hurtado/School of Economics and Business, vol. 30(1), pages 57-72, Abril.
  • Handle: RePEc:ila:anaeco:v:30:y:2015:i:1:p:57-72
    as

    Download full text from publisher

    File URL: https://www.rae-ear.org/index.php/rae/article/view/437/568
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Todd E. Clark, 2004. "Can out-of-sample forecast comparisons help prevent overfitting?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(2), pages 115-139.
    2. Carlos A. Medel & Sergio C. Salgado, 2013. "Does the Bic Estimate and Forecast Better than the Aic?," Revista de Analisis Economico – Economic Analysis Review, Universidad Alberto Hurtado/School of Economics and Business, vol. 28(1), pages 47-64, April.
    3. Clive Granger & Yongil Jeon, 2004. "Forecasting Performance of Information Criteria with Many Macro Series," Journal of Applied Statistics, Taylor & Francis Journals, vol. 31(10), pages 1227-1240.
    4. Clive W. J. Granger & Yongil Jeon, 2006. "Dynamics of Model Overfitting Measured in terms of Autoregressive Roots," Journal of Time Series Analysis, Wiley Blackwell, vol. 27(3), pages 347-365, May.
    5. McQuarrie, Allan D., 1999. "A small-sample correction for the Schwarz SIC model selection criterion," Statistics & Probability Letters, Elsevier, vol. 44(1), pages 79-86, August.
    6. Carlos A. Medel, 2013. "How informative are in-sample information criteria to forecasting? The case of Chilean GDP," Latin American Journal of Economics-formerly Cuadernos de Economía, Instituto de Economía. Pontificia Universidad Católica de Chile., vol. 50(1), pages 133-161, May.
    7. Graham Elliott & Allan Timmermann, 2016. "Economic Forecasting," Economics Books, Princeton University Press, edition 1, number 10740.
    8. Kevin D. Hoover & Stephen J. Perez, 1999. "Data mining reconsidered: encompassing and the general-to-specific approach to specification search," Econometrics Journal, Royal Economic Society, vol. 2(2), pages 167-191.
    9. Cavanaugh, Joseph E., 1997. "Unifying the derivations for the Akaike and corrected Akaike information criteria," Statistics & Probability Letters, Elsevier, vol. 33(2), pages 201-208, April.
    10. Granger, C. W. J. & Newbold, P., 1974. "Spurious regressions in econometrics," Journal of Econometrics, Elsevier, vol. 2(2), pages 111-120, July.
    11. Kilian, Lutz, 2001. "Impulse Response Analysis in Vector Autoregressions with Unknown Lag Order," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 20(3), pages 161-179, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carlos Medel, 2017. "Forecasting Chilean inflation with the hybrid new keynesian Phillips curve: globalisation, combination, and accuracy," Journal Economía Chilena (The Chilean Economy), Central Bank of Chile, vol. 20(3), pages 004-050, December.
    2. Carlos A. Medel, 2018. "Forecasting Inflation with the Hybrid New Keynesian Phillips Curve: A Compact-Scale Global VAR Approach," International Economic Journal, Taylor & Francis Journals, vol. 32(3), pages 331-371, July.

    More about this item

    Keywords

    Nonparametric Modelling; Information Criteria; Overfitting; Out-of-sample Analysis;

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation
    • E23 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Production
    • E27 - Macroeconomics and Monetary Economics - - Consumption, Saving, Production, Employment, and Investment - - - Forecasting and Simulation: Models and Applications
    • E31 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Price Level; Inflation; Deflation
    • E37 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles - - - Forecasting and Simulation: Models and Applications
    • F31 - International Economics - - International Finance - - - Foreign Exchange

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ila:anaeco:v:30:y:2015:i:1:p:57-72. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mauricio Tejada). General contact details of provider: http://edirc.repec.org/data/deilacl.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.