IDEAS home Printed from https://ideas.repec.org/p/ven/wpaper/201818.html
   My bibliography  Save this paper

A scoring rule for factor and autoregressive models under misspecification

Author

Listed:
  • Roberto Casarin

    () (Department of Economics, University of Venice Cà Foscari)

  • Fausto Corradin

    () (Department of Economics, University of Venice Cà Foscari)

  • Francesco Ravazzolo

    () (Free University of Bozen-Bolzano)

  • Domenico Sartore

    () (Department of Economics, University of Venice Cà Foscari)

Abstract

Factor models (FM) are now widely used for forecasting with large set of time series. Another class of models, which can be easily estimated and used in a large dimensional setting, is multivariate autoregressive models (MAR), where independent autoregressive processes are assumed for the series in the panel. We compare the forecasting abilities of FM and MAR models when assuming both models are misspecified and the data generating process is a vector autoregressive model. We establish which conditions need to be satisfied for a FM to overperform MAR in terms of mean square forecasting error. The condition indicates in presence of misspecification that FM is not always overperforming MAR and that the FM predictive performance depends crucially on the parameter values of the data generating process. Building on the theoretical relationship between FM and MAR predictive performances, we provide a scoring rule which can be evaluated on the data to either select the model, or combine the models in forecasting exercises. Some numerical illustrations are provided both on simulated data and on wel-known large economic datasets. The empirical results show that the frequency of the true positive signals is larger when FM and MAR forecasting performances differ substantially and it decreases as the horizon increases.

Suggested Citation

  • Roberto Casarin & Fausto Corradin & Francesco Ravazzolo & Domenico Sartore, 2018. "A scoring rule for factor and autoregressive models under misspecification," Working Papers 2018:18, Department of Economics, University of Venice "Ca' Foscari".
  • Handle: RePEc:ven:wpaper:2018:18
    as

    Download full text from publisher

    File URL: http://www.unive.it/pag/fileadmin/user_upload/dipartimenti/economia/doc/Pubblicazioni_scientifiche/working_papers/2018/WP_DSE_casarin_corradin_sartore_ravazzolo_18_18.pdf
    File Function: First version, anno
    Download Restriction: no

    References listed on IDEAS

    as
    1. Roberto Casarin & Stefano Grassi & Francesco Ravazzolo & Herman K. van Dijk, 2015. "Dynamic predictive density combinations for large data sets in economics and finance," Working Paper 2015/12, Norges Bank.
    2. Gary M. Koop, 2013. "Forecasting with Medium and Large Bayesian VARS," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 177-203, March.
    3. Billio, Monica & Casarin, Roberto & Ravazzolo, Francesco & van Dijk, Herman K., 2013. "Time-varying combinations of predictive densities using nonlinear filtering," Journal of Econometrics, Elsevier, vol. 177(2), pages 213-232.
    4. Kapetanios, G. & Mitchell, J. & Price, S. & Fawcett, N., 2015. "Generalised density forecast combinations," Journal of Econometrics, Elsevier, vol. 188(1), pages 150-165.
    5. Stock, James H. & Watson, Mark W., 2014. "Estimating turning points using large data sets," Journal of Econometrics, Elsevier, vol. 178(P2), pages 368-381.
    6. Marta Bańbura, 2008. "Large Bayesian VARs," 2008 Meeting Papers 334, Society for Economic Dynamics.
    7. Dimitris Korobilis, 2013. "Var Forecasting Using Bayesian Variable Selection," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 204-230, March.
    8. Gary Koop & Dimitris Korobilis, 2012. "Forecasting Inflation Using Dynamic Model Averaging," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 53(3), pages 867-886, August.
    9. Hendry, David F. & Martinez, Andrew B., 2017. "Evaluating multi-step system forecasts with relatively few forecast-error observations," International Journal of Forecasting, Elsevier, vol. 33(2), pages 359-372.
    10. repec:taf:jnlbes:v:34:y:2016:i:4:p:574-589 is not listed on IDEAS
    11. James H. Stock & Mark W. Watson, 2005. "Implications of Dynamic Factor Models for VAR Analysis," NBER Working Papers 11467, National Bureau of Economic Research, Inc.
    12. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2006. "A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 499-526.
    13. repec:wly:japmet:v:25:y:2010:i:1:p:71-92 is not listed on IDEAS
    14. Korobilis, Dimitris, 2016. "Prior selection for panel vector autoregressions," Computational Statistics & Data Analysis, Elsevier, vol. 101(C), pages 110-120.
    15. Daniel Felix Ahelegbey & Monica Billio & Roberto Casarin, 2016. "Bayesian Graphical Models for STructural Vector Autoregressive Processes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(2), pages 357-386, March.
    16. Koop, Gary & Korobilis, Dimitris, 2013. "Large time-varying parameter VARs," Journal of Econometrics, Elsevier, vol. 177(2), pages 185-198.
    17. Jan J. J. Groen & Richard Paap & Francesco Ravazzolo, 2013. "Real-Time Inflation Forecasting in a Changing World," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(1), pages 29-44, January.
    18. Hyunyoung Choi & Hal Varian, 2012. "Predicting the Present with Google Trends," The Economic Record, The Economic Society of Australia, vol. 88(s1), pages 2-9, June.
    19. repec:taf:jnlasa:v:111:y:2016:i:516:p:1608-1622 is not listed on IDEAS
    20. repec:hrv:faseco:33192198 is not listed on IDEAS
    21. Bassetti, Federico & Casarin, Roberto & Leisen, Fabrizio, 2014. "Beta-product dependent Pitman–Yor processes for Bayesian inference," Journal of Econometrics, Elsevier, vol. 180(1), pages 49-72.
    22. Davide Pettenuzzo & Francesco Ravazzolo, 2016. "Optimal Portfolio Choice Under Decision‐Based Model Combinations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1312-1332, November.
    23. James Mitchell & Stephen G. Hall, 2005. "Evaluating, Comparing and Combining Density Forecasts Using the KLIC with an Application to the Bank of England and NIESR ‘Fan’ Charts of Inflation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(s1), pages 995-1033, December.
    24. Schorfheide, Frank, 2005. "VAR forecasting under misspecification," Journal of Econometrics, Elsevier, vol. 128(1), pages 99-136, September.
    25. Marta Banbura & Domenico Giannone & Lucrezia Reichlin, 2010. "Large Bayesian vector auto regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 71-92.
    26. Sims, Christopher A & Zha, Tao, 1998. "Bayesian Methods for Dynamic Multivariate Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 949-968, November.
    27. Knut Are Aastveit & Francesco Ravazzolo & Herman K. van Dijk, 2018. "Combined Density Nowcasting in an Uncertain Economic Environment," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(1), pages 131-145, January.
    28. Gefang, Deborah, 2014. "Bayesian doubly adaptive elastic-net Lasso for VAR shrinkage," International Journal of Forecasting, Elsevier, vol. 30(1), pages 1-11.
    29. Cubadda, Gianluca & Guardabascio, Barbara, 2019. "Representation, estimation and forecasting of the multivariate index-augmented autoregressive model," International Journal of Forecasting, Elsevier, vol. 35(1), pages 67-79.
    30. Tilmann Gneiting & Roopesh Ranjan, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(3), pages 411-422, July.
    31. Carriero, Andrea & Kapetanios, George & Marcellino, Massimiliano, 2016. "Structural analysis with Multivariate Autoregressive Index models," Journal of Econometrics, Elsevier, vol. 192(2), pages 332-348.
    32. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    33. George, Edward I. & Sun, Dongchu & Ni, Shawn, 2008. "Bayesian stochastic search for VAR model restrictions," Journal of Econometrics, Elsevier, vol. 142(1), pages 553-580, January.
    34. Conflitti, Cristina & De Mol, Christine & Giannone, Domenico, 2015. "Optimal combination of survey forecasts," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1096-1103.
    35. Diebold, Francis X. & Pauly, Peter, 1990. "The use of prior information in forecast combination," International Journal of Forecasting, Elsevier, vol. 6(4), pages 503-508, December.
    36. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    37. Qi, Xin & Luo, Ruiyan & Zhao, Hongyu, 2013. "Sparse principal component analysis by choice of norm," Journal of Multivariate Analysis, Elsevier, vol. 114(C), pages 127-160.
    38. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    39. Gneiting, Tilmann & Ranjan, Roopesh, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(3), pages 411-422.
    40. Dr. James Mitchell, 2005. "Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR ÔfanÕ charts of inflation," National Institute of Economic and Social Research (NIESR) Discussion Papers 253, National Institute of Economic and Social Research.
    41. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    42. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    43. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    44. Carvalho, Carlos M. & Chang, Jeffrey & Lucas, Joseph E. & Nevins, Joseph R. & Wang, Quanli & West, Mike, 2008. "High-Dimensional Sparse Factor Modeling: Applications in Gene Expression Genomics," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1438-1456.
    45. repec:taf:jnlasa:v:113:y:2018:i:522:p:675-685 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Factor models; Large datasets; Multivariate autoregressive models; Forecasting; Scoring rules; VAR models.;

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ven:wpaper:2018:18. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Geraldine Ludbrook). General contact details of provider: http://edirc.repec.org/data/dsvenit.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.