IDEAS home Printed from https://ideas.repec.org/a/ahq/wpaper/v24y2020i2p66-103.html
   My bibliography  Save this article

A Scoring Rule for Factor and Autoregressive Models Under Misspecification

Author

Listed:
  • Roberto Casarin

    (University Ca’ Foscari of Venice)

  • Fausto Corradin

    (GRETA Associates, University Ca’ Foscari of Venice)

  • Francesco Ravazzolo

    (Free University of Bozen-Bolzano, Italy)

  • Nguyen Domenico Sartore

    (University Ca’ Foscari of Venice)

  • Wing-Keung Wong

    (Department of Finance, Fintech Center, Big Data Research Center, Asia University, Taiwan)

Abstract

. Factor models (FM) are now widely used for forecasting with large set of time series. Another class of models, which can be easily estimated and used in a large dimensional setting, is multivariate autoregressive models (MAR), where independent autoregressive processes are assumed for the series in the panel. When applied to big data, the estimation, model selection and combination of both models can be time consuming. We assume both FM and MAR models are misspecified and provide a scoring rule which can be evaluated on an initial training sample to either select or combine the models in forecasting exercises on the whole sample. Some numerical illustrations are provided both on simulated data and on well known large economic datasets. The empirical results show that the frequency of the true positive signals is larger when FM and MAR forecasting performances differ substantially and it decreases as the horizon increases.

Suggested Citation

  • Roberto Casarin & Fausto Corradin & Francesco Ravazzolo & Nguyen Domenico Sartore & Wing-Keung Wong, 2020. "A Scoring Rule for Factor and Autoregressive Models Under Misspecification," International Association of Decision Sciences, Asia University, Taiwan, vol. 24(2), pages 66-103, June.
  • Handle: RePEc:ahq:wpaper:v:24:y:2020:i:2:p:66-103
    as

    Download full text from publisher

    File URL: http://iadsb.info/wp-content/uploads/papers/2020/A-Scoring-Rule-for-Factor-and-Autoregressive-Models-Under-Misspecification.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hyunyoung Choi & Hal Varian, 2012. "Predicting the Present with Google Trends," The Economic Record, The Economic Society of Australia, vol. 88(s1), pages 2-9, June.
    2. Roberto Casarin & Stefano Grassi & Francesco Ravazzolo & Herman K. van Dijk, 2015. "Dynamic predictive density combinations for large data sets in economics and finance," Working Paper 2015/12, Norges Bank.
    3. Federico Bassetti & Roberto Casarin & Francesco Ravazzolo, 2018. "Bayesian Nonparametric Calibration and Combination of Predictive Distributions," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 675-685, April.
    4. Marta Banbura & Domenico Giannone & Lucrezia Reichlin, 2010. "Large Bayesian vector auto regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 71-92.
    5. Dimitris Korobilis, 2013. "Var Forecasting Using Bayesian Variable Selection," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 204-230, March.
    6. Kapetanios, G. & Mitchell, J. & Price, S. & Fawcett, N., 2015. "Generalised density forecast combinations," Journal of Econometrics, Elsevier, vol. 188(1), pages 150-165.
    7. Monica Billio & Roberto Casarin & Francesco Ravazzolo & Herman K. van Dijk, 2013. "Interactions between Eurozone and US Booms and Busts: A Bayesian Panel Markov-switching VAR Model," Tinbergen Institute Discussion Papers 13-142/III, Tinbergen Institute, revised 01 Nov 2014.
    8. Billio, Monica & Casarin, Roberto & Rossini, Luca, 2019. "Bayesian nonparametric sparse VAR models," Journal of Econometrics, Elsevier, vol. 212(1), pages 97-115.
    9. Gary M. Koop, 2013. "Forecasting with Medium and Large Bayesian VARS," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(2), pages 177-203, March.
    10. Davide Pettenuzzo & Francesco Ravazzolo, 2016. "Optimal Portfolio Choice Under Decision‐Based Model Combinations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1312-1332, November.
    11. Billio, Monica & Casarin, Roberto & Ravazzolo, Francesco & van Dijk, Herman K., 2013. "Time-varying combinations of predictive densities using nonlinear filtering," Journal of Econometrics, Elsevier, vol. 177(2), pages 213-232.
    12. Koop, Gary & Korobilis, Dimitris & Pettenuzzo, Davide, 2019. "Bayesian compressed vector autoregressions," Journal of Econometrics, Elsevier, vol. 210(1), pages 135-154.
    13. Stock, James H. & Watson, Mark W., 2014. "Estimating turning points using large data sets," Journal of Econometrics, Elsevier, vol. 178(P2), pages 368-381.
    14. Jan J. J. Groen & Richard Paap & Francesco Ravazzolo, 2013. "Real-Time Inflation Forecasting in a Changing World," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(1), pages 29-44, January.
    15. Diebold, Francis X. & Pauly, Peter, 1990. "The use of prior information in forecast combination," International Journal of Forecasting, Elsevier, vol. 6(4), pages 503-508, December.
    16. Gary Koop & Dimitris Korobilis, 2012. "Forecasting Inflation Using Dynamic Model Averaging," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 53(3), pages 867-886, August.
    17. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    18. Veronika Ročková & Edward I. George, 2016. "Fast Bayesian Factor Analysis via Automatic Rotations to Sparsity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1608-1622, October.
    19. Qi, Xin & Luo, Ruiyan & Zhao, Hongyu, 2013. "Sparse principal component analysis by choice of norm," Journal of Multivariate Analysis, Elsevier, vol. 114(C), pages 127-160.
    20. Conflitti, Cristina & De Mol, Christine & Giannone, Domenico, 2015. "Optimal combination of survey forecasts," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1096-1103.
    21. repec:hrv:faseco:33192198 is not listed on IDEAS
    22. Knut Are Aastveit & Francesco Ravazzolo & Herman K. van Dijk, 2018. "Combined Density Nowcasting in an Uncertain Economic Environment," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(1), pages 131-145, January.
    23. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    24. Gneiting, Tilmann & Ranjan, Roopesh, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(3), pages 411-422.
    25. Hendry, David F. & Martinez, Andrew B., 2017. "Evaluating multi-step system forecasts with relatively few forecast-error observations," International Journal of Forecasting, Elsevier, vol. 33(2), pages 359-372.
    26. Bassetti, Federico & Casarin, Roberto & Leisen, Fabrizio, 2014. "Beta-product dependent Pitman–Yor processes for Bayesian inference," Journal of Econometrics, Elsevier, vol. 180(1), pages 49-72.
    27. Koop, Gary & Korobilis, Dimitris, 2013. "Large time-varying parameter VARs," Journal of Econometrics, Elsevier, vol. 177(2), pages 185-198.
    28. Gefang, Deborah, 2014. "Bayesian doubly adaptive elastic-net Lasso for VAR shrinkage," International Journal of Forecasting, Elsevier, vol. 30(1), pages 1-11.
    29. Bianchi, Daniele & Billio, Monica & Casarin, Roberto & Guidolin, Massimo, 2019. "Modeling systemic risk with Markov Switching Graphical SUR models," Journal of Econometrics, Elsevier, vol. 210(1), pages 58-74.
    30. Cubadda, Gianluca & Guardabascio, Barbara, 2019. "Representation, estimation and forecasting of the multivariate index-augmented autoregressive model," International Journal of Forecasting, Elsevier, vol. 35(1), pages 67-79.
    31. James H. Stock & Mark W. Watson, 2005. "Implications of Dynamic Factor Models for VAR Analysis," NBER Working Papers 11467, National Bureau of Economic Research, Inc.
    32. Marcellino, Massimiliano & Stock, James H. & Watson, Mark W., 2006. "A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series," Journal of Econometrics, Elsevier, vol. 135(1-2), pages 499-526.
    33. Dr. James Mitchell, 2005. "Evaluating, comparing and combining density forecasts using the KLIC with an application to the Bank of England and NIESR ÔfanÕ charts of inflation," National Institute of Economic and Social Research (NIESR) Discussion Papers 253, National Institute of Economic and Social Research.
    34. Korobilis, Dimitris, 2016. "Prior selection for panel vector autoregressions," Computational Statistics & Data Analysis, Elsevier, vol. 101(C), pages 110-120.
    35. Carriero, Andrea & Kapetanios, George & Marcellino, Massimiliano, 2016. "Structural analysis with Multivariate Autoregressive Index models," Journal of Econometrics, Elsevier, vol. 192(2), pages 332-348.
    36. Daniel Felix Ahelegbey & Monica Billio & Roberto Casarin, 2016. "Bayesian Graphical Models for STructural Vector Autoregressive Processes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(2), pages 357-386, March.
    37. Tilmann Gneiting & Roopesh Ranjan, 2011. "Comparing Density Forecasts Using Threshold- and Quantile-Weighted Scoring Rules," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(3), pages 411-422, July.
    38. Marta Banbura & Domenico Giannone & Lucrezia Reichlin, 2010. "Large Bayesian vector auto regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 25(1), pages 71-92.
    39. James Mitchell & Stephen G. Hall, 2005. "Evaluating, Comparing and Combining Density Forecasts Using the KLIC with an Application to the Bank of England and NIESR ‘Fan’ Charts of Inflation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 67(s1), pages 995-1033, December.
    40. Mark W. Watson & James H. Stock, 2004. "Combination forecasts of output growth in a seven-country data set," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 23(6), pages 405-430.
    41. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    42. Schorfheide, Frank, 2005. "VAR forecasting under misspecification," Journal of Econometrics, Elsevier, vol. 128(1), pages 99-136, September.
    43. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    44. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    45. Marcelo C. Medeiros & Gabriel F. R. Vasconcelos & Álvaro Veiga & Eduardo Zilberman, 2021. "Forecasting Inflation in a Data-Rich Environment: The Benefits of Machine Learning Methods," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 98-119, January.
    46. Sims, Christopher A & Zha, Tao, 1998. "Bayesian Methods for Dynamic Multivariate Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 949-968, November.
    47. Carvalho, Carlos M. & Chang, Jeffrey & Lucas, Joseph E. & Nevins, Joseph R. & Wang, Quanli & West, Mike, 2008. "High-Dimensional Sparse Factor Modeling: Applications in Gene Expression Genomics," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1438-1456.
    48. George, Edward I. & Sun, Dongchu & Ni, Shawn, 2008. "Bayesian stochastic search for VAR model restrictions," Journal of Econometrics, Elsevier, vol. 142(1), pages 553-580, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Roberto Casarin & Stefano Grassi & Francesco Ravazzolo & Herman K. van Dijk, 2015. "Dynamic predictive density combinations for large data sets in economics and finance," Working Paper 2015/12, Norges Bank.
    2. Barbara Rossi, 2019. "Forecasting in the Presence of Instabilities: How Do We Know Whether Models Predict Well and How to Improve Them," Working Papers 1162, Barcelona Graduate School of Economics.
    3. Billio, Monica & Casarin, Roberto & Rossini, Luca, 2019. "Bayesian nonparametric sparse VAR models," Journal of Econometrics, Elsevier, vol. 212(1), pages 97-115.
    4. Roberto Casarin & Stefano Grassi & Francesco Ravazzolo & Herman K. van Dijk, 2020. "A Bayesian Dynamic Compositional Model for Large Density Combinations in Finance," Working Paper series 20-27, Rimini Centre for Economic Analysis.
    5. Roberto Casarin & Stefano Grassi & Francesco Ravazzolo & Herman K. van Dijk, 2021. "A Bayesian Dynamic Compositional Model for Large Density Combinations in Finance," Tinbergen Institute Discussion Papers 21-016/III, Tinbergen Institute.
    6. Davide Pettenuzzo & Francesco Ravazzolo, 2016. "Optimal Portfolio Choice Under Decision‐Based Model Combinations," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1312-1332, November.
    7. Federico Bassetti & Roberto Casarin & Francesco Ravazzolo, 2019. "Density Forecasting," BEMPS - Bozen Economics & Management Paper Series BEMPS59, Faculty of Economics and Management at the Free University of Bozen.
    8. Korobilis, Dimitris, 2018. "Machine Learning Macroeconometrics A Primer," Essex Finance Centre Working Papers 22666, University of Essex, Essex Business School.
    9. Monica Billio & Roberto Casarin & Luca Rossini, 2016. "Bayesian nonparametric sparse seemingly unrelated regression model (SUR)," Working Papers 2016:20, Department of Economics, University of Venice "Ca' Foscari".
    10. Magnus Reif, 2020. "Macroeconomics, Nonlinearities, and the Business Cycle," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 87, December.
    11. Daniel Felix Ahelegbey & Monica Billio & Roberto Casarin, 2016. "Sparse Graphical Vector Autoregression: A Bayesian Approach," Annals of Economics and Statistics, GENES, issue 123-124, pages 333-361.
    12. Roberto Casarin & Stefano Grassi & Francesco Ravazzolo & Herman K. van Dijk, 2019. "Forecast density combinations with dynamic learning for large data sets in economics and finance," Working Paper 2019/7, Norges Bank.
    13. Koop, Gary & Korobilis, Dimitris & Pettenuzzo, Davide, 2019. "Bayesian compressed vector autoregressions," Journal of Econometrics, Elsevier, vol. 210(1), pages 135-154.
    14. Berg, Tim O. & Henzel, Steffen R., 2015. "Point and density forecasts for the euro area using Bayesian VARs," International Journal of Forecasting, Elsevier, vol. 31(4), pages 1067-1095.
    15. Billio, Monica & Casarin, Roberto & Ravazzolo, Francesco & van Dijk, Herman K., 2013. "Time-varying combinations of predictive densities using nonlinear filtering," Journal of Econometrics, Elsevier, vol. 177(2), pages 213-232.
    16. Michael S. Smith & Shaun P. Vahey, 2016. "Asymmetric Forecast Densities for U.S. Macroeconomic Variables from a Gaussian Copula Model of Cross-Sectional and Serial Dependence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 416-434, July.
    17. Gary Koop, 2012. "Using VARs and TVP-VARs with Many Macroeconomic Variables," Central European Journal of Economic Modelling and Econometrics, Central European Journal of Economic Modelling and Econometrics, vol. 4(3), pages 143-167, September.
    18. Knut Are Aastveit & Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2017. "Have Standard VARS Remained Stable Since the Crisis?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(5), pages 931-951, August.
    19. Gregor Kastner & Florian Huber, 2020. "Sparse Bayesian vector autoregressions in huge dimensions," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(7), pages 1142-1165, November.
    20. repec:gam:jecnmx:v:4:y:2016:i:1:p:17:d:65855 is not listed on IDEAS
    21. Raffaella Giacomini & Barbara Rossi, 2015. "Forecasting in Nonstationary Environments: What Works and What Doesn't in Reduced-Form and Structural Models," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 207-229, August.

    More about this item

    Keywords

    Factor models; Large datasets; Multivariate autoregressive models; Forecasting; Scoring rules; VAR models;
    All these keywords.

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ahq:wpaper:v:24:y:2020:i:2:p:66-103. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Vincent Pan). General contact details of provider: https://edirc.repec.org/data/dfasitw.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.