IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2210.09828.html
   My bibliography  Save this paper

Modelling Large Dimensional Datasets with Markov Switching Factor Models

Author

Listed:
  • Matteo Barigozzi
  • Daniele Massacci

Abstract

We study a novel large dimensional approximate factor model with regime changes in the loadings driven by a latent first order Markov process. By exploiting the equivalent linear representation of the model, we first recover the latent factors by means of Principal Component Analysis. We then cast the model in state-space form, and we estimate loadings and transition probabilities through an EM algorithm based on a modified version of the Baum-Lindgren-Hamilton-Kim filter and smoother that makes use of the factors previously estimated. Our approach is appealing as it provides closed form expressions for all estimators. More importantly, it does not require knowledge of the true number of factors. We derive the theoretical properties of the proposed estimation procedure, and we show their good finite sample performance through a comprehensive set of Monte Carlo experiments. The empirical usefulness of our approach is illustrated through three applications to large U.S. datasets of stock returns, macroeconomic variables, and inflation indexes.

Suggested Citation

  • Matteo Barigozzi & Daniele Massacci, 2022. "Modelling Large Dimensional Datasets with Markov Switching Factor Models," Papers 2210.09828, arXiv.org, revised Dec 2024.
  • Handle: RePEc:arx:papers:2210.09828
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2210.09828
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Harding, Don & Pagan, Adrian, 2006. "Synchronization of cycles," Journal of Econometrics, Elsevier, vol. 132(1), pages 59-79, May.
    2. Leroux, Brian G., 1992. "Maximum-likelihood estimation for hidden Markov models," Stochastic Processes and their Applications, Elsevier, vol. 40(1), pages 127-143, February.
    3. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    4. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2021. "Economic Predictions With Big Data: The Illusion of Sparsity," Econometrica, Econometric Society, vol. 89(5), pages 2409-2437, September.
    5. Gabriel Perez‐Quiros & Allan Timmermann, 2000. "Firm Size and Cyclical Variations in Stock Returns," Journal of Finance, American Finance Association, vol. 55(3), pages 1229-1262, June.
    6. Giovanni Urga & Fa Wang, 2022. "Estimation and Inference for High Dimensional Factor Model with Regime Switching," Papers 2205.12126, arXiv.org, revised Apr 2023.
    7. Hansen, Bruce E, 1996. "Erratum: The Likelihood Ratio Test under Nonstandard Conditions: Testing the Markov Switching Model of GNP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(2), pages 195-198, March-Apr.
    8. Stock, James H. & Watson, Mark W., 2014. "Estimating turning points using large data sets," Journal of Econometrics, Elsevier, vol. 178(P2), pages 368-381.
    9. Lorenzo Trapani, 2018. "A Randomized Sequential Procedure to Determine the Number of Factors," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1341-1349, July.
    10. Bai, Jushan & Ng, Serena, 2013. "Principal components estimation and identification of static factors," Journal of Econometrics, Elsevier, vol. 176(1), pages 18-29.
    11. Matteo Barigozzi & Matteo Luciani, 2019. "Quasi Maximum Likelihood Estimation of Non-Stationary Large Approximate Dynamic Factor Models," Papers 1910.09841, arXiv.org.
    12. Forni, Mario & Hallin, Marc & Lippi, Marco & Zaffaroni, Paolo, 2017. "Dynamic factor models with infinite-dimensional factor space: Asymptotic analysis," Journal of Econometrics, Elsevier, vol. 199(1), pages 74-92.
    13. Andrew Ang & Allan Timmermann, 2012. "Regime Changes and Financial Markets," Annual Review of Financial Economics, Annual Reviews, vol. 4(1), pages 313-337, October.
    14. Andrew Ang & Geert Bekaert, 2002. "International Asset Allocation With Regime Shifts," The Review of Financial Studies, Society for Financial Studies, vol. 15(4), pages 1137-1187.
    15. Catherine Doz & Laurent Ferrara & Pierre-Alain Pionnier, 2020. "Business cycle dynamics after the Great Recession: An extended Markov-Switching Dynamic Factor Model," OECD Statistics Working Papers 2020/01, OECD Publishing.
    16. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    17. Kapetanios, George, 2010. "A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(3), pages 397-409.
    18. Massimo Guidolin, 2011. "Markov Switching Models in Empirical Finance," Advances in Econometrics, in: Missing Data Methods: Time-Series Methods and Applications, pages 1-86, Emerald Group Publishing Limited.
    19. Urga, Giovanni & Wang, Fa, 2024. "Estimation and inference for high dimensional factor model with regime switching," Journal of Econometrics, Elsevier, vol. 241(2).
    20. Michael W. McCracken & Serena Ng, 2016. "FRED-MD: A Monthly Database for Macroeconomic Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 574-589, October.
    21. Baltagi, Badi H. & Feng, Qu & Kao, Chihwa, 2016. "Estimation of heterogeneous panels with structural breaks," Journal of Econometrics, Elsevier, vol. 191(1), pages 176-195.
    22. Chauvet, Marcelle & Piger, Jeremy, 2008. "A Comparison of the Real-Time Performance of Business Cycle Dating Methods," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 42-49, January.
    23. Markus Pelger & Ruoxuan Xiong, 2022. "State-Varying Factor Models of Large Dimensions," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(3), pages 1315-1333, June.
    24. Catherine Doz & Domenico Giannone & Lucrezia Reichlin, 2012. "A Quasi–Maximum Likelihood Approach for Large, Approximate Dynamic Factor Models," The Review of Economics and Statistics, MIT Press, vol. 94(4), pages 1014-1024, November.
    25. Goldfeld, Stephen M. & Quandt, Richard E., 1973. "A Markov model for switching regressions," Journal of Econometrics, Elsevier, vol. 1(1), pages 3-15, March.
    26. Breitung, Jörg & Eickmeier, Sandra, 2011. "Testing for structural breaks in dynamic factor models," Journal of Econometrics, Elsevier, vol. 163(1), pages 71-84, July.
    27. Chen, Liang & Dolado, Juan J. & Gonzalo, Jesús, 2014. "Detecting big structural breaks in large factor models," Journal of Econometrics, Elsevier, vol. 180(1), pages 30-48.
    28. Motta, Giovanni & Hafner, Christian M. & von Sachs, Rainer, 2011. "Locally Stationary Factor Models: Identification And Nonparametric Estimation," Econometric Theory, Cambridge University Press, vol. 27(6), pages 1279-1319, December.
    29. Xu Cheng & Zhipeng Liao & Frank Schorfheide, 2016. "Shrinkage Estimation of High-Dimensional Factor Models with Structural Instabilities," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 83(4), pages 1511-1543.
    30. Perez-Quiros, Gabriel & Timmermann, Allan, 2001. "Business cycle asymmetries in stock returns: Evidence from higher order moments and conditional densities," Journal of Econometrics, Elsevier, vol. 103(1-2), pages 259-306, July.
    31. Chamberlain, Gary & Rothschild, Michael, 1983. "Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets," Econometrica, Econometric Society, vol. 51(5), pages 1281-1304, September.
    32. Donald Rubin & Dorothy Thayer, 1982. "EM algorithms for ML factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 47(1), pages 69-76, March.
    33. Barigozzi, Matteo & Trapani, Lorenzo, 2020. "Sequential testing for structural stability in approximate factor models," Stochastic Processes and their Applications, Elsevier, vol. 130(8), pages 5149-5187.
    34. Jushan Bai & Kunpeng Li, 2016. "Maximum Likelihood Estimation and Inference for Approximate Factor Models of High Dimension," The Review of Economics and Statistics, MIT Press, vol. 98(2), pages 298-309, May.
    35. Boivin, Jean & Ng, Serena, 2006. "Are more data always better for factor analysis?," Journal of Econometrics, Elsevier, vol. 132(1), pages 169-194, May.
    36. Massimo Guidolin & Allan Timmermann, 2006. "An econometric model of nonlinear dynamics in the joint distribution of stock and bond returns," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(1), pages 1-22, January.
    37. Massacci, Daniele, 2017. "Least squares estimation of large dimensional threshold factor models," Journal of Econometrics, Elsevier, vol. 197(1), pages 101-129.
    38. Kim, Chang-Jin, 1994. "Dynamic linear models with Markov-switching," Journal of Econometrics, Elsevier, vol. 60(1-2), pages 1-22.
    39. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    40. Alexei Onatski, 2010. "Determining the Number of Factors from Empirical Distribution of Eigenvalues," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 1004-1016, November.
    41. Han, Xu & Inoue, Atsushi, 2015. "Tests For Parameter Instability In Dynamic Factor Models," Econometric Theory, Cambridge University Press, vol. 31(5), pages 1117-1152, October.
    42. Massimo Guidolin & Allan Timmermann, 2008. "International asset allocation under regime switching, skew, and kurtosis preferences," The Review of Financial Studies, Society for Financial Studies, vol. 21(2), pages 889-935, April.
    43. Diebold, Francis X & Rudebusch, Glenn D, 1996. "Measuring Business Cycles: A Modern Perspective," The Review of Economics and Statistics, MIT Press, vol. 78(1), pages 67-77, February.
    44. Zhongjun Qu & Fan Zhuo, 2021. "Likelihood Ratio-Based Tests for Markov Regime Switching," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 88(2), pages 937-968.
    45. Marine Carrasco & Liang Hu & Werner Ploberger, 2014. "Optimal Test for Markov Switching Parameters," Econometrica, Econometric Society, vol. 82(2), pages 765-784, March.
    46. Seung C. Ahn & Alex R. Horenstein, 2013. "Eigenvalue Ratio Test for the Number of Factors," Econometrica, Econometric Society, vol. 81(3), pages 1203-1227, May.
    47. Hamilton, J.D., 2016. "Macroeconomic Regimes and Regime Shifts," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 163-201, Elsevier.
    48. Su, Liangjun & Wang, Xia, 2017. "On time-varying factor models: Estimation and testing," Journal of Econometrics, Elsevier, vol. 198(1), pages 84-101.
    49. Bai, Jushan, 1997. "Estimating Multiple Breaks One at a Time," Econometric Theory, Cambridge University Press, vol. 13(3), pages 315-352, June.
    50. Margaret M. McConnell & Gabriel Perez-Quiros, 2000. "Output fluctuations in the United States: what has changed since the early 1980s?," Proceedings, Federal Reserve Bank of San Francisco, issue Mar.
    51. Hie Joo Ahn & Matteo Luciani, 2020. "Common and Idiosyncratic Inflation," Finance and Economics Discussion Series 2020-024r1, Board of Governors of the Federal Reserve System (U.S.), revised 01 Aug 2024.
    52. Barigozzi, Matteo & Cho, Haeran & Fryzlewicz, Piotr, 2018. "Simultaneous multiple change-point and factor analysis for high-dimensional time series," Journal of Econometrics, Elsevier, vol. 206(1), pages 187-225.
    53. Watson, Mark W. & Stock, James H., 2014. "Estimating turning points using large data sets," Scholarly Articles 33192198, Harvard University Department of Economics.
    54. Chib, Siddhartha, 1998. "Estimation and comparison of multiple change-point models," Journal of Econometrics, Elsevier, vol. 86(2), pages 221-241, June.
    55. Duan, Jiangtao & Bai, Jushan & Han, Xu, 2023. "Quasi-maximum likelihood estimation of break point in high-dimensional factor models," Journal of Econometrics, Elsevier, vol. 233(1), pages 209-236.
    56. Arthur F. Burns & Wesley C. Mitchell, 1946. "Measuring Business Cycles," NBER Books, National Bureau of Economic Research, Inc, number burn46-1, October.
    57. Eugene F. Fama & Kenneth R. French, 2016. "Dissecting Anomalies with a Five-Factor Model," The Review of Financial Studies, Society for Financial Studies, vol. 29(1), pages 69-103.
    58. Daniele Massacci, 2023. "Testing for Regime Changes in Portfolios with a Large Number of Assets: A Robust Approach to Factor Heteroskedasticity," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 316-367.
    59. Corradi, Valentina & Swanson, Norman R., 2014. "Testing for structural stability of factor augmented forecasting models," Journal of Econometrics, Elsevier, vol. 182(1), pages 100-118.
    60. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    61. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    62. Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-384, March.
    63. Chauvet, Marcelle, 1998. "An Econometric Characterization of Business Cycle Dynamics with Factor Structure and Regime Switching," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 969-996, November.
    64. Alessi, Lucia & Barigozzi, Matteo & Capasso, Marco, 2010. "Improved penalization for determining the number of factors in approximate factor models," Statistics & Probability Letters, Elsevier, vol. 80(23-24), pages 1806-1813, December.
    65. Hallin, Marc & Liska, Roman, 2007. "Determining the Number of Factors in the General Dynamic Factor Model," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 603-617, June.
    66. Hamilton, James D., 1996. "Specification testing in Markov-switching time-series models," Journal of Econometrics, Elsevier, vol. 70(1), pages 127-157, January.
    67. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    68. Connor, Gregory & Korajczyk, Robert A., 1986. "Performance measurement with the arbitrage pricing theory : A new framework for analysis," Journal of Financial Economics, Elsevier, vol. 15(3), pages 373-394, March.
    69. Jushan Bai & Xu Han, 2016. "Structural Changes in High Dimensional Factor Models," Frontiers of Economics in China-Selected Publications from Chinese Universities, Higher Education Press, vol. 11(1), pages 9-39, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Matteo Barigozzi, 2022. "On Estimation and Inference of Large Approximate Dynamic Factor Models via the Principal Component Analysis," Papers 2211.01921, arXiv.org, revised Jul 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers 2019-4, University of Hawaii Economic Research Organization, University of Hawaii at Manoa.
    2. Matteo Barigozzi & Marc Hallin, 2023. "Dynamic Factor Models: a Genealogy," Papers 2310.17278, arXiv.org, revised Jan 2024.
    3. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    4. Urga, Giovanni & Wang, Fa, 2022. "Estimation and inference for high dimensional factor model with regime switching," MPRA Paper 113172, University Library of Munich, Germany.
    5. Urga, Giovanni & Wang, Fa, 2024. "Estimation and inference for high dimensional factor model with regime switching," Journal of Econometrics, Elsevier, vol. 241(2).
    6. Barigozzi, Matteo & Trapani, Lorenzo, 2020. "Sequential testing for structural stability in approximate factor models," Stochastic Processes and their Applications, Elsevier, vol. 130(8), pages 5149-5187.
    7. Urga, Giovanni & Wang, Fa, 2022. "Estimation and Inference for High Dimensional Factor Model with Regime Switching," MPRA Paper 117012, University Library of Munich, Germany, revised 10 Apr 2023.
    8. Luke Hartigan, 2015. "Changes in the Factor Structure of the U.S. Economy: Permanent Breaks or Business Cycle Regimes?," Discussion Papers 2015-17, School of Economics, The University of New South Wales.
    9. Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2020. "Estimation of large dimensional conditional factor models in finance," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 219-282, Elsevier.
    10. Ma, Chenchen & Tu, Yundong, 2023. "Group fused Lasso for large factor models with multiple structural breaks," Journal of Econometrics, Elsevier, vol. 233(1), pages 132-154.
    11. Jianqing Fan & Kunpeng Li & Yuan Liao, 2020. "Recent Developments on Factor Models and its Applications in Econometric Learning," Papers 2009.10103, arXiv.org.
    12. Baltagi, Badi H. & Kao, Chihwa & Wang, Fa, 2021. "Estimating and testing high dimensional factor models with multiple structural changes," Journal of Econometrics, Elsevier, vol. 220(2), pages 349-365.
    13. Bai, Jushan & Duan, Jiangtao & Han, Xu, 2024. "The likelihood ratio test for structural changes in factor models," Journal of Econometrics, Elsevier, vol. 238(2).
    14. Poncela, Pilar & Ruiz, Esther & Miranda, Karen, 2021. "Factor extraction using Kalman filter and smoothing: This is not just another survey," International Journal of Forecasting, Elsevier, vol. 37(4), pages 1399-1425.
    15. Bai, Jushan & Duan, Jiangtao & Han, Xu, 2024. "Reprint of: The likelihood ratio test for structural changes in factor models," Journal of Econometrics, Elsevier, vol. 244(2).
    16. Karim Barhoumi & Olivier Darné & Laurent Ferrara, 2014. "Dynamic factor models: A review of the literature," OECD Journal: Journal of Business Cycle Measurement and Analysis, OECD Publishing, Centre for International Research on Economic Tendency Surveys, vol. 2013(2), pages 73-107.
    17. Barigozzi, Matteo & Cho, Haeran & Fryzlewicz, Piotr, 2018. "Simultaneous multiple change-point and factor analysis for high-dimensional time series," Journal of Econometrics, Elsevier, vol. 206(1), pages 187-225.
    18. Matteo Barigozzi, 2023. "Quasi Maximum Likelihood Estimation of High-Dimensional Factor Models: A Critical Review," Papers 2303.11777, arXiv.org, revised May 2024.
    19. Luke Hartigan & James Morley, 2020. "A Factor Model Analysis of the Australian Economy and the Effects of Inflation Targeting," The Economic Record, The Economic Society of Australia, vol. 96(314), pages 271-293, September.
    20. Duan, Jiangtao & Bai, Jushan & Han, Xu, 2023. "Quasi-maximum likelihood estimation of break point in high-dimensional factor models," Journal of Econometrics, Elsevier, vol. 233(1), pages 209-236.

    More about this item

    JEL classification:

    • C34 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Truncated and Censored Models; Switching Regression Models
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • E3 - Macroeconomics and Monetary Economics - - Prices, Business Fluctuations, and Cycles
    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2210.09828. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.