IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2306.14004.html

Latent Factor Analysis in Short Panels

Author

Listed:
  • Alain-Philippe Fortin
  • Patrick Gagliardini
  • Olivier Scaillet

Abstract

We develop a pseudo maximum likelihood method for latent factor analysis in short panels without imposing sphericity nor Gaussianity. We derive an asymptotically uniformly most powerful invariant test for the number of factors. On a large panel of monthly U.S. stock returns, we separate month after month systematic and idiosyncratic risks in short subperiods of bear vs. bull market. We observe an uptrend in the paths of total and idiosyncratic volatilities. The systematic risk explains a large part of the cross-sectional total variance in bear markets but is not driven by a single factor and not spanned by observed factors.

Suggested Citation

  • Alain-Philippe Fortin & Patrick Gagliardini & Olivier Scaillet, 2023. "Latent Factor Analysis in Short Panels," Papers 2306.14004, arXiv.org, revised Oct 2025.
    • Handle: RePEc:arx:papers:2306.14004
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2306.14004
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ang, Andrew & Liu, Jun & Schwarz, Krista, 2020. "Using Stocks or Portfolios in Tests of Factor Models," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 55(3), pages 709-750, May.
    2. Patrick Gagliardini & Elisa Ossola & Olivier Scaillet, 2016. "Time‐Varying Risk Premium in Large Cross‐Sectional Equity Data Sets," Econometrica, Econometric Society, vol. 84, pages 985-1046, May.
    3. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    4. Nicola Borri & Denis Chetverikov & Yukun Liu & Aleh Tsyvinski, 2024. "One Factor to Bind the Cross-Section of Returns," Papers 2404.08129, arXiv.org.
    5. Qihui Chen & Zheng Fang, 2018. "Improved Inference on the Rank of a Matrix," Papers 1812.02337, arXiv.org, revised Mar 2019.
    6. Lunde A. & Timmermann A., 2004. "Duration Dependence in Stock Prices: An Analysis of Bull and Bear Markets," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 253-273, July.
    7. Min Seong Kim, 2022. "Robust Inference for Diffusion-Index Forecasts With Cross-Sectionally Dependent Data," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(3), pages 1153-1167, June.
    8. Matteo Barigozzi & Marc Hallin, 2016. "Generalized dynamic factor models and volatilities: recovering the market volatility shocks," Econometrics Journal, Royal Economic Society, vol. 19(1), pages 33-60, February.
    9. Kapetanios, George, 2010. "A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(3), pages 397-409.
    10. Guanhao Feng & Stefano Giglio & Dacheng Xiu, 2020. "Taming the Factor Zoo: A Test of New Factors," Journal of Finance, American Finance Association, vol. 75(3), pages 1327-1370, June.
    11. Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2019. "A diagnostic criterion for approximate factor structure," Journal of Econometrics, Elsevier, vol. 212(2), pages 503-521.
    12. Stephen A. Ross, 2013. "The Arbitrage Theory of Capital Asset Pricing," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 1, pages 11-30, World Scientific Publishing Co. Pte. Ltd..
    13. Qihui Chen & Zheng Fang, 2019. "Improved inference on the rank of a matrix," Quantitative Economics, Econometric Society, vol. 10(4), pages 1787-1824, November.
    14. Fama, Eugene F. & French, Kenneth R., 2015. "A five-factor asset pricing model," Journal of Financial Economics, Elsevier, vol. 116(1), pages 1-22.
    15. Walter Ledermann, 1937. "On the rank of the reduced correlational matrix in multiple-factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 2(2), pages 85-93, June.
    16. Connor, Gregory & Korajczyk, Robert A, 1993. "A Test for the Number of Factors in an Approximate Factor Model," Journal of Finance, American Finance Association, vol. 48(4), pages 1263-1291, September.
    17. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    18. Al-Sadoon, Majid M., 2017. "A unifying theory of tests of rank," Journal of Econometrics, Elsevier, vol. 199(1), pages 49-62.
    19. Fama, Eugene F. & French, Kenneth R., 1993. "Common risk factors in the returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 33(1), pages 3-56, February.
    20. Marcus Hagedorn & Iourii Manovskii & Kurt Mitman, 2025. "The Impact of Unemployment Benefit Extensions on Employment: The 2014 Employment Miracle?," American Economic Journal: Macroeconomics, American Economic Association, vol. 17(4), pages 168-203, October.
    21. Chamberlain, Gary & Rothschild, Michael, 1983. "Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets," Econometrica, Econometric Society, vol. 51(5), pages 1281-1304, September.
    22. John H. Cochrane, 2011. "Presidential Address: Discount Rates," Journal of Finance, American Finance Association, vol. 66(4), pages 1047-1108, August.
    23. Joachim Freyberger, 2018. "Non-parametric Panel Data Models with Interactive Fixed Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(3), pages 1824-1851.
    24. Jianqing Fan & Alex Furger & Dacheng Xiu, 2016. "Incorporating Global Industrial Classification Standard Into Portfolio Allocation: A Simple Factor-Based Large Covariance Matrix Estimator With High-Frequency Data," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(4), pages 489-503, October.
    25. Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2020. "Estimation of large dimensional conditional factor models in finance," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 219-282, Elsevier.
    26. John Y. Campbell & Martin Lettau & Burton Malkiel & Yexiao Xu, 2023. "Idiosyncratic Equity Risk Two Decades Later," Critical Finance Review, now publishers, vol. 12(1-4), pages 203-223, August.
    27. Eric Renault & Thijs Van Der & Bas J M Werker, 2023. "Arbitrage Pricing Theory for Idiosyncratic Variance Factors," Journal of Financial Econometrics, Oxford University Press, vol. 21(5), pages 1403-1442.
    28. Ahn, Seung C. & Horenstein, Alex R. & Wang, Na, 2018. "Beta Matrix and Common Factors in Stock Returns," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 53(3), pages 1417-1440, June.
    29. Torben G. Andersen & Tim Bollerslev & Francis X. Diebold & Paul Labys, 2003. "Modeling and Forecasting Realized Volatility," Econometrica, Econometric Society, vol. 71(2), pages 579-625, March.
    30. E. Andreou & P. Gagliardini & E. Ghysels & M. Rubin, 2019. "Inference in Group Factor Models With an Application to Mixed‐Frequency Data," Econometrica, Econometric Society, vol. 87(4), pages 1267-1305, July.
    31. Alexei Onatski, 2010. "Determining the Number of Factors from Empirical Distribution of Eigenvalues," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 1004-1016, November.
    32. Shanken, Jay, 1992. "On the Estimation of Beta-Pricing Models," The Review of Financial Studies, Society for Financial Studies, vol. 5(1), pages 1-33.
    33. Paolo Zaffaroni, 2025. "Factor Models for Conditional Asset Pricing," Journal of Political Economy, University of Chicago Press, vol. 133(8), pages 2615-2642.
    34. Valentina Raponi & Cesare Robotti & Paolo Zaffaroni & Andrew Karolyi, 2020. "Testing Beta-Pricing Models Using Large Cross-Sections," The Review of Financial Studies, Society for Financial Studies, vol. 33(6), pages 2796-2842.
    35. Engle, Robert F., 1984. "Wald, likelihood ratio, and Lagrange multiplier tests in econometrics," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 13, pages 775-826, Elsevier.
    36. Tomohiro Ando & Jushan Bai, 2015. "Asset Pricing with a General Multifactor Structure," Journal of Financial Econometrics, Oxford University Press, vol. 13(3), pages 556-604.
    37. Mehmet Caner & Xu Han, 2014. "Selecting the Correct Number of Factors in Approximate Factor Models: The Large Panel Case With Group Bridge Estimators," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 32(3), pages 359-374, July.
    38. Matteo Barigozzi & Marc Hallin, 2026. "The Dynamic, the Static, and the Weak: Factor Models and the Analysis of High‐Dimensional Time Series," Journal of Time Series Analysis, Wiley Blackwell, vol. 47(1), pages 201-219, January.
    39. Seung C. Ahn & Alex R. Horenstein, 2013. "Eigenvalue Ratio Test for the Number of Factors," Econometrica, Econometric Society, vol. 81(3), pages 1203-1227, May.
    40. Alexei Onatski, 2009. "Testing Hypotheses About the Number of Factors in Large Factor Models," Econometrica, Econometric Society, vol. 77(5), pages 1447-1479, September.
    41. Carhart, Mark M, 1997. "On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March.
    42. Javier Maldonado & Esther Ruiz, 2021. "Accurate Confidence Regions for Principal Components Factors," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 83(6), pages 1432-1453, December.
    43. Kleibergen, Frank & Paap, Richard, 2006. "Generalized reduced rank tests using the singular value decomposition," Journal of Econometrics, Elsevier, vol. 133(1), pages 97-126, July.
    44. Nikolay Gospodinov & Raymond Kan & Cesare Robotti, 2014. "Misspecification-Robust Inference in Linear Asset-Pricing Models with Irrelevant Risk Factors," The Review of Financial Studies, Society for Financial Studies, vol. 27(7), pages 2139-2170.
    45. Gobillon, Laurent & Magnac, Thierry & Roux, Sébastien, 2022. "Lifecycle Wages and Human Capital Investments: Selection and Missing Data," CEPR Discussion Papers 16999, Centre for Economic Policy Research.
    46. Kim, Soohun & Skoulakis, Georgios, 2018. "Ex-post risk premia estimation and asset pricing tests using large cross sections: The regression-calibration approach," Journal of Econometrics, Elsevier, vol. 204(2), pages 159-188.
    47. Alain-Philippe Fortin & Patrick Gagliardini & Olivier Scaillet, 2022. "Eigenvalue tests for the number of latent factors in short panels," Papers 2210.16042, arXiv.org.
    48. Albert Satorra & Peter Bentler, 2001. "A scaled difference chi-square test statistic for moment structure analysis," Psychometrika, Springer;The Psychometric Society, vol. 66(4), pages 507-514, December.
    49. Andreou, E. & Gagliardini, P. & Ghysels, E. & Rubin, M., 2025. "Spanning latent and observable factors," Journal of Econometrics, Elsevier, vol. 248(C).
    50. Theis Ingerslev Jensen & Bryan Kelly & Lasse Heje Pedersen, 2023. "Is There a Replication Crisis in Finance?," Journal of Finance, American Finance Association, vol. 78(5), pages 2465-2518, October.
    51. Bai, Jushan & Ng, Serena, 2006. "Evaluating latent and observed factors in macroeconomics and finance," Journal of Econometrics, Elsevier, vol. 131(1-2), pages 507-537.
    52. Jushan Bai & Kunpeng Li, 2016. "Maximum Likelihood Estimation and Inference for Approximate Factor Models of High Dimension," The Review of Economics and Statistics, MIT Press, vol. 98(2), pages 298-309, May.
    53. Miravete, Eugenio J., 2011. "Convolution and composition of totally positive random variables in economics," Journal of Mathematical Economics, Elsevier, vol. 47(4-5), pages 479-490.
    54. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    55. Joseph P. Romano & Azeem M. Shaikh & Michael Wolf, 2010. "Hypothesis Testing in Econometrics," Annual Review of Economics, Annual Reviews, vol. 2(1), pages 75-104, September.
    56. Aigner, Dennis J. & Hsiao, Cheng & Kapteyn, Arie & Wansbeek, Tom, 1984. "Latent variable models in econometrics," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 23, pages 1321-1393, Elsevier.
    57. Ole E. Barndorff‐Nielsen & Neil Shephard, 2002. "Econometric analysis of realized volatility and its use in estimating stochastic volatility models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(2), pages 253-280, May.
    58. Yacine Aït-Sahalia & Jean Jacod, 2014. "High-Frequency Financial Econometrics," Economics Books, Princeton University Press, edition 1, number 10261, December.
    59. Jegadeesh, Narasimhan & Titman, Sheridan, 1993. "Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency," Journal of Finance, American Finance Association, vol. 48(1), pages 65-91, March.
    60. Albert Satorra & Peter Bentler, 2010. "Ensuring Positiveness of the Scaled Difference Chi-square Test Statistic," Psychometrika, Springer;The Psychometric Society, vol. 75(2), pages 243-248, June.
    61. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    62. Jushan Bai & Serena Ng, 2006. "Confidence Intervals for Diffusion Index Forecasts and Inference for Factor-Augmented Regressions," Econometrica, Econometric Society, vol. 74(4), pages 1133-1150, July.
    63. Connor, Gregory & Korajczyk, Robert A., 1986. "Performance measurement with the arbitrage pricing theory : A new framework for analysis," Journal of Financial Economics, Elsevier, vol. 15(3), pages 373-394, March.
    64. Miguel Sarzosa & Sergio Urzúa, 2021. "Bullying among adolescents: The role of skills," Quantitative Economics, Econometric Society, vol. 12(3), pages 945-980, July.
    65. Chamberlain, Gary, 1992. "Efficiency Bounds for Semiparametric Regression," Econometrica, Econometric Society, vol. 60(3), pages 567-596, May.
    66. Bai, Jushan, 2024. "Likelihood approach to dynamic panel models with interactive effects," Journal of Econometrics, Elsevier, vol. 240(1).
    67. Kelly, Bryan T. & Pruitt, Seth & Su, Yinan, 2019. "Characteristics are covariances: A unified model of risk and return," Journal of Financial Economics, Elsevier, vol. 134(3), pages 501-524.
    68. Lancaster, Tony, 2000. "The incidental parameter problem since 1948," Journal of Econometrics, Elsevier, vol. 95(2), pages 391-413, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ding, Yi & Engle, Robert & Li, Yingying & Zheng, Xinghua, 2025. "Multiplicative factor model for volatility," Journal of Econometrics, Elsevier, vol. 249(PB).
    2. Damir Filipovic & Paul Schneider, 2024. "Fundamental Properties of Linear Factor Models," Papers 2409.02521, arXiv.org, revised Sep 2026.
    3. Andreou, E. & Gagliardini, P. & Ghysels, E. & Rubin, M., 2025. "Spanning latent and observable factors," Journal of Econometrics, Elsevier, vol. 248(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2020. "Estimation of large dimensional conditional factor models in finance," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 219-282, Elsevier.
    2. Alain-Philippe Fortin & Patrick Gagliardini & O. Scaillet, 2022. "Eigenvalue tests for the number of latent factors in short panels," Swiss Finance Institute Research Paper Series 22-81, Swiss Finance Institute.
    3. Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2019. "A diagnostic criterion for approximate factor structure," Journal of Econometrics, Elsevier, vol. 212(2), pages 503-521.
    4. Chaieb, Ines & Langlois, Hugues & Scaillet, Olivier, 2021. "Factors and risk premia in individual international stock returns," Journal of Financial Economics, Elsevier, vol. 141(2), pages 669-692.
    5. Andreou, E. & Gagliardini, P. & Ghysels, E. & Rubin, M., 2025. "Spanning latent and observable factors," Journal of Econometrics, Elsevier, vol. 248(C).
    6. Gu, Shihao & Kelly, Bryan & Xiu, Dacheng, 2021. "Autoencoder asset pricing models," Journal of Econometrics, Elsevier, vol. 222(1), pages 429-450.
    7. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," ISER Discussion Paper 1053r, Institute of Social and Economic Research, The University of Osaka, revised Mar 2020.
    8. Stefano Giglio & Dacheng Xiu, 2017. "Inference on Risk Premia in the Presence of Omitted Factors," NBER Working Papers 23527, National Bureau of Economic Research, Inc.
    9. Matteo Bagnara, 2024. "Asset Pricing and Machine Learning: A critical review," Journal of Economic Surveys, Wiley Blackwell, vol. 38(1), pages 27-56, February.
    10. Zhipeng Liao & Yan Liu & Zhenzhen Xie, 2024. "Optimal Cross-Sectional Regression," Management Science, INFORMS, vol. 70(11), pages 7911-7942, November.
    11. repec:gnv:wpaper:unige:76321 is not listed on IDEAS
    12. Li, Kunpeng & Li, Qi & Lu, Lina, 2018. "Quasi maximum likelihood analysis of high dimensional constrained factor models," Journal of Econometrics, Elsevier, vol. 206(2), pages 574-612.
    13. Patrick Gagliardini & Elisa Ossola & Olivier Scaillet, 2016. "Time‐Varying Risk Premium in Large Cross‐Sectional Equity Data Sets," Econometrica, Econometric Society, vol. 84, pages 985-1046, May.
    14. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers halshs-02262202, HAL.
    15. Dai, Chaoxing & Lu, Kun & Xiu, Dacheng, 2019. "Knowing factors or factor loadings, or neither? Evaluating estimators of large covariance matrices with noisy and asynchronous data," Journal of Econometrics, Elsevier, vol. 208(1), pages 43-79.
    16. Aït-Sahalia, Yacine & Xiu, Dacheng, 2017. "Using principal component analysis to estimate a high dimensional factor model with high-frequency data," Journal of Econometrics, Elsevier, vol. 201(2), pages 384-399.
    17. Anatolyev, Stanislav & Mikusheva, Anna, 2022. "Factor models with many assets: Strong factors, weak factors, and the two-pass procedure," Journal of Econometrics, Elsevier, vol. 229(1), pages 103-126.
    18. Sun, Yucheng & Xu, Wen & Zhang, Chuanhai, 2023. "Identifying latent factors based on high-frequency data," Journal of Econometrics, Elsevier, vol. 233(1), pages 251-270.
    19. Bakalli, Gaetan & Guerrier, Stéphane & Scaillet, Olivier, 2023. "A penalized two-pass regression to predict stock returns with time-varying risk premia," Journal of Econometrics, Elsevier, vol. 237(2).
    20. Bai, Jushan & Ando, Tomohiro, 2013. "Multifactor asset pricing with a large number of observable risk factors and unobservable common and group-specific factors," MPRA Paper 52785, University Library of Munich, Germany, revised Dec 2013.
    21. Matteo Barigozzi & Marc Hallin, 2023. "Dynamic Factor Models: a Genealogy," Papers 2310.17278, arXiv.org, revised Jan 2024.

    More about this item

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C58 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Financial Econometrics
    • G12 - Financial Economics - - General Financial Markets - - - Asset Pricing; Trading Volume; Bond Interest Rates

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2306.14004. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.