IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v40y2025i7d10.1007_s00180-025-01611-8.html
   My bibliography  Save this article

Hierarchical relations among principal component and factor analysis procedures elucidated from a comprehensive model

Author

Listed:
  • Kohei Adachi

    (Kyoto Women’s University)

Abstract

In this review article, the term “hierarchy” is related to constrained-ness, but not to superiority. Procedures A and B forming a hierarchy means that A is a constrained variant of B or vice versa. A goal of this article is to present a hierarchy of principal component analysis (PCA) and factor analysis (FA) procedures, which follows from a comprehensive FA (CompFA) model. This model can be regarded as a hybrid of PCA and prevalent FA models. First, we show how a non-random version of the CompFA model leads to the following hierarchy: PCA is a constrained variant of completely decomposed FA, which itself is a constrained variant of matrix decomposition FA. Then, we prove that a random version of the CompFA model leads to minimum rank FA (MRFA) and constraining MRFA leads to random PCA (RPCA), so as to present the following hierarchy: Probabilistic PCA is a constrained variant of prevalent FA, and the latter is a constrained variant of RPCA, which is itself a constrained variant of MRFA. Finally, this hierarchy and the above hierarchy following from the non-random version are unified into one. We further utilize the unified hierarchy to present a strategy for selecting a procedure suitable to a data set.

Suggested Citation

  • Kohei Adachi, 2025. "Hierarchical relations among principal component and factor analysis procedures elucidated from a comprehensive model," Computational Statistics, Springer, vol. 40(7), pages 3911-3946, September.
    • Handle: RePEc:spr:compst:v:40:y:2025:i:7:d:10.1007_s00180-025-01611-8
    DOI: 10.1007/s00180-025-01611-8
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-025-01611-8
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00180-025-01611-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Michael E. Tipping & Christopher M. Bishop, 1999. "Probabilistic Principal Component Analysis," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 61(3), pages 611-622.
    2. Schneeweiss, H. & Mathes, H., 1995. "Factor Analysis and Principal Components," Journal of Multivariate Analysis, Elsevier, vol. 55(1), pages 105-124, October.
    3. Stegeman, Alwin, 2016. "A new method for simultaneous estimation of the factor model parameters, factor scores, and unique parts," Computational Statistics & Data Analysis, Elsevier, vol. 99(C), pages 189-203.
    4. Kohei Adachi, 2013. "Factor Analysis with EM Algorithm Never Gives Improper Solutions when Sample Covariance and Initial Parameter Matrices Are Proper," Psychometrika, Springer;The Psychometric Society, vol. 78(2), pages 380-394, April.
    5. Kohei Adachi, 2022. "Factor Analysis Procedures Revisited from the Comprehensive Model with Unique Factors Decomposed into Specific Factors and Errors," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 967-991, September.
    6. Carl Eckart & Gale Young, 1936. "The approximation of one matrix by another of lower rank," Psychometrika, Springer;The Psychometric Society, vol. 1(3), pages 211-218, September.
    7. Henry Kaiser, 1958. "The varimax criterion for analytic rotation in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 23(3), pages 187-200, September.
    8. Kohei Uno & Kohei Adachi & Nickolay T. Trendafilov, 2019. "Clustered Common Factor Exploration in Factor Analysis," Psychometrika, Springer;The Psychometric Society, vol. 84(4), pages 1048-1067, December.
    9. Alexander Shapiro & Jos Berge, 2002. "Statistical inference of minimum rank factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 67(1), pages 79-94, March.
    10. Donald Rubin & Dorothy Thayer, 1982. "EM algorithms for ML factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 47(1), pages 69-76, March.
    11. Robert Jennrich & Stephen Robinson, 1969. "A Newton-Raphson algorithm for maximum likelihood factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 34(1), pages 111-123, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kohei Adachi, 2022. "Factor Analysis Procedures Revisited from the Comprehensive Model with Unique Factors Decomposed into Specific Factors and Errors," Psychometrika, Springer;The Psychometric Society, vol. 87(3), pages 967-991, September.
    2. Lorenzo Finesso & Peter Spreij, 2016. "Factor analysis models via I-divergence optimization," Psychometrika, Springer;The Psychometric Society, vol. 81(3), pages 702-726, September.
    3. Thomas Despois & Catherine Doz, 2023. "Identifying and interpreting the factors in factor models via sparsity: Different approaches," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 533-555, June.
    4. Jin, Shaobo & Moustaki, Irini & Yang-Wallentin, Fan, 2018. "Approximated penalized maximum likelihood for exploratory factor analysis: an orthogonal case," LSE Research Online Documents on Economics 88118, London School of Economics and Political Science, LSE Library.
    5. Matteo Barigozzi, 2023. "Asymptotic equivalence of Principal Components and Quasi Maximum Likelihood estimators in Large Approximate Factor Models," Papers 2307.09864, arXiv.org, revised Jun 2024.
    6. Johannes Burge & Priyank Jaini, 2017. "Accuracy Maximization Analysis for Sensory-Perceptual Tasks: Computational Improvements, Filter Robustness, and Coding Advantages for Scaled Additive Noise," PLOS Computational Biology, Public Library of Science, vol. 13(2), pages 1-32, February.
    7. Jianhua Zhao & Changchun Shang & Shulan Li & Ling Xin & Philip L. H. Yu, 2025. "Choosing the number of factors in factor analysis with incomplete data via a novel hierarchical Bayesian information criterion," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 19(1), pages 209-235, March.
    8. Masanori Ichikawa & Sadanori Konishi, 1995. "Application of the bootstrap methods in factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 60(1), pages 77-93, March.
    9. Sundberg, Rolf & Feldmann, Uwe, 2016. "Exploratory factor analysis—Parameter estimation and scores prediction with high-dimensional data," Journal of Multivariate Analysis, Elsevier, vol. 148(C), pages 49-59.
    10. Zhao, Jianhua & Shi, Lei, 2014. "Automated learning of factor analysis with complete and incomplete data," Computational Statistics & Data Analysis, Elsevier, vol. 72(C), pages 205-218.
    11. Kei Hirose & Yoshikazu Terada, 2023. "Sparse and Simple Structure Estimation via Prenet Penalization," Psychometrika, Springer;The Psychometric Society, vol. 88(4), pages 1381-1406, December.
    12. Rosember Guerra-Urzola & Katrijn Van Deun & Juan C. Vera & Klaas Sijtsma, 2021. "A Guide for Sparse PCA: Model Comparison and Applications," Psychometrika, Springer;The Psychometric Society, vol. 86(4), pages 893-919, December.
    13. Marie Chavent & Vanessa Kuentz-Simonet & Jérôme Saracco, 2012. "Orthogonal rotation in PCAMIX," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 6(2), pages 131-146, July.
    14. Veronika Ročková & Edward I. George, 2016. "Fast Bayesian Factor Analysis via Automatic Rotations to Sparsity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1608-1622, October.
    15. Yi-Hao Kao & Benjamin Van Roy, 2014. "Directed Principal Component Analysis," Operations Research, INFORMS, vol. 62(4), pages 957-972, August.
    16. Kei Hirose & Miyuki Imada, 2018. "Sparse factor regression via penalized maximum likelihood estimation," Statistical Papers, Springer, vol. 59(2), pages 633-662, June.
    17. Guerra Urzola, Rosember & Van Deun, Katrijn & Vera, J. C. & Sijtsma, K., 2021. "A guide for sparse PCA : Model comparison and applications," Other publications TiSEM 4d35b931-7f49-444b-b92f-a, Tilburg University, School of Economics and Management.
    18. Matteo Barigozzi & Matteo Luciani, 2019. "Quasi Maximum Likelihood Estimation and Inference of Large Approximate Dynamic Factor Models via the EM algorithm," Papers 1910.03821, arXiv.org, revised Sep 2024.
    19. Dorota Toczydlowska & Gareth W. Peters & Man Chung Fung & Pavel V. Shevchenko, 2017. "Stochastic Period and Cohort Effect State-Space Mortality Models Incorporating Demographic Factors via Probabilistic Robust Principal Components," Risks, MDPI, vol. 5(3), pages 1-77, July.
    20. Thomas Despois & Catherine Doz, 2022. "Identifying and interpreting the factors in factor models via sparsity : Different approaches," Working Papers halshs-03626503, HAL.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:40:y:2025:i:7:d:10.1007_s00180-025-01611-8. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.