IDEAS home Printed from https://ideas.repec.org/a/spr/aistmt/v73y2021i4d10.1007_s10463-020-00776-x.html
   My bibliography  Save this article

Determining the number of canonical correlation pairs for high-dimensional vectors

Author

Listed:
  • Jiasen Zheng

    (Renmin University of China)

  • Lixing Zhu

    (Hong Kong Baptist University
    Beijing Normal University)

Abstract

For two random vectors whose dimensions are both proportional to the sample size, we in this paper propose two ridge ratio criteria to determine the number of canonical correlation pairs. The criteria are, respectively, based on eigenvalue difference-based and centered eigenvalue-based ridge ratios. Unlike existing methods, the criteria make the ratio at the index we want to identify stick out to show a visualized “valley-cliff” pattern and thus can adequately avoid the local optimal solutions that often occur in the eigenvalues multiplicity cases. The numerical studies also suggest its advantage over existing scree plot-based method that is not a visualization method and more seriously underestimates the number of pairs than the proposed ones and the AIC and $$C_p$$ C p criteria that often extremely over-estimate the number, and the BIC criterion that has very serious underestimation problem. A real data set is analyzed for illustration.

Suggested Citation

  • Jiasen Zheng & Lixing Zhu, 2021. "Determining the number of canonical correlation pairs for high-dimensional vectors," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(4), pages 737-756, August.
    • Handle: RePEc:spr:aistmt:v:73:y:2021:i:4:d:10.1007_s10463-020-00776-x
    DOI: 10.1007/s10463-020-00776-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10463-020-00776-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10463-020-00776-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Zhu, Xuehu & Guo, Xu & Wang, Tao & Zhu, Lixing, 2020. "Dimensionality determination: A thresholding double ridge ratio approach," Computational Statistics & Data Analysis, Elsevier, vol. 146(C).
    2. Christopher R Cabanski & Yuan Qi & Xiaoying Yin & Eric Bair & Michele C Hayward & Cheng Fan & Jianying Li & Matthew D Wilkerson & J S Marron & Charles M Perou & D Neil Hayes, 2010. "SWISS MADE: Standardized WithIn Class Sum of Squares to Evaluate Methodologies and Dataset Elements," PLOS ONE, Public Library of Science, vol. 5(3), pages 1-13, March.
    3. Fujikoshi, Yasunori & Sakurai, Tetsuro, 2009. "High-dimensional asymptotic expansions for the distributions of canonical correlations," Journal of Multivariate Analysis, Elsevier, vol. 100(1), pages 231-242, January.
    4. Gunderson, Brenda K. & Muirhead, Robb J., 1997. "On Estimating the Dimensionality in Canonical Correlation Analysis," Journal of Multivariate Analysis, Elsevier, vol. 62(1), pages 121-136, July.
    5. Headrick, Todd C., 2002. "Fast fifth-order polynomial transforms for generating univariate and multivariate nonnormal distributions," Computational Statistics & Data Analysis, Elsevier, vol. 40(4), pages 685-711, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nielsen, Morten Orregaard & Shimotsu, Katsumi, 2007. "Determining the cointegrating rank in nonstationary fractional systems by the exact local Whittle approach," Journal of Econometrics, Elsevier, vol. 141(2), pages 574-596, December.
    2. Max Auerswald & Morten Moshagen, 2015. "Generating Correlated, Non-normally Distributed Data Using a Non-linear Structural Model," Psychometrika, Springer;The Psychometric Society, vol. 80(4), pages 920-937, December.
    3. Mohan D. Pant & Todd C. Headrick, 2017. "Simulating Uniform- and Triangular- Based Double Power Method Distributions," Journal of Statistical and Econometric Methods, SCIENPRESS Ltd, vol. 6(1), pages 1-1.
    4. Fujikoshi, Yasunori & Sakurai, Tetsuro, 2016. "High-dimensional consistency of rank estimation criteria in multivariate linear model," Journal of Multivariate Analysis, Elsevier, vol. 149(C), pages 199-212.
    5. Nicolas Depraetere & Martina Vandebroek, 2014. "Order selection in finite mixtures of linear regressions," Statistical Papers, Springer, vol. 55(3), pages 871-911, August.
    6. Engle, Robert F. & Marcucci, Juri, 2006. "A long-run Pure Variance Common Features model for the common volatilities of the Dow Jones," Journal of Econometrics, Elsevier, vol. 132(1), pages 7-42, May.
    7. repec:jss:jstsof:19:i03 is not listed on IDEAS
    8. Hakan Demirtas & Robab Ahmadian & Sema Atis & Fatma Ezgi Can & Ilker Ercan, 2016. "A nonnormal look at polychoric correlations: modeling the change in correlations before and after discretization," Computational Statistics, Springer, vol. 31(4), pages 1385-1401, December.
    9. Robinson, Peter M. & Yajima, Yoshihiro, 2002. "Determination of cointegrating rank in fractional systems," Journal of Econometrics, Elsevier, vol. 106(2), pages 217-241, February.
    10. Willa Chen & Clifford Hurvich, 2004. "Semiparametric Estimation of Fractional Cointegrating Subspaces," Econometrics 0412007, University Library of Munich, Germany.
    11. Junmin Liu & Deli Zhu & Luoyao Yu & Xuehu Zhu, 2023. "Specification testing of partially linear single-index models: a groupwise dimension reduction-based adaptive-to-model approach," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 32(1), pages 232-262, March.
    12. Headrick, Todd C. & Mugdadi, Abdel, 2006. "On simulating multivariate non-normal distributions from the generalized lambda distribution," Computational Statistics & Data Analysis, Elsevier, vol. 50(11), pages 3343-3353, July.
    13. Wei Luo & Bing Li, 2016. "Combining eigenvalues and variation of eigenvectors for order determination," Biometrika, Biometrika Trust, vol. 103(4), pages 875-887.
    14. Nagahara, Yuichi, 2004. "A method of simulating multivariate nonnormal distributions by the Pearson distribution system and estimation," Computational Statistics & Data Analysis, Elsevier, vol. 47(1), pages 1-29, August.
    15. Zeng, Yicheng & Zhu, Lixing, 2023. "Order determination for spiked-type models with a divergent number of spikes," Computational Statistics & Data Analysis, Elsevier, vol. 182(C).
    16. Paul Dudgeon, 2017. "Some Improvements in Confidence Intervals for Standardized Regression Coefficients," Psychometrika, Springer;The Psychometric Society, vol. 82(4), pages 928-951, December.
    17. Hakan Demirtas, 2016. "A Note on the Relationship Between the Phi Coefficient and the Tetrachoric Correlation Under Nonnormal Underlying Distributions," The American Statistician, Taylor & Francis Journals, vol. 70(2), pages 143-148, May.
    18. Xuehu Zhu & Jun Lu & Jun Zhang & Lixing Zhu, 2021. "Testing for conditional independence: A groupwise dimension reduction‐based adaptive‐to‐model approach," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 48(2), pages 549-576, June.
    19. Fosgerau, Mogens & Mabit, Stefan L., 2013. "Easy and flexible mixture distributions," Economics Letters, Elsevier, vol. 120(2), pages 206-210.
    20. Oscar L. Olvera Astivia & Bruno D. Zumbo, 2019. "A Note on the Solution Multiplicity of the Vale–Maurelli Intermediate Correlation Equation," Journal of Educational and Behavioral Statistics, , vol. 44(2), pages 127-143, April.
    21. Sui Peng & Huixiang Chen & Yong Lin & Tong Shu & Xingyu Lin & Junjie Tang & Wenyuan Li & Weijie Wu, 2019. "Probabilistic Power Flow for Hybrid AC/DC Grids with Ninth-Order Polynomial Normal Transformation and Inherited Latin Hypercube Sampling," Energies, MDPI, vol. 12(16), pages 1-21, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:aistmt:v:73:y:2021:i:4:d:10.1007_s10463-020-00776-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.