IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v151y2020ics0167947320300955.html
   My bibliography  Save this article

High-dimensional two-sample mean vectors test and support recovery with factor adjustment

Author

Listed:
  • He, Yong
  • Zhang, Mingjuan
  • Zhang, Xinsheng
  • Zhou, Wang

Abstract

Testing the equality of two mean vectors is a classical problem in multivariate analysis. In this article, we consider the test in the high-dimensional setting. Existing tests often assume that the covariance matrix (or its inverse) of the underlying variables is sparse, which is rarely true in social science due to the existence of latent common factors. In the article, we introduce a maximum-type test statistic based on the factor-adjusted data. The factor-adjustment step increases the signal-to-noise ratio and thus results in more powerful test. We obtain the limiting null distribution of the maximum-type test statistic, which is the extreme value distribution of type I. To overcome the well-known slow convergence rate of the test statistic’s distribution to the limiting extreme value distribution, we also propose a multiplier bootstrap method to improve the finite-sample performance. In addition, a multiple testing procedure with false discovery rate (FDR) control is proposed for identifying specific locations that differ significantly between the two groups. Thorough numerical studies are conducted to show the superiority of the test over other state-of-the-art tests. The performance of the test is also assessed through a real stock market dataset.

Suggested Citation

  • He, Yong & Zhang, Mingjuan & Zhang, Xinsheng & Zhou, Wang, 2020. "High-dimensional two-sample mean vectors test and support recovery with factor adjustment," Computational Statistics & Data Analysis, Elsevier, vol. 151(C).
    • Handle: RePEc:eee:csdana:v:151:y:2020:i:c:s0167947320300955
    DOI: 10.1016/j.csda.2020.107004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947320300955
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2020.107004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Johnstone, Iain M. & Lu, Arthur Yu, 2009. "On Consistency and Sparsity for Principal Components Analysis in High Dimensions," Journal of the American Statistical Association, American Statistical Association, vol. 104(486), pages 682-693.
    2. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    3. Jushan Bai & Serena Ng, 2002. "Determining the Number of Factors in Approximate Factor Models," Econometrica, Econometric Society, vol. 70(1), pages 191-221, January.
    4. Boivin, Jean & Ng, Serena, 2006. "Are more data always better for factor analysis?," Journal of Econometrics, Elsevier, vol. 132(1), pages 169-194, May.
    5. Jinyuan Chang & Qiwei Yao & Wen Zhou, 2017. "Testing for high-dimensional white noise using maximum cross-correlations," Biometrika, Biometrika Trust, vol. 104(1), pages 111-127.
    6. Kong, Xin-Bing & Liu, Zhi & Zhou, Wang, 2019. "A rank test for the number of factors with high-frequency data," Journal of Econometrics, Elsevier, vol. 211(2), pages 439-460.
    7. Seung C. Ahn & Alex R. Horenstein, 2013. "Eigenvalue Ratio Test for the Number of Factors," Econometrica, Econometric Society, vol. 81(3), pages 1203-1227, May.
    8. Alexei Onatski, 2009. "Testing Hypotheses About the Number of Factors in Large Factor Models," Econometrica, Econometric Society, vol. 77(5), pages 1447-1479, September.
    9. Jianqing Fan & Yuan Ke & Qiang Sun & Wen-Xin Zhou, 2019. "FarmTest: Factor-Adjusted Robust Multiple Testing With Approximate False Discovery Control," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(528), pages 1880-1893, October.
    10. Chen, Song Xi & Li, Jun & Zhong, Pingshou, 2014. "Two-Sample Tests for High Dimensional Means with Thresholding and Data Transformation," MPRA Paper 59815, University Library of Munich, Germany.
    11. Srivastava, Muni S., 2009. "A test for the mean vector with fewer observations than the dimension under non-normality," Journal of Multivariate Analysis, Elsevier, vol. 100(3), pages 518-532, March.
    12. Victor Chernozhukov & Denis Chetverikov & Kengo Kato, 2012. "Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors," Papers 1212.6906, arXiv.org, revised Jan 2018.
    13. Gongjun Xu & Lifeng Lin & Peng Wei & Wei Pan, 2016. "An adaptive two-sample test for high-dimensional means," Biometrika, Biometrika Trust, vol. 103(3), pages 609-624.
    14. Jinyuan Chang & Wen Zhou & Wen-Xin Zhou & Lan Wang, 2017. "Comparing large covariance matrices under weak conditions on the dependence structure and its application to gene clustering," Biometrics, The International Biometric Society, vol. 73(1), pages 31-41, March.
    15. Tony Cai & Weidong Liu & Yin Xia, 2013. "Two-Sample Covariance Matrix Testing and Support Recovery in High-Dimensional and Sparse Settings," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(501), pages 265-277, March.
    16. Chen, Song Xi & Qin, Yingli, 2010. "A Two Sample Test for High Dimensional Data with Applications to Gene-set Testing," MPRA Paper 59642, University Library of Munich, Germany.
    17. T. Tony Cai & Weidong Liu & Yin Xia, 2014. "Two-sample test of high dimensional means under dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(2), pages 349-372, March.
    18. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    19. Jianqing Fan & Xu Han, 2017. "Estimation of the false discovery proportion with unknown dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1143-1164, September.
    20. Chang, Jinyuan & Yao, Qiwei & Zhou, Wen, 2017. "Testing for high-dimensional white noise using maximum cross-correlations," LSE Research Online Documents on Economics 68531, London School of Economics and Political Science, LSE Library.
    21. Jinyuan Chang & Chao Zheng & Wen‐Xin Zhou & Wen Zhou, 2017. "Simulation‐based hypothesis testing of high dimensional means under covariance heterogeneity," Biometrics, The International Biometric Society, vol. 73(4), pages 1300-1310, December.
    22. Xin-Bing Kong, 2017. "On the number of common factors with high-frequency data," Biometrika, Biometrika Trust, vol. 104(2), pages 397-410.
    23. Fama, Eugene F. & French, Kenneth R., 2015. "A five-factor asset pricing model," Journal of Financial Economics, Elsevier, vol. 116(1), pages 1-22.
    24. Fama, Eugene F. & French, Kenneth R., 1993. "Common risk factors in the returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 33(1), pages 3-56, February.
    25. William F. Sharpe, 1964. "Capital Asset Prices: A Theory Of Market Equilibrium Under Conditions Of Risk," Journal of Finance, American Finance Association, vol. 19(3), pages 425-442, September.
    26. R. Cont, 2001. "Empirical properties of asset returns: stylized facts and statistical issues," Quantitative Finance, Taylor & Francis Journals, vol. 1(2), pages 223-236.
    27. Aurore Delaigle & Peter Hall & Jiashun Jin, 2011. "Robustness and accuracy of methods for high dimensional data analysis based on Student's t‐statistic," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 73(3), pages 283-301, June.
    28. Stock, James H & Watson, Mark W, 2002. "Macroeconomic Forecasting Using Diffusion Indexes," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(2), pages 147-162, April.
    29. Stock J.H. & Watson M.W., 2002. "Forecasting Using Principal Components From a Large Number of Predictors," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 1167-1179, December.
    30. Kong, Xin-Bing & Liu, Cheng, 2018. "Testing against constant factor loading matrix with large panel high-frequency data," Journal of Econometrics, Elsevier, vol. 204(2), pages 301-319.
    31. Srivastava, Muni S. & Du, Meng, 2008. "A test for the mean vector with fewer observations than the dimension," Journal of Multivariate Analysis, Elsevier, vol. 99(3), pages 386-402, March.
    32. Yu, Long & He, Yong & Zhang, Xinsheng, 2019. "Robust factor number specification for large-dimensional elliptical factor model," Journal of Multivariate Analysis, Elsevier, vol. 174(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fan, Jianqing & Xue, Lingzhou & Yao, Jiawei, 2017. "Sufficient forecasting using factor models," Journal of Econometrics, Elsevier, vol. 201(2), pages 292-306.
    2. Zhaoxing Gao & Ruey S. Tsay, 2021. "Divide-and-Conquer: A Distributed Hierarchical Factor Approach to Modeling Large-Scale Time Series Data," Papers 2103.14626, arXiv.org.
    3. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    4. Zhaoxing Gao & Ruey S. Tsay, 2020. "A Two-Way Transformed Factor Model for Matrix-Variate Time Series," Papers 2011.09029, arXiv.org.
    5. Bodnar, Taras & Reiß, Markus, 2016. "Exact and asymptotic tests on a factor model in low and large dimensions with applications," Journal of Multivariate Analysis, Elsevier, vol. 150(C), pages 125-151.
    6. Gao, Zhaoxing & Tsay, Ruey S., 2023. "A Two-Way Transformed Factor Model for Matrix-Variate Time Series," Econometrics and Statistics, Elsevier, vol. 27(C), pages 83-101.
    7. Sun, Yucheng & Xu, Wen & Zhang, Chuanhai, 2023. "Identifying latent factors based on high-frequency data," Journal of Econometrics, Elsevier, vol. 233(1), pages 251-270.
    8. Jianqing Fan & Kunpeng Li & Yuan Liao, 2020. "Recent Developments on Factor Models and its Applications in Econometric Learning," Papers 2009.10103, arXiv.org.
    9. Fan, Jianqing & Jiang, Bai & Sun, Qiang, 2022. "Bayesian factor-adjusted sparse regression," Journal of Econometrics, Elsevier, vol. 230(1), pages 3-19.
    10. Gagliardini, Patrick & Ossola, Elisa & Scaillet, Olivier, 2019. "A diagnostic criterion for approximate factor structure," Journal of Econometrics, Elsevier, vol. 212(2), pages 503-521.
    11. Stock, J.H. & Watson, M.W., 2016. "Dynamic Factor Models, Factor-Augmented Vector Autoregressions, and Structural Vector Autoregressions in Macroeconomics," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 415-525, Elsevier.
    12. Choi, Sung Hoon & Kim, Donggyu, 2023. "Large volatility matrix analysis using global and national factor models," Journal of Econometrics, Elsevier, vol. 235(2), pages 1917-1933.
    13. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," DSSR Discussion Papers 96, Graduate School of Economics and Management, Tohoku University.
    14. Catherine Doz & Peter Fuleky, 2019. "Dynamic Factor Models," Working Papers 2019-4, University of Hawaii Economic Research Organization, University of Hawaii at Manoa.
    15. Yoshimasa Uematsu & Takashi Yamagata, 2019. "Estimation of Weak Factor Models," ISER Discussion Paper 1053r, Institute of Social and Economic Research, Osaka University, revised Mar 2020.
    16. Jiang, Pan & Perez, M. Fabricio, 2021. "Follow the leader: Index tracking with factor models," Journal of Empirical Finance, Elsevier, vol. 64(C), pages 337-350.
    17. Fan, Jianqing & Ke, Yuan & Liao, Yuan, 2021. "Augmented factor models with applications to validating market risk factors and forecasting bond risk premia," Journal of Econometrics, Elsevier, vol. 222(1), pages 269-294.
    18. Yinchu Zhu, 2019. "How well can we learn large factor models without assuming strong factors?," Papers 1910.10382, arXiv.org, revised Nov 2019.
    19. Kwangmin Jung & Donggyu Kim & Seunghyeon Yu, 2021. "Next Generation Models for Portfolio Risk Management: An Approach Using Financial Big Data," Papers 2102.12783, arXiv.org, revised Feb 2022.
    20. Aït-Sahalia, Yacine & Xiu, Dacheng, 2017. "Using principal component analysis to estimate a high dimensional factor model with high-frequency data," Journal of Econometrics, Elsevier, vol. 201(2), pages 384-399.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:151:y:2020:i:c:s0167947320300955. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.