IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v175y2022ics0167947322001402.html
   My bibliography  Save this article

A high dimensional dissimilarity measure

Author

Listed:
  • Modarres, Reza

Abstract

A new dissimilarity measure for high-dimensional, low sample size settings to compare high dimensional probability distributions is proposed. The asymptotic behavior of the new dissimilarity index is studied theoretically. Numerical experiments from high dimensional distributions exhibit the usefulness of the method. The eigenvalues of the matrix of dissimilarities for comparing two high dimensional samples are determined and shown to be related to the asymptotic value of the dissimilarity index. A dissimilarity visualization plot that is useful for detection of outliers and change points is proposed and utilized to find the change points in S&P500 stock return data.

Suggested Citation

  • Modarres, Reza, 2022. "A high dimensional dissimilarity measure," Computational Statistics & Data Analysis, Elsevier, vol. 175(C).
    • Handle: RePEc:eee:csdana:v:175:y:2022:i:c:s0167947322001402
    DOI: 10.1016/j.csda.2022.107560
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167947322001402
    Download Restriction: Full text for ScienceDirect subscribers only.
    File URL: https://libkey.io/10.1016/j.csda.2022.107560?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Peter Hall & J. S. Marron & Amnon Neeman, 2005. "Geometric representation of high dimension, low sample size data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(3), pages 427-444, June.
    2. Biswas, Munmun & Ghosh, Anil K., 2014. "A nonparametric two-sample test applicable to high dimensional data," Journal of Multivariate Analysis, Elsevier, vol. 123(C), pages 160-171.
    3. Borysov, Petro & Hannig, Jan & Marron, J.S., 2014. "Asymptotics of hierarchical clustering for growing dimension," Journal of Multivariate Analysis, Elsevier, vol. 124(C), pages 465-479.
    4. Kazuyoshi Yata & Makoto Aoshima, 2020. "Geometric consistency of principal component scores for high‐dimensional mixture models and its application," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(3), pages 899-921, September.
    5. Reza Modarres, 2020. "Graphical Comparison of High‐Dimensional Distributions," International Statistical Review, International Statistical Institute, vol. 88(3), pages 698-714, December.
    6. B. O'Neill, 2014. "Some Useful Moment Results in Sampling Problems," The American Statistician, Taylor & Francis Journals, vol. 68(4), pages 282-296, November.
    7. Marco Marozzi & Amitava Mukherjee & Jan Kalina, 2020. "Interpoint distance tests for high-dimensional comparison studies," Journal of Applied Statistics, Taylor & Francis Journals, vol. 47(4), pages 653-665, March.
    8. Jeongyoun Ahn & J. S. Marron & Keith M. Muller & Yueh-Yun Chi, 2007. "The high-dimension, low-sample-size geometric representation holds under mild conditions," Biometrika, Biometrika Trust, vol. 94(3), pages 760-766.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nakayama, Yugo & Yata, Kazuyoshi & Aoshima, Makoto, 2021. "Clustering by principal component analysis with Gaussian kernel in high-dimension, low-sample-size settings," Journal of Multivariate Analysis, Elsevier, vol. 185(C).
    2. Jun Li, 2018. "Asymptotic normality of interpoint distances for high-dimensional data with applications to the two-sample problem," Biometrika, Biometrika Trust, vol. 105(3), pages 529-546.
    3. Paul, Biplab & De, Shyamal K. & Ghosh, Anil K., 2022. "Some clustering-based exact distribution-free k-sample tests applicable to high dimension, low sample size data," Journal of Multivariate Analysis, Elsevier, vol. 190(C).
    4. Modarres, Reza, 2023. "Analysis of distance matrices," Statistics & Probability Letters, Elsevier, vol. 193(C).
    5. Yata, Kazuyoshi & Aoshima, Makoto, 2013. "PCA consistency for the power spiked model in high-dimensional settings," Journal of Multivariate Analysis, Elsevier, vol. 122(C), pages 334-354.
    6. Jung, Sungkyu & Sen, Arusharka & Marron, J.S., 2012. "Boundary behavior in High Dimension, Low Sample Size asymptotics of PCA," Journal of Multivariate Analysis, Elsevier, vol. 109(C), pages 190-203.
    7. Wang, Shao-Hsuan & Huang, Su-Yun, 2022. "Perturbation theory for cross data matrix-based PCA," Journal of Multivariate Analysis, Elsevier, vol. 190(C).
    8. Kazuyoshi Yata & Makoto Aoshima, 2012. "Inference on High-Dimensional Mean Vectors with Fewer Observations Than the Dimension," Methodology and Computing in Applied Probability, Springer, vol. 14(3), pages 459-476, September.
    9. Saha, Enakshi & Sarkar, Soham & Ghosh, Anil K., 2017. "Some high-dimensional one-sample tests based on functions of interpoint distances," Journal of Multivariate Analysis, Elsevier, vol. 161(C), pages 83-95.
    10. Yugo Nakayama & Kazuyoshi Yata & Makoto Aoshima, 2020. "Bias-corrected support vector machine with Gaussian kernel in high-dimension, low-sample-size settings," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 72(5), pages 1257-1286, October.
    11. Shin-ichi Tsukada, 2019. "High dimensional two-sample test based on the inter-point distance," Computational Statistics, Springer, vol. 34(2), pages 599-615, June.
    12. Patrick K. Kimes & Yufeng Liu & David Neil Hayes & James Stephen Marron, 2017. "Statistical significance for hierarchical clustering," Biometrics, The International Biometric Society, vol. 73(3), pages 811-821, September.
    13. Borysov, Petro & Hannig, Jan & Marron, J.S., 2014. "Asymptotics of hierarchical clustering for growing dimension," Journal of Multivariate Analysis, Elsevier, vol. 124(C), pages 465-479.
    14. Chung, Hee Cheol & Ahn, Jeongyoun, 2021. "Subspace rotations for high-dimensional outlier detection," Journal of Multivariate Analysis, Elsevier, vol. 183(C).
    15. Mondal, Pronoy K. & Biswas, Munmun & Ghosh, Anil K., 2015. "On high dimensional two-sample tests based on nearest neighbors," Journal of Multivariate Analysis, Elsevier, vol. 141(C), pages 168-178.
    16. Shen, Dan & Shen, Haipeng & Marron, J.S., 2013. "Consistency of sparse PCA in High Dimension, Low Sample Size contexts," Journal of Multivariate Analysis, Elsevier, vol. 115(C), pages 317-333.
    17. Miecznikowski Jeffrey C. & Gaile Daniel P. & Chen Xiwei & Tritchler David L., 2016. "Identification of consistent functional genetic modules," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 15(1), pages 1-18, March.
    18. Makoto Aoshima & Kazuyoshi Yata, 2014. "A distance-based, misclassification rate adjusted classifier for multiclass, high-dimensional data," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 66(5), pages 983-1010, October.
    19. Kazuyoshi Yata & Makoto Aoshima, 2020. "Geometric consistency of principal component scores for high‐dimensional mixture models and its application," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(3), pages 899-921, September.
    20. Yata, Kazuyoshi & Aoshima, Makoto, 2012. "Effective PCA for high-dimension, low-sample-size data with noise reduction via geometric representations," Journal of Multivariate Analysis, Elsevier, vol. 105(1), pages 193-215.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:175:y:2022:i:c:s0167947322001402. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.