IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v55y2011i6p2090-2103.html
   My bibliography  Save this article

Identifying cluster number for subspace projected functional data clustering

Author

Listed:
  • Li, Pai-Ling
  • Chiou, Jeng-Min

Abstract

We propose a new approach, the forward functional testing (FFT) procedure, to cluster number selection for functional data clustering. We present a framework of subspace projected functional data clustering based on the functional multiplicative random-effects model, and propose to perform functional hypothesis tests on equivalence of cluster structures to identify the number of clusters. The aim is to find the maximum number of distinctive clusters while retaining significant differences between cluster structures. The null hypotheses comprise equalities between the cluster mean functions and between the sets of cluster eigenfunctions of the covariance kernels. Bootstrap resampling methods are developed to construct reference distributions of the derived test statistics. We compare several other cluster number selection criteria, extended from methods of multivariate data, with the proposed FFT procedure. The performance of the proposed approaches is examined by simulation studies, with applications to clustering gene expression profiles.

Suggested Citation

  • Li, Pai-Ling & Chiou, Jeng-Min, 2011. "Identifying cluster number for subspace projected functional data clustering," Computational Statistics & Data Analysis, Elsevier, vol. 55(6), pages 2090-2103, June.
    • Handle: RePEc:eee:csdana:v:55:y:2011:i:6:p:2090-2103
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(11)00003-X
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jeng‐Min Chiou & Pai‐Ling Li, 2007. "Functional clustering and identifying substructures of longitudinal data," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 69(4), pages 679-699, September.
    2. Hardy, Andre, 1996. "On the number of clusters," Computational Statistics & Data Analysis, Elsevier, vol. 23(1), pages 83-96, November.
    3. Fang Yao & Hans-Georg Müller & Andrew J. Clifford & Steven R. Dueker & Jennifer Follett & Yumei Lin & Bruce A. Buchholz & John S. Vogel, 2003. "Shrinkage Estimation for Functional Principal Component Scores with Application to the Population Kinetics of Plasma Folate," Biometrics, The International Biometric Society, vol. 59(3), pages 676-685, September.
    4. Glenn Milligan & Martha Cooper, 1985. "An examination of procedures for determining the number of clusters in a data set," Psychometrika, Springer;The Psychometric Society, vol. 50(2), pages 159-179, June.
    5. James G.M. & Sugar C.A., 2003. "Clustering for Sparsely Sampled Functional Data," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 397-408, January.
    6. Sangalli, Laura M. & Secchi, Piercesare & Vantini, Simone & Vitelli, Valeria, 2010. "k-mean alignment for curve clustering," Computational Statistics & Data Analysis, Elsevier, vol. 54(5), pages 1219-1233, May.
    7. Shubhankar Ray & Bani Mallick, 2006. "Functional clustering by Bayesian wavelet methods," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(2), pages 305-332, April.
    8. Zeng, Yujing & Garcia-Frias, Javier, 2006. "A novel HMM-based clustering algorithm for the analysis of gene expression time-course data," Computational Statistics & Data Analysis, Elsevier, vol. 50(9), pages 2472-2494, May.
    9. Chiou, Jeng-Min & Li, Pai-Ling, 2008. "Correlation-Based Functional Clustering via Subspace Projection," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1684-1692.
    10. Robert Tibshirani & Guenther Walther & Trevor Hastie, 2001. "Estimating the number of clusters in a data set via the gap statistic," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(2), pages 411-423.
    11. Ma, Ping & Zhong, Wenxuan, 2008. "Penalized Clustering of Large-Scale Functional Data With Multiple Covariates," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 625-636, June.
    12. Sugar, Catherine A. & James, Gareth M., 2003. "Finding the Number of Clusters in a Dataset: An Information-Theoretic Approach," Journal of the American Statistical Association, American Statistical Association, vol. 98, pages 750-763, January.
    13. Yao, Fang & Muller, Hans-Georg & Wang, Jane-Ling, 2005. "Functional Data Analysis for Sparse Longitudinal Data," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 577-590, June.
    14. Luis Angel Garcia-Escudero & Alfonso Gordaliza, 2005. "A Proposal for Robust Curve Clustering," Journal of Classification, Springer;The Classification Society, vol. 22(2), pages 185-201, September.
    15. Antonio Cuevas & Manuel Febrero & Ricardo Fraiman, 2007. "Robust estimation and classification for functional data via projection-based depth notions," Computational Statistics, Springer, vol. 22(3), pages 481-496, September.
    16. C. Abraham & P. A. Cornillon & E. Matzner‐Løber & N. Molinari, 2003. "Unsupervised Curve Clustering using B‐Splines," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 30(3), pages 581-595, September.
    17. Lawrence Hubert & Phipps Arabie, 1985. "Comparing partitions," Journal of Classification, Springer;The Classification Society, vol. 2(1), pages 193-218, December.
    18. Liu, Xueli & Yang, Mark C.K., 2009. "Simultaneous curve registration and clustering for functional data," Computational Statistics & Data Analysis, Elsevier, vol. 53(4), pages 1361-1376, February.
    19. Tarpey, Thaddeus, 2007. "Linear Transformations and the k-Means Clustering Algorithm: Applications to Clustering Curves," The American Statistician, American Statistical Association, vol. 61, pages 34-40, February.
    20. Mingjin Yan & Keying Ye, 2007. "Determining the Number of Clusters Using the Weighted Gap Statistic," Biometrics, The International Biometric Society, vol. 63(4), pages 1031-1037, December.
    21. Serban, Nicoleta & Wasserman, Larry, 2005. "CATS: Clustering After Transformation and Smoothing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 990-999, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Li, Pai-Ling & Chiou, Jeng-Min & Shyr, Yu, 2017. "Functional data classification using covariate-adjusted subspace projection," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 21-34.
    2. Berrendero, J.R. & Justel, A. & Svarc, M., 2011. "Principal components for multivariate functional data," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2619-2634, September.
    3. Alonso, Andrés M. & Casado, David & Romo, Juan, 2012. "Supervised classification for functional data: A weighted distance approach," Computational Statistics & Data Analysis, Elsevier, vol. 56(7), pages 2334-2346.
    4. Maria Ruiz-Medina & Rosa Espejo & Elvira Romano, 2014. "Spatial functional normal mixed effect approach for curve classification," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 8(3), pages 257-285, September.
    5. Rhoden, Imke & Weller, Daniel & Voit, Ann-Katrin, 2021. "Spatio-temporal dynamics of European innovation: An exploratory approach via multivariate functional data cluster analysis," Ruhr Economic Papers 926, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Peter Radchenko & Xinghao Qiao & Gareth M. James, 2015. "Index Models for Sparsely Sampled Functional Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(510), pages 824-836, June.
    7. Epifanio, Irene & Ventura-Campos, Noelia, 2011. "Functional data analysis in shape analysis," Computational Statistics & Data Analysis, Elsevier, vol. 55(9), pages 2758-2773, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kim, Joonpyo & Oh, Hee-Seok, 2020. "Pseudo-quantile functional data clustering," Journal of Multivariate Analysis, Elsevier, vol. 178(C).
    2. Slaets, Leen & Claeskens, Gerda & Hubert, Mia, 2012. "Phase and amplitude-based clustering for functional data," Computational Statistics & Data Analysis, Elsevier, vol. 56(7), pages 2360-2374.
    3. Jacques, Julien & Preda, Cristian, 2014. "Model-based clustering for multivariate functional data," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 92-106.
    4. Liu, Xueli & Yang, Mark C.K., 2009. "Simultaneous curve registration and clustering for functional data," Computational Statistics & Data Analysis, Elsevier, vol. 53(4), pages 1361-1376, February.
    5. Yaeji Lim & Hee-Seok Oh & Ying Kuen Cheung, 2019. "Multiscale Clustering for Functional Data," Journal of Classification, Springer;The Classification Society, vol. 36(2), pages 368-391, July.
    6. Michio Yamamoto, 2012. "Clustering of functional data in a low-dimensional subspace," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 6(3), pages 219-247, October.
    7. Vogt, Michael & Linton, Oliver, 2020. "Multiscale clustering of nonparametric regression curves," Journal of Econometrics, Elsevier, vol. 216(1), pages 305-325.
    8. Julien Jacques & Cristian Preda, 2014. "Functional data clustering: a survey," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 8(3), pages 231-255, September.
    9. Li, Pai-Ling & Chiou, Jeng-Min & Shyr, Yu, 2017. "Functional data classification using covariate-adjusted subspace projection," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 21-34.
    10. Michael Vogt & Oliver Linton, 2015. "Classification of nonparametric regression functions in heterogeneous panels," CeMMAP working papers CWP06/15, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    11. J. Fernando Vera & Rodrigo Macías, 2021. "On the Behaviour of K-Means Clustering of a Dissimilarity Matrix by Means of Full Multidimensional Scaling," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 489-513, June.
    12. Véronique Cariou & Stéphane Verdun & Emmanuelle Diaz & El Qannari & Evelyne Vigneau, 2009. "Comparison of three hypothesis testing approaches for the selection of the appropriate number of clusters of variables," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 3(3), pages 227-241, December.
    13. Michael Vogt & Oliver Linton, 2015. "Classification of nonparametric regression functions in heterogeneous panels," CeMMAP working papers 06/15, Institute for Fiscal Studies.
    14. Julian Rossbroich & Jeffrey Durieux & Tom F. Wilderjans, 2022. "Model Selection Strategies for Determining the Optimal Number of Overlapping Clusters in Additive Overlapping Partitional Clustering," Journal of Classification, Springer;The Classification Society, vol. 39(2), pages 264-301, July.
    15. Cho, Haeran & Goude, Yannig & Brossat, Xavier & Yao, Qiwei, 2013. "Modeling and forecasting daily electricity load curves: a hybrid approach," LSE Research Online Documents on Economics 49634, London School of Economics and Political Science, LSE Library.
    16. Ja‐Yoon Jang & Hee‐Seok Oh & Yaeji Lim & Ying Kuen Cheung, 2021. "Ensemble clustering for step data via binning," Biometrics, The International Biometric Society, vol. 77(1), pages 293-304, March.
    17. Fischer, Aurélie, 2011. "On the number of groups in clustering," Statistics & Probability Letters, Elsevier, vol. 81(12), pages 1771-1781.
    18. Qingzhi Zhong & Huazhen Lin & Yi Li, 2021. "Cluster non‐Gaussian functional data," Biometrics, The International Biometric Society, vol. 77(3), pages 852-865, September.
    19. Juhyun Park & Jeongyoun Ahn, 2017. "Clustering multivariate functional data with phase variation," Biometrics, The International Biometric Society, vol. 73(1), pages 324-333, March.
    20. Coffey, N. & Hinde, J. & Holian, E., 2014. "Clustering longitudinal profiles using P-splines and mixed effects models applied to time-course gene expression data," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 14-29.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:55:y:2011:i:6:p:2090-2103. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.