IDEAS home Printed from https://ideas.repec.org/a/spr/jclass/v33y2016i2d10.1007_s00357-016-9208-4.html
   My bibliography  Save this article

A Survey on Feature Weighting Based K-Means Algorithms

Author

Listed:
  • Renato Cordeiro Amorim

    (University of Hertfordshire)

Abstract

In a real-world data set, there is always the possibility, rather high in our opinion, that different features may have different degrees of relevance. Most machine learning algorithms deal with this fact by either selecting or deselecting features in the data preprocessing phase. However, we maintain that even among relevant features there may be different degrees of relevance, and this should be taken into account during the clustering process. With over 50 years of history, K-Means is arguably the most popular partitional clustering algorithm there is. The first K-Means based clustering algorithm to compute feature weights was designed just over 30 years ago. Various such algorithms have been designed since but there has not been, to our knowledge, a survey integrating empirical evidence of cluster recovery ability, common flaws, and possible directions for future research. This paper elaborates on the concept of feature weighting and addresses these issues by critically analyzing some of the most popular, or innovative, feature weighting mechanisms based in K-Means.

Suggested Citation

  • Renato Cordeiro Amorim, 2016. "A Survey on Feature Weighting Based K-Means Algorithms," Journal of Classification, Springer;The Classification Society, vol. 33(2), pages 210-242, July.
    • Handle: RePEc:spr:jclass:v:33:y:2016:i:2:d:10.1007_s00357-016-9208-4
    DOI: 10.1007/s00357-016-9208-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00357-016-9208-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00357-016-9208-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Richard Bellman, 1957. "On a Dynamic Programming Approach to the Caterer Problem--I," Management Science, INFORMS, vol. 3(3), pages 270-278, April.
    2. R. Gnanadesikan & J. Kettenring & S. Tsao, 1995. "Weighting and selection of variables for cluster analysis," Journal of Classification, Springer;The Classification Society, vol. 12(1), pages 113-136, March.
    3. Fuad Aleskerov & Boris Goldengorin & Panos M. Pardalos (ed.), 2014. "Clusters, Orders, and Trees: Methods and Applications," Springer Optimization and Its Applications, Springer, edition 127, number 978-1-4939-0742-7, September.
    4. Renato Cordeiro Amorim & Boris Mirkin, 2014. "Selecting the Minkowski Exponent for Intelligent K-Means with Feature Weighting," Springer Optimization and Its Applications, in: Fuad Aleskerov & Boris Goldengorin & Panos M. Pardalos (ed.), Clusters, Orders, and Trees: Methods and Applications, edition 127, pages 103-117, Springer.
    5. Wayne DeSarbo & J. Carroll & Linda Clark & Paul Green, 1984. "Synthesized clustering: A method for amalgamating alternative clustering bases with differential weighting of variables," Psychometrika, Springer;The Psychometric Society, vol. 49(1), pages 57-78, March.
    6. Paul Green & Jonathan Kim & Frank Carmone, 1990. "A preliminary study of optimal variable weighting in k-means clustering," Journal of Classification, Springer;The Classification Society, vol. 7(2), pages 271-285, September.
    7. Lawrence Hubert & Phipps Arabie, 1985. "Comparing partitions," Journal of Classification, Springer;The Classification Society, vol. 2(1), pages 193-218, December.
    8. Wayne DeSarbo & William Cron, 1988. "A maximum likelihood methodology for clusterwise linear regression," Journal of Classification, Springer;The Classification Society, vol. 5(2), pages 249-282, September.
    9. Wayne DeSarbo & Vijay Mahajan, 1984. "Constrained classification: The use of a priori information in cluster analysis," Psychometrika, Springer;The Psychometric Society, vol. 49(2), pages 187-215, June.
    10. Geert Soete, 1986. "Optimal variable weighting for ultrametric and additive tree clustering," Quality & Quantity: International Journal of Methodology, Springer, vol. 20(2), pages 169-180, June.
    11. Geert Soete, 1988. "OVWTRE: A program for optimal variable weighting for ultrametric and additive tree fitting," Journal of Classification, Springer;The Classification Society, vol. 5(1), pages 101-104, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Renato Cordeiro Amorim & Vladimir Makarenkov & Boris Mirkin, 2020. "Core Clustering as a Tool for Tackling Noise in Cluster Labels," Journal of Classification, Springer;The Classification Society, vol. 37(1), pages 143-157, April.
    2. Jaehong Yu & Hua Zhong & Seoung Bum Kim, 2020. "An Ensemble Feature Ranking Algorithm for Clustering Analysis," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 462-489, July.
    3. Hossein Baloochian & Hamid Reza Ghaffary, 2019. "Multiclass Classification Based on Multi-criteria Decision-making," Journal of Classification, Springer;The Classification Society, vol. 36(1), pages 140-151, April.
    4. Kim, Nam Hyok & He, Feng & Zhang, Hongjie & Hong, Kwon Ryong & Ri, Kwang-Chol, 2023. "A data envelopment analysis-based clustering approach under dynamic situations," European Journal of Operational Research, Elsevier, vol. 311(1), pages 251-262.
    5. Salvatore Ingrassia & Antonio Punzo, 2020. "Cluster Validation for Mixtures of Regressions via the Total Sum of Squares Decomposition," Journal of Classification, Springer;The Classification Society, vol. 37(2), pages 526-547, July.
    6. Chakraborty, Saptarshi & Das, Swagatam, 2018. "Simultaneous variable weighting and determining the number of clusters—A weighted Gaussian means algorithm," Statistics & Probability Letters, Elsevier, vol. 137(C), pages 148-156.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tsai, Chieh-Yuan & Chiu, Chuang-Cheng, 2008. "Developing a feature weight self-adjustment mechanism for a K-means clustering algorithm," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4658-4672, June.
    2. Renato Amorim, 2015. "Feature Relevance in Ward’s Hierarchical Clustering Using the L p Norm," Journal of Classification, Springer;The Classification Society, vol. 32(1), pages 46-62, April.
    3. Susan Brudvig & Michael J. Brusco & J. Dennis Cradit, 2019. "Joint selection of variables and clusters: recovering the underlying structure of marketing data," Journal of Marketing Analytics, Palgrave Macmillan, vol. 7(1), pages 1-12, March.
    4. Michael Brusco & J. Cradit, 2001. "A variable-selection heuristic for K-means clustering," Psychometrika, Springer;The Psychometric Society, vol. 66(2), pages 249-270, June.
    5. Douglas Steinley & Michael Brusco, 2008. "Selection of Variables in Cluster Analysis: An Empirical Comparison of Eight Procedures," Psychometrika, Springer;The Psychometric Society, vol. 73(1), pages 125-144, March.
    6. Wayne DeSarbo & Richard Oliver & Arvind Rangaswamy, 1989. "A simulated annealing methodology for clusterwise linear regression," Psychometrika, Springer;The Psychometric Society, vol. 54(4), pages 707-736, September.
    7. Dolnicar, Sara & Grün, Bettina & Leisch, Friedrich, 2016. "Increasing sample size compensates for data problems in segmentation studies," Journal of Business Research, Elsevier, vol. 69(2), pages 992-999.
    8. Giuseppe RICCIARDO LAMONICA, 2002. "La funzionalita' nelle zone omogenee delle Marche," Working Papers 165, Universita' Politecnica delle Marche (I), Dipartimento di Scienze Economiche e Sociali.
    9. Yu Ding & Wayne S. DeSarbo & Dominique M. Hanssens & Kamel Jedidi & John G. Lynch & Donald R. Lehmann, 2020. "The past, present, and future of measurement and methods in marketing analysis," Marketing Letters, Springer, vol. 31(2), pages 175-186, September.
    10. J. Fernando Vera & Rodrigo Macías, 2021. "On the Behaviour of K-Means Clustering of a Dissimilarity Matrix by Means of Full Multidimensional Scaling," Psychometrika, Springer;The Psychometric Society, vol. 86(2), pages 489-513, June.
    11. Sanjeena Subedi & Drew Neish & Stephen Bak & Zeny Feng, 2020. "Cluster analysis of microbiome data by using mixtures of Dirichlet–multinomial regression models," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 69(5), pages 1163-1187, November.
    12. Salvatore D. Tomarchio & Paul D. McNicholas & Antonio Punzo, 2021. "Matrix Normal Cluster-Weighted Models," Journal of Classification, Springer;The Classification Society, vol. 38(3), pages 556-575, October.
    13. Yaling Deng & Shuliang Zou & Daming You, 2018. "Theoretical Guidance on Evacuation Decisions after a Big Nuclear Accident under the Assumption That Evacuation Is Desirable," Sustainability, MDPI, vol. 10(9), pages 1-14, August.
    14. Francesco Dotto & Alessio Farcomeni & Luis Angel García-Escudero & Agustín Mayo-Iscar, 2017. "A fuzzy approach to robust regression clustering," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 11(4), pages 691-710, December.
    15. Sanjeena Subedi & Antonio Punzo & Salvatore Ingrassia & Paul McNicholas, 2013. "Clustering and classification via cluster-weighted factor analyzers," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 7(1), pages 5-40, March.
    16. Gabriele Perrone & Gabriele Soffritti, 2023. "Seemingly unrelated clusterwise linear regression for contaminated data," Statistical Papers, Springer, vol. 64(3), pages 883-921, June.
    17. R. Gnanadesikan & J. Kettenring & S. Tsao, 1995. "Weighting and selection of variables for cluster analysis," Journal of Classification, Springer;The Classification Society, vol. 12(1), pages 113-136, March.
    18. Stéphanie Bougeard & Hervé Abdi & Gilbert Saporta & Ndèye Niang, 2018. "Clusterwise analysis for multiblock component methods," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 12(2), pages 285-313, June.
    19. Jan-Michael Becker & Christian Ringle & Marko Sarstedt & Franziska Völckner, 2015. "How collinearity affects mixture regression results," Marketing Letters, Springer, vol. 26(4), pages 643-659, December.
    20. Marbac, Matthieu & Sedki, Mohammed, 2017. "A family of block-wise one-factor distributions for modeling high-dimensional binary data," Computational Statistics & Data Analysis, Elsevier, vol. 114(C), pages 130-145.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jclass:v:33:y:2016:i:2:d:10.1007_s00357-016-9208-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.