IDEAS home Printed from https://ideas.repec.org/a/eee/jmvana/v166y2018icp282-299.html
   My bibliography  Save this article

Adaptively weighted large-margin angle-based classifiers

Author

Listed:
  • Fu, Sheng
  • Zhang, Sanguo
  • Liu, Yufeng

Abstract

Large-margin classifiers are powerful techniques for classification problems. Although binary large-margin classifiers are heavily studied, multicategory problems are more complicated and challenging. A common approach is to construct k different decision functions for a k-class problem with a sum-to-zero constraint. However, such a constraint can be inefficient. Moreover, many large-margin classifiers can be sensitive to outliers in the training sample. In this article, we use the angle-based classification framework to avoid the explicit sum-to-zero constraint, and we propose two adaptively weighted large-margin classification techniques. Our new methods are Fisher consistent and more robust against outliers under suitable conditions. Numerical experiments further indicate that our methods give competitive and stable performance when compared with existing approaches.

Suggested Citation

  • Fu, Sheng & Zhang, Sanguo & Liu, Yufeng, 2018. "Adaptively weighted large-margin angle-based classifiers," Journal of Multivariate Analysis, Elsevier, vol. 166(C), pages 282-299.
    • Handle: RePEc:eee:jmvana:v:166:y:2018:i:c:p:282-299
    DOI: 10.1016/j.jmva.2018.03.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047259X17303767
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jmva.2018.03.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. NESTEROV, Yurii, 2012. "Efficiency of coordinate descent methods on huge-scale optimization problems," LIDAM Reprints CORE 2511, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    2. Liu, Yufeng & Zhang, Hao Helen & Wu, Yichao, 2011. "Hard or Soft Classification? Large-Margin Unified Machines," Journal of the American Statistical Association, American Statistical Association, vol. 106(493), pages 166-177.
    3. YichaoWu, & Liu, Yufeng, 2007. "Robust Truncated Hinge Loss Support Vector Machines," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 974-983, September.
    4. Chong Zhang & Yufeng Liu, 2014. "Multicategory angle-based large-margin classification," Biometrika, Biometrika Trust, vol. 101(3), pages 625-640.
    5. Liu, Yufeng & Shen, Xiaotong, 2006. "Multicategory -Learning," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 500-509, June.
    6. Lee, Yoonkyung & Lin, Yi & Wahba, Grace, 2004. "Multicategory Support Vector Machines: Theory and Application to the Classification of Microarray Data and Satellite Radiance Data," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 67-81, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yang, Yi & Guo, Yuxuan & Chang, Xiangyu, 2021. "Angle-based cost-sensitive multicategory classification," Computational Statistics & Data Analysis, Elsevier, vol. 156(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Yi & Guo, Yuxuan & Chang, Xiangyu, 2021. "Angle-based cost-sensitive multicategory classification," Computational Statistics & Data Analysis, Elsevier, vol. 156(C).
    2. Chong Zhang & Yufeng Liu, 2016. "Comments on: Probability enhanced effective dimension reduction for classifying sparse functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(1), pages 44-46, March.
    3. Chong Zhang & Yufeng Liu, 2016. "Comments on: Probability enhanced effective dimension reduction for classifying sparse functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(1), pages 44-46, March.
    4. Park, Beomjin & Park, Changyi, 2021. "Kernel variable selection for multicategory support vector machines," Journal of Multivariate Analysis, Elsevier, vol. 186(C).
    5. Lobna Abid & Afif Masmoudi & Sonia Zouari-Ghorbel, 2018. "The Consumer Loan’s Payment Default Predictive Model: an Application of the Logistic Regression and the Discriminant Analysis in a Tunisian Commercial Bank," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 9(3), pages 948-962, September.
    6. Pedro Duarte Silva, A., 2017. "Optimization approaches to Supervised Classification," European Journal of Operational Research, Elsevier, vol. 261(2), pages 772-788.
    7. Fang Yao & Yichao Wu & Jialin Zou, 2016. "Probability-enhanced effective dimension reduction for classifying sparse functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(1), pages 1-22, March.
    8. Park, Beomjin & Park, Changyi, 2023. "Multiclass Laplacian support vector machine with functional analysis of variance decomposition," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).
    9. Fang Yao & Yichao Wu & Jialin Zou, 2016. "Rejoinder on: Probability enhanced effective dimension reduction for classifying sparse functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(1), pages 52-58, March.
    10. Xingye Qiao & Yufeng Liu, 2009. "Adaptive Weighted Learning for Unbalanced Multicategory Classification," Biometrics, The International Biometric Society, vol. 65(1), pages 159-168, March.
    11. Fang Yao & Yichao Wu & Jialin Zou, 2016. "Rejoinder on: Probability enhanced effective dimension reduction for classifying sparse functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(1), pages 52-58, March.
    12. Sermpinis, Georgios & Stasinakis, Charalampos & Theofilatos, Konstantinos & Karathanasopoulos, Andreas, 2015. "Modeling, forecasting and trading the EUR exchange rates with hybrid rolling genetic algorithms—Support vector regression forecast combinations," European Journal of Operational Research, Elsevier, vol. 247(3), pages 831-846.
    13. Fan, Yiwei & Zhao, Junlong, 2022. "Safe sample screening rules for multicategory angle-based support vector machines," Computational Statistics & Data Analysis, Elsevier, vol. 173(C).
    14. Fang Yao & Yichao Wu & Jialin Zou, 2016. "Probability-enhanced effective dimension reduction for classifying sparse functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(1), pages 1-22, March.
    15. Zhu Wang, 2022. "MM for penalized estimation," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 31(1), pages 54-75, March.
    16. Yoonkyung Lee, 2014. "Comments on: Support vector machines maximizing geometric margins for multi-class classification," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 22(3), pages 852-855, October.
    17. Xuefei Lu & Alessandro Rudi & Emanuele Borgonovo & Lorenzo Rosasco, 2020. "Faster Kriging: Facing High-Dimensional Simulators," Operations Research, INFORMS, vol. 68(1), pages 233-249, January.
    18. Crystal T. Nguyen & Daniel J. Luckett & Anna R. Kahkoska & Grace E. Shearrer & Donna Spruijt‐Metz & Jaimie N. Davis & Michael R. Kosorok, 2020. "Estimating individualized treatment regimes from crossover designs," Biometrics, The International Biometric Society, vol. 76(3), pages 778-788, September.
    19. Hao Zhang, 2016. "Comments on: Probability enhanced effective dimension reduction for classifying sparse functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 25(1), pages 47-51, March.
    20. TAYLOR, Adrien B. & HENDRICKX, Julien M. & François GLINEUR, 2016. "Exact worst-case performance of first-order methods for composite convex optimization," LIDAM Discussion Papers CORE 2016052, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jmvana:v:166:y:2018:i:c:p:282-299. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.