IDEAS home Printed from https://ideas.repec.org/a/spr/coopap/v73y2019i2d10.1007_s10589-019-00075-z.html
   My bibliography  Save this article

A semismooth Newton method for support vector classification and regression

Author

Listed:
  • Juan Yin

    (Beijing Institute of Technology)

  • Qingna Li

    (Beijing Institute of Technology)

Abstract

Support vector machine is an important and fundamental technique in machine learning. In this paper, we apply a semismooth Newton method to solve two typical SVM models: the L2-loss SVC model and the $$\epsilon $$ ϵ -L2-loss SVR model. The semismooth Newton method is widely used in optimization community. A common belief on the semismooth Newton method is its fast convergence rate as well as high computational complexity. Our contribution in this paper is that by exploring the sparse structure of the models, we significantly reduce the computational complexity, meanwhile keeping the quadratic convergence rate. Extensive numerical experiments demonstrate the outstanding performance of the semismooth Newton method, especially for problems with huge size of sample data (for news20.binary problem with 19,996 features and 1,355,191 samples, it only takes 3 s). In particular, for the $$\epsilon $$ ϵ -L2-loss SVR model, the semismooth Newton method significantly outperforms the leading solvers including DCD and TRON.

Suggested Citation

  • Juan Yin & Qingna Li, 2019. "A semismooth Newton method for support vector classification and regression," Computational Optimization and Applications, Springer, vol. 73(2), pages 477-508, June.
    • Handle: RePEc:spr:coopap:v:73:y:2019:i:2:d:10.1007_s10589-019-00075-z
    DOI: 10.1007/s10589-019-00075-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10589-019-00075-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10589-019-00075-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. J. Y. Bello Cruz & O. P. Ferreira & L. F. Prudente, 2016. "On the global convergence of the inexact semi-smooth Newton method for absolute value equation," Computational Optimization and Applications, Springer, vol. 65(1), pages 93-108, September.
    2. Francisco Facchinei & Christian Kanzow & Sebastian Karl & Simone Sagratella, 2015. "The semismooth Newton method for the solution of quasi-variational inequalities," Computational Optimization and Applications, Springer, vol. 62(1), pages 85-109, September.
    3. Weizhe Gu & Wei-Po Chen & Chun-Hsu Ko & Yuh-Jye Lee & Jein-Shan Chen, 2018. "Two smooth support vector machines for $$\varepsilon $$ ε -insensitive regression," Computational Optimization and Applications, Springer, vol. 70(1), pages 171-199, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Butyn, Emerson & Karas, Elizabeth W. & de Oliveira, Welington, 2022. "A derivative-free trust-region algorithm with copula-based models for probability maximization problems," European Journal of Operational Research, Elsevier, vol. 298(1), pages 59-75.
    2. Didier Aussel & Simone Sagratella, 2017. "Sufficient conditions to compute any solution of a quasivariational inequality via a variational inequality," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 85(1), pages 3-18, February.
    3. Yuan Liang & Chaoqian Li, 2023. "Modified Picard-like Method for Solving Absolute Value Equations," Mathematics, MDPI, vol. 11(4), pages 1-18, February.
    4. Axel Dreves, 2019. "An algorithm for equilibrium selection in generalized Nash equilibrium problems," Computational Optimization and Applications, Springer, vol. 73(3), pages 821-837, July.
    5. Axel Dreves & Simone Sagratella, 2020. "Nonsingularity and Stationarity Results for Quasi-Variational Inequalities," Journal of Optimization Theory and Applications, Springer, vol. 185(3), pages 711-743, June.
    6. Yekini Shehu & Aviv Gibali & Simone Sagratella, 2020. "Inertial Projection-Type Methods for Solving Quasi-Variational Inequalities in Real Hilbert Spaces," Journal of Optimization Theory and Applications, Springer, vol. 184(3), pages 877-894, March.
    7. Christian Kanzow & Daniel Steck, 2018. "Augmented Lagrangian and exact penalty methods for quasi-variational inequalities," Computational Optimization and Applications, Springer, vol. 69(3), pages 801-824, April.
    8. Leonardo Galli & Christian Kanzow & Marco Sciandrone, 2018. "A nonmonotone trust-region method for generalized Nash equilibrium and related problems with strong convergence properties," Computational Optimization and Applications, Springer, vol. 69(3), pages 629-652, April.
    9. Pin-Bo Chen & Gui-Hua Lin & Xide Zhu & Fusheng Bai, 2021. "Smoothing Newton method for nonsmooth second-order cone complementarity problems with application to electric power markets," Journal of Global Optimization, Springer, vol. 80(3), pages 635-659, July.
    10. Lorenzo Lampariello & Christoph Neumann & Jacopo M. Ricci & Simone Sagratella & Oliver Stein, 2020. "An explicit Tikhonov algorithm for nested variational inequalities," Computational Optimization and Applications, Springer, vol. 77(2), pages 335-350, November.
    11. Lampariello, Lorenzo & Neumann, Christoph & Ricci, Jacopo M. & Sagratella, Simone & Stein, Oliver, 2021. "Equilibrium selection for multi-portfolio optimization," European Journal of Operational Research, Elsevier, vol. 295(1), pages 363-373.
    12. An Wang & Yang Cao & Jing-Xian Chen, 2019. "Modified Newton-Type Iteration Methods for Generalized Absolute Value Equations," Journal of Optimization Theory and Applications, Springer, vol. 181(1), pages 216-230, April.
    13. Tommaso Colombo & Simone Sagratella, 2020. "Distributed algorithms for convex problems with linear coupling constraints," Journal of Global Optimization, Springer, vol. 77(1), pages 53-73, May.
    14. Fabiana R. Oliveira & Fabrícia R. Oliveira, 2021. "A Global Newton Method for the Nonsmooth Vector Fields on Riemannian Manifolds," Journal of Optimization Theory and Applications, Springer, vol. 190(1), pages 259-273, July.
    15. Milan Hladík, 2018. "Bounds for the solutions of absolute value equations," Computational Optimization and Applications, Springer, vol. 69(1), pages 243-266, January.
    16. Ke, Yi-Fen & Ma, Chang-Feng, 2017. "SOR-like iteration method for solving absolute value equations," Applied Mathematics and Computation, Elsevier, vol. 311(C), pages 195-202.
    17. Lorenzo Lampariello & Simone Sagratella, 2020. "Numerically tractable optimistic bilevel problems," Computational Optimization and Applications, Springer, vol. 76(2), pages 277-303, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:coopap:v:73:y:2019:i:2:d:10.1007_s10589-019-00075-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.