IDEAS home Printed from https://ideas.repec.org/a/eee/apmaco/v409y2021ics0096300320308572.html
   My bibliography  Save this article

Novel DCA based algorithms for a special class of nonconvex problems with application in machine learning

Author

Listed:
  • Le Thi, Hoai An
  • Le, Hoai Minh
  • Phan, Duy Nhat
  • Tran, Bach

Abstract

We address the problem of minimizing the sum of a nonconvex, differentiable function and composite functions by DC (Difference of Convex functions) programming and DCA (DC Algorithm), powerful tools of nonconvex optimization. The main idea of DCA relies on DC decompositions of the objective function, it consists in approximating a DC (nonconvex) program by a sequence of convex ones. We first develop a standard DCA scheme especially dealing with the very specific structure of this problem. Furthermore, we extend DCA to give rise to the so-named DCA-Like, which is based on a new and efficient way to approximate the DC objective function without knowing a DC decomposition. We further improve DCA based algorithms by incorporating the Nesterov’s acceleration technique into them. The convergence properties and the convergence rate under Kurdyka-Łojasiewicz assumption of extended DCAs are rigorously studied. We prove that DCA-Like and the accelerated versions subsequently converge from every initial point to a critical point of the considered problem. Finally, we investigate the proposed algorithms for an important problem in machine learning: the t-distributed stochastic neighbor embedding. Numerical experiments on several benchmark datasets illustrate the efficiency of our algorithms.

Suggested Citation

  • Le Thi, Hoai An & Le, Hoai Minh & Phan, Duy Nhat & Tran, Bach, 2021. "Novel DCA based algorithms for a special class of nonconvex problems with application in machine learning," Applied Mathematics and Computation, Elsevier, vol. 409(C).
    • Handle: RePEc:eee:apmaco:v:409:y:2021:i:c:s0096300320308572
    DOI: 10.1016/j.amc.2020.125904
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0096300320308572
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.amc.2020.125904?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hédy Attouch & Jérôme Bolte & Patrick Redont & Antoine Soubeyran, 2010. "Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality," Mathematics of Operations Research, INFORMS, vol. 35(2), pages 438-457, May.
    2. Le Thi, H.A. & Pham Dinh, T. & Le, H.M. & Vo, X.T., 2015. "DC approximation approaches for sparse optimization," European Journal of Operational Research, Elsevier, vol. 244(1), pages 26-46.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Le Thi Khanh Hien & Duy Nhat Phan & Nicolas Gillis, 2022. "Inertial alternating direction method of multipliers for non-convex non-smooth optimization," Computational Optimization and Applications, Springer, vol. 83(1), pages 247-285, September.
    2. Francesco Rinaldi & Damiano Zeffiro, 2023. "Avoiding bad steps in Frank-Wolfe variants," Computational Optimization and Applications, Springer, vol. 84(1), pages 225-264, January.
    3. Kely D. V. Villacorta & Paulo R. Oliveira & Antoine Soubeyran, 2014. "A Trust-Region Method for Unconstrained Multiobjective Problems with Applications in Satisficing Processes," Journal of Optimization Theory and Applications, Springer, vol. 160(3), pages 865-889, March.
    4. Bo Jiang & Tianyi Lin & Shiqian Ma & Shuzhong Zhang, 2019. "Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis," Computational Optimization and Applications, Springer, vol. 72(1), pages 115-157, January.
    5. Zehui Jia & Jieru Huang & Xingju Cai, 2021. "Proximal-like incremental aggregated gradient method with Bregman distance in weakly convex optimization problems," Journal of Global Optimization, Springer, vol. 80(4), pages 841-864, August.
    6. Glaydston Carvalho Bento & João Xavier Cruz Neto & Antoine Soubeyran & Valdinês Leite Sousa Júnior, 2016. "Dual Descent Methods as Tension Reduction Systems," Journal of Optimization Theory and Applications, Springer, vol. 171(1), pages 209-227, October.
    7. Bolte, Jérôme & Le, Tam & Pauwels, Edouard & Silveti-Falls, Antonio, 2022. "Nonsmooth Implicit Differentiation for Machine Learning and Optimization," TSE Working Papers 22-1314, Toulouse School of Economics (TSE).
    8. Masoud Ahookhosh & Le Thi Khanh Hien & Nicolas Gillis & Panagiotis Patrinos, 2021. "A Block Inertial Bregman Proximal Algorithm for Nonsmooth Nonconvex Problems with Application to Symmetric Nonnegative Matrix Tri-Factorization," Journal of Optimization Theory and Applications, Springer, vol. 190(1), pages 234-258, July.
    9. Alexander Y. Kruger & Nguyen H. Thao, 2015. "Quantitative Characterizations of Regularity Properties of Collections of Sets," Journal of Optimization Theory and Applications, Springer, vol. 164(1), pages 41-67, January.
    10. Jing Zhao & Qiao-Li Dong & Michael Th. Rassias & Fenghui Wang, 2022. "Two-step inertial Bregman alternating minimization algorithm for nonconvex and nonsmooth problems," Journal of Global Optimization, Springer, vol. 84(4), pages 941-966, December.
    11. Fornasier, Massimo & Maly, Johannes & Naumova, Valeriya, 2021. "Robust recovery of low-rank matrices with non-orthogonal sparse decomposition from incomplete measurements," Applied Mathematics and Computation, Elsevier, vol. 392(C).
    12. Emanuel Laude & Peter Ochs & Daniel Cremers, 2020. "Bregman Proximal Mappings and Bregman–Moreau Envelopes Under Relative Prox-Regularity," Journal of Optimization Theory and Applications, Springer, vol. 184(3), pages 724-761, March.
    13. W. Ackooij & S. Demassey & P. Javal & H. Morais & W. Oliveira & B. Swaminathan, 2021. "A bundle method for nonsmooth DC programming with application to chance-constrained problems," Computational Optimization and Applications, Springer, vol. 78(2), pages 451-490, March.
    14. Nguyen Hieu Thao, 2018. "A convergent relaxation of the Douglas–Rachford algorithm," Computational Optimization and Applications, Springer, vol. 70(3), pages 841-863, July.
    15. Jérôme Bolte & Edouard Pauwels, 2016. "Majorization-Minimization Procedures and Convergence of SQP Methods for Semi-Algebraic and Tame Programs," Mathematics of Operations Research, INFORMS, vol. 41(2), pages 442-465, May.
    16. Hoai An Le Thi & Manh Cuong Nguyen, 2017. "DCA based algorithms for feature selection in multi-class support vector machine," Annals of Operations Research, Springer, vol. 249(1), pages 273-300, February.
    17. D. Russell Luke & Shoham Sabach & Marc Teboulle & Kobi Zatlawey, 2017. "A simple globally convergent algorithm for the nonsmooth nonconvex single source localization problem," Journal of Global Optimization, Springer, vol. 69(4), pages 889-909, December.
    18. Bian, Fengmiao & Zhang, Xiaoqun, 2021. "A parameterized Douglas–Rachford splitting algorithm for nonconvex optimization," Applied Mathematics and Computation, Elsevier, vol. 410(C).
    19. J. X. Cruz Neto & P. R. Oliveira & P. A. Soares & A. Soubeyran, 2014. "Proximal Point Method on Finslerian Manifolds and the “Effort–Accuracy” Trade-off," Journal of Optimization Theory and Applications, Springer, vol. 162(3), pages 873-891, September.
    20. Liu, Jingjing & Ma, Ruijie & Zeng, Xiaoyang & Liu, Wanquan & Wang, Mingyu & Chen, Hui, 2021. "An efficient non-convex total variation approach for image deblurring and denoising," Applied Mathematics and Computation, Elsevier, vol. 397(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:apmaco:v:409:y:2021:i:c:s0096300320308572. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/applied-mathematics-and-computation .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.