IDEAS home Printed from https://ideas.repec.org/a/spr/coopap/v88y2024i2d10.1007_s10589-024-00560-0.html
   My bibliography  Save this article

An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization

Author

Listed:
  • Ruyu Liu

    (South China University of Technology)

  • Shaohua Pan

    (South China University of Technology)

  • Yuqia Wu

    (The Hong Kong Polytechnic University)

  • Xiaoqi Yang

    (The Hong Kong Polytechnic University)

Abstract

This paper focuses on the minimization of a sum of a twice continuously differentiable function f and a nonsmooth convex function. An inexact regularized proximal Newton method is proposed by an approximation to the Hessian of f involving the $$\varrho $$ ϱ th power of the KKT residual. For $$\varrho =0$$ ϱ = 0 , we justify the global convergence of the iterate sequence for the KL objective function and its R-linear convergence rate for the KL objective function of exponent 1/2. For $$\varrho \in (0,1)$$ ϱ ∈ ( 0 , 1 ) , by assuming that cluster points satisfy a locally Hölderian error bound of order q on a second-order stationary point set and a local error bound of order $$q>1\!+\!\varrho $$ q > 1 + ϱ on the common stationary point set, respectively, we establish the global convergence of the iterate sequence and its superlinear convergence rate with order depending on q and $$\varrho $$ ϱ . A dual semismooth Newton augmented Lagrangian method is also developed for seeking an inexact minimizer of subproblems. Numerical comparisons with two state-of-the-art methods on $$\ell _1$$ ℓ 1 -regularized Student’s t-regressions, group penalized Student’s t-regressions, and nonconvex image restoration confirm the efficiency of the proposed method.

Suggested Citation

  • Ruyu Liu & Shaohua Pan & Yuqia Wu & Xiaoqi Yang, 2024. "An inexact regularized proximal Newton method for nonconvex and nonsmooth optimization," Computational Optimization and Applications, Springer, vol. 88(2), pages 603-641, June.
  • Handle: RePEc:spr:coopap:v:88:y:2024:i:2:d:10.1007_s10589-024-00560-0
    DOI: 10.1007/s10589-024-00560-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10589-024-00560-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10589-024-00560-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lorenzo Stella & Andreas Themelis & Panagiotis Patrinos, 2017. "Forward–backward quasi-Newton methods for nonsmooth optimization problems," Computational Optimization and Applications, Springer, vol. 67(3), pages 443-487, July.
    2. Christian Kanzow & Theresa Lechner, 2021. "Correction to: Globalized inexact proximal Newton-type methods for nonconvex composite functions," Computational Optimization and Applications, Springer, vol. 80(2), pages 679-680, November.
    3. Yuqia Wu & Shaohua Pan & Shujun Bi, 2021. "Kurdyka–Łojasiewicz Property of Zero-Norm Composite Functions," Journal of Optimization Theory and Applications, Springer, vol. 188(1), pages 94-112, January.
    4. Boris Mordukhovich & Wei Ouyang, 2015. "Higher-order metric subregularity and its applications," Journal of Global Optimization, Springer, vol. 63(4), pages 777-795, December.
    5. Jinchao Li & Martin S. Andersen & Lieven Vandenberghe, 2017. "Inexact proximal Newton methods for self-concordant functions," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 85(1), pages 19-41, February.
    6. Christian Kanzow & Theresa Lechner, 2021. "Globalized inexact proximal Newton-type methods for nonconvex composite functions," Computational Optimization and Applications, Springer, vol. 78(2), pages 377-410, March.
    7. Tianxiang Liu & Akiko Takeda, 2022. "An inexact successive quadratic approximation method for a class of difference-of-convex optimization problems," Computational Optimization and Applications, Springer, vol. 82(1), pages 141-173, May.
    8. Tao Pham Dinh & Yi-Shuai Niu, 2011. "An efficient DC programming approach for portfolio decision with higher moments," Computational Optimization and Applications, Springer, vol. 50(3), pages 525-554, December.
    9. Ming Yuan & Yi Lin, 2006. "Model selection and estimation in regression with grouped variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(1), pages 49-67, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shummin Nakayama & Yasushi Narushima & Hiroshi Yabe, 2021. "Inexact proximal memoryless quasi-Newton methods based on the Broyden family for minimizing composite functions," Computational Optimization and Applications, Springer, vol. 79(1), pages 127-154, May.
    2. Helmut Gfrerer, 2025. "On a globally convergent semismooth* Newton method in nonsmooth nonconvex optimization," Computational Optimization and Applications, Springer, vol. 91(1), pages 67-124, May.
    3. Ryosuke Shimmura & Joe Suzuki, 2024. "Newton-Type Methods with the Proximal Gradient Step for Sparse Estimation," SN Operations Research Forum, Springer, vol. 5(2), pages 1-27, June.
    4. Christian Kanzow & Theresa Lechner, 2021. "Globalized inexact proximal Newton-type methods for nonconvex composite functions," Computational Optimization and Applications, Springer, vol. 78(2), pages 377-410, March.
    5. Simeon vom Dahl & Christian Kanzow, 2024. "An inexact regularized proximal Newton method without line search," Computational Optimization and Applications, Springer, vol. 89(3), pages 585-624, December.
    6. Christian Kanzow & Theresa Lechner, 2022. "COAP 2021 Best Paper Prize," Computational Optimization and Applications, Springer, vol. 83(3), pages 723-726, December.
    7. Tianxiang Liu & Akiko Takeda, 2022. "An inexact successive quadratic approximation method for a class of difference-of-convex optimization problems," Computational Optimization and Applications, Springer, vol. 82(1), pages 141-173, May.
    8. Tutz, Gerhard & Pößnecker, Wolfgang & Uhlmann, Lorenz, 2015. "Variable selection in general multinomial logit models," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 207-222.
    9. Guillaume Sagnol & Edouard Pauwels, 2019. "An unexpected connection between Bayes A-optimal designs and the group lasso," Statistical Papers, Springer, vol. 60(2), pages 565-584, April.
    10. Bakalli, Gaetan & Guerrier, Stéphane & Scaillet, Olivier, 2023. "A penalized two-pass regression to predict stock returns with time-varying risk premia," Journal of Econometrics, Elsevier, vol. 237(2).
    11. Shuichi Kawano, 2014. "Selection of tuning parameters in bridge regression models via Bayesian information criterion," Statistical Papers, Springer, vol. 55(4), pages 1207-1223, November.
    12. Wongsa-art, Pipat & Kim, Namhyun & Xia, Yingcun & Moscone, Francesco, 2024. "Varying coefficient panel data models and methods under correlated error components: Application to disparities in mental health services in England," Regional Science and Urban Economics, Elsevier, vol. 106(C).
    13. Dong, C. & Li, S., 2021. "Specification Lasso and an Application in Financial Markets," Cambridge Working Papers in Economics 2139, Faculty of Economics, University of Cambridge.
    14. Lam, Clifford, 2008. "Estimation of large precision matrices through block penalization," LSE Research Online Documents on Economics 31543, London School of Economics and Political Science, LSE Library.
    15. Weiyang Ding & Michael K. Ng & Wenxing Zhang, 2024. "A generalized alternating direction implicit method for consensus optimization: application to distributed sparse logistic regression," Journal of Global Optimization, Springer, vol. 90(3), pages 727-753, November.
    16. Gregory Vaughan & Robert Aseltine & Kun Chen & Jun Yan, 2017. "Stagewise generalized estimating equations with grouped variables," Biometrics, The International Biometric Society, vol. 73(4), pages 1332-1342, December.
    17. Pradeep Ravikumar & John Lafferty & Han Liu & Larry Wasserman, 2009. "Sparse additive models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(5), pages 1009-1030, November.
    18. Yucheng Yang & Zhong Zheng & Weinan E, 2020. "Interpretable Neural Networks for Panel Data Analysis in Economics," Papers 2010.05311, arXiv.org, revised Nov 2020.
    19. Madeleine Cule & Richard Samworth & Michael Stewart, 2010. "Maximum likelihood estimation of a multi‐dimensional log‐concave density," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(5), pages 545-607, November.
    20. Toshio Honda, 2021. "The de-biased group Lasso estimation for varying coefficient models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(1), pages 3-29, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:coopap:v:88:y:2024:i:2:d:10.1007_s10589-024-00560-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.