IDEAS home Printed from https://ideas.repec.org/a/eee/apmaco/v392y2021ics009630032030672x.html
   My bibliography  Save this article

New convergence results for the inexact variable metric forward–backward method

Author

Listed:
  • Bonettini, S.
  • Prato, M.
  • Rebegoldi, S.

Abstract

Forward–backward methods are valid tools to solve a variety of optimization problems where the objective function is the sum of a smooth, possibly nonconvex term plus a convex, possibly nonsmooth function. The corresponding iteration is built on two main ingredients: the computation of the gradient of the smooth part and the evaluation of the proximity (or resolvent) operator associated with the convex term. One of the main difficulties, from both implementation and theoretical point of view, arises when the proximity operator is computed in an inexact way. The aim of this paper is to provide new convergence results about forward–backward methods with inexact computation of the proximity operator, under the assumption that the objective function satisfies the Kurdyka–Łojasiewicz property. In particular, we adopt an inexactness criterion which can be implemented in practice, while preserving the main theoretical properties of the proximity operator. The main result is the proof of the convergence of the iterates generated by the forward–backward algorithm in Bonettini et al. (2017) to a stationary point. Convergence rate estimates are also provided. At the best of our knowledge, there exists no other inexact forward–backward algorithm with proved convergence in the nonconvex case and equipped with an explicit procedure to inexactly compute the proximity operator.

Suggested Citation

  • Bonettini, S. & Prato, M. & Rebegoldi, S., 2021. "New convergence results for the inexact variable metric forward–backward method," Applied Mathematics and Computation, Elsevier, vol. 392(C).
    • Handle: RePEc:eee:apmaco:v:392:y:2021:i:c:s009630032030672x
    DOI: 10.1016/j.amc.2020.125719
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S009630032030672X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.amc.2020.125719?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Emilie Chouzenoux & Jean-Christophe Pesquet & Audrey Repetti, 2014. "Variable Metric Forward–Backward Algorithm for Minimizing the Sum of a Differentiable Function and a Convex Function," Journal of Optimization Theory and Applications, Springer, vol. 162(1), pages 107-132, July.
    2. Unknown, 2005. "Forward," 2005 Conference: Slovenia in the EU - Challenges for Agriculture, Food Science and Rural Affairs, November 10-11, 2005, Moravske Toplice, Slovenia 183804, Slovenian Association of Agricultural Economists (DAES).
    3. S. Bonettini & M. Prato & S. Rebegoldi, 2018. "A block coordinate variable metric linesearch based proximal gradient method," Computational Optimization and Applications, Springer, vol. 71(1), pages 5-52, September.
    4. Bubba, Tatiana A. & Porta, Federica & Zanghirati, Gaetano & Bonettini, Silvia, 2018. "A nonsmooth regularization approach based on shearlets for Poisson noise removal in ROI tomography," Applied Mathematics and Computation, Elsevier, vol. 318(C), pages 131-152.
    5. P. Tseng & S. Yun, 2009. "Block-Coordinate Gradient Descent Method for Linearly Constrained Nonsmooth Separable Optimization," Journal of Optimization Theory and Applications, Springer, vol. 140(3), pages 513-535, March.
    6. Emilie Chouzenoux & Jean-Christophe Pesquet & Audrey Repetti, 2016. "A block coordinate variable metric forward–backward algorithm," Journal of Global Optimization, Springer, vol. 66(3), pages 457-485, November.
    7. Heinz H. Bauschke & Jérôme Bolte & Marc Teboulle, 2017. "A Descent Lemma Beyond Lipschitz Gradient Continuity: First-Order Methods Revisited and Applications," Mathematics of Operations Research, INFORMS, vol. 42(2), pages 330-348, May.
    8. Porta, Federica & Loris, Ignace, 2015. "On some steplength approaches for proximal algorithms," Applied Mathematics and Computation, Elsevier, vol. 253(C), pages 345-362.
    9. Peter Ochs & Jalal Fadili & Thomas Brox, 2019. "Non-smooth Non-convex Bregman Minimization: Unification and New Algorithms," Journal of Optimization Theory and Applications, Springer, vol. 181(1), pages 244-278, April.
    10. Pierre Frankel & Guillaume Garrigos & Juan Peypouquet, 2015. "Splitting Methods with Variable Metric for Kurdyka–Łojasiewicz Functions and General Convergence Rates," Journal of Optimization Theory and Applications, Springer, vol. 165(3), pages 874-900, June.
    11. Ching-pei Lee & Stephen J. Wright, 2019. "Inexact Successive quadratic approximation for regularized optimization," Computational Optimization and Applications, Springer, vol. 72(3), pages 641-674, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Silvia Bonettini & Peter Ochs & Marco Prato & Simone Rebegoldi, 2023. "An abstract convergence framework with application to inertial inexact forward–backward methods," Computational Optimization and Applications, Springer, vol. 84(2), pages 319-362, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. S. Bonettini & M. Prato & S. Rebegoldi, 2018. "A block coordinate variable metric linesearch based proximal gradient method," Computational Optimization and Applications, Springer, vol. 71(1), pages 5-52, September.
    2. Silvia Bonettini & Peter Ochs & Marco Prato & Simone Rebegoldi, 2023. "An abstract convergence framework with application to inertial inexact forward–backward methods," Computational Optimization and Applications, Springer, vol. 84(2), pages 319-362, March.
    3. Emilie Chouzenoux & Jean-Christophe Pesquet & Audrey Repetti, 2016. "A block coordinate variable metric forward–backward algorithm," Journal of Global Optimization, Springer, vol. 66(3), pages 457-485, November.
    4. Masoud Ahookhosh & Le Thi Khanh Hien & Nicolas Gillis & Panagiotis Patrinos, 2021. "A Block Inertial Bregman Proximal Algorithm for Nonsmooth Nonconvex Problems with Application to Symmetric Nonnegative Matrix Tri-Factorization," Journal of Optimization Theory and Applications, Springer, vol. 190(1), pages 234-258, July.
    5. Christian Kanzow & Theresa Lechner, 2021. "Globalized inexact proximal Newton-type methods for nonconvex composite functions," Computational Optimization and Applications, Springer, vol. 78(2), pages 377-410, March.
    6. Wei Peng & Hui Zhang & Xiaoya Zhang & Lizhi Cheng, 2020. "Global complexity analysis of inexact successive quadratic approximation methods for regularized optimization under mild assumptions," Journal of Global Optimization, Springer, vol. 78(1), pages 69-89, September.
    7. Daoli Zhu & Sien Deng & Minghua Li & Lei Zhao, 2021. "Level-Set Subdifferential Error Bounds and Linear Convergence of Bregman Proximal Gradient Method," Journal of Optimization Theory and Applications, Springer, vol. 189(3), pages 889-918, June.
    8. Ching-pei Lee & Stephen J. Wright, 2020. "Inexact Variable Metric Stochastic Block-Coordinate Descent for Regularized Optimization," Journal of Optimization Theory and Applications, Springer, vol. 185(1), pages 151-187, April.
    9. Tianxiang Liu & Akiko Takeda, 2022. "An inexact successive quadratic approximation method for a class of difference-of-convex optimization problems," Computational Optimization and Applications, Springer, vol. 82(1), pages 141-173, May.
    10. Le Thi Khanh Hien & Duy Nhat Phan & Nicolas Gillis, 2022. "Inertial alternating direction method of multipliers for non-convex non-smooth optimization," Computational Optimization and Applications, Springer, vol. 83(1), pages 247-285, September.
    11. Radu Ioan Bot & Dang-Khoa Nguyen, 2020. "The Proximal Alternating Direction Method of Multipliers in the Nonconvex Setting: Convergence Analysis and Rates," Mathematics of Operations Research, INFORMS, vol. 45(2), pages 682-712, May.
    12. Peter Ochs, 2018. "Local Convergence of the Heavy-Ball Method and iPiano for Non-convex Optimization," Journal of Optimization Theory and Applications, Springer, vol. 177(1), pages 153-180, April.
    13. Ching-pei Lee & Stephen J. Wright, 2019. "Inexact Successive quadratic approximation for regularized optimization," Computational Optimization and Applications, Springer, vol. 72(3), pages 641-674, April.
    14. Yaohua Hu & Chong Li & Kaiwen Meng & Xiaoqi Yang, 2021. "Linear convergence of inexact descent method and inexact proximal gradient algorithms for lower-order regularization problems," Journal of Global Optimization, Springer, vol. 79(4), pages 853-883, April.
    15. Masoud Ahookhosh & Le Thi Khanh Hien & Nicolas Gillis & Panagiotis Patrinos, 2021. "Multi-block Bregman proximal alternating linearized minimization and its application to orthogonal nonnegative matrix factorization," Computational Optimization and Applications, Springer, vol. 79(3), pages 681-715, July.
    16. Szilárd Csaba László, 2023. "A Forward–Backward Algorithm With Different Inertial Terms for Structured Non-Convex Minimization Problems," Journal of Optimization Theory and Applications, Springer, vol. 198(1), pages 387-427, July.
    17. Radu Ioan Boţ & Ernö Robert Csetnek & Szilárd Csaba László, 2016. "An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions," EURO Journal on Computational Optimization, Springer;EURO - The Association of European Operational Research Societies, vol. 4(1), pages 3-25, February.
    18. Bubba, Tatiana A. & Porta, Federica & Zanghirati, Gaetano & Bonettini, Silvia, 2018. "A nonsmooth regularization approach based on shearlets for Poisson noise removal in ROI tomography," Applied Mathematics and Computation, Elsevier, vol. 318(C), pages 131-152.
    19. Daniel Reem & Simeon Reich & Alvaro Pierro, 2019. "A Telescopic Bregmanian Proximal Gradient Method Without the Global Lipschitz Continuity Assumption," Journal of Optimization Theory and Applications, Springer, vol. 182(3), pages 851-884, September.
    20. Pilar Lopez-Llompart & G. Mathias Kondolf, 2016. "Encroachments in floodways of the Mississippi River and Tributaries Project," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 81(1), pages 513-542, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:apmaco:v:392:y:2021:i:c:s009630032030672x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/applied-mathematics-and-computation .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.