IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v178y2018i3d10.1007_s10957-018-1341-2.html
   My bibliography  Save this article

A Line-Search Algorithm Inspired by the Adaptive Cubic Regularization Framework and Complexity Analysis

Author

Listed:
  • El Houcine Bergou

    (Université Paris-Saclay)

  • Youssef Diouane

    (Université de Toulouse)

  • Serge Gratton

    (Université de Toulouse)

Abstract

Adaptive regularized framework using cubics has emerged as an alternative to line-search and trust-region algorithms for smooth nonconvex optimization, with an optimal complexity among second-order methods. In this paper, we propose and analyze the use of an iteration dependent scaled norm in the adaptive regularized framework using cubics. Within such a scaled norm, the obtained method behaves as a line-search algorithm along the quasi-Newton direction with a special backtracking strategy. Under appropriate assumptions, the new algorithm enjoys the same convergence and complexity properties as adaptive regularized algorithm using cubics. The complexity for finding an approximate first-order stationary point can be improved to be optimal whenever a second-order version of the proposed algorithm is regarded. In a similar way, using the same scaled norm to define the trust-region neighborhood, we show that the trust-region algorithm behaves as a line-search algorithm. The good potential of the obtained algorithms is shown on a set of large-scale optimization problems.

Suggested Citation

  • El Houcine Bergou & Youssef Diouane & Serge Gratton, 2018. "A Line-Search Algorithm Inspired by the Adaptive Cubic Regularization Framework and Complexity Analysis," Journal of Optimization Theory and Applications, Springer, vol. 178(3), pages 885-913, September.
    • Handle: RePEc:spr:joptap:v:178:y:2018:i:3:d:10.1007_s10957-018-1341-2
    DOI: 10.1007/s10957-018-1341-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-018-1341-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10957-018-1341-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. E. Bergou & Y. Diouane & S. Gratton, 2017. "On the use of the energy norm in trust-region and adaptive cubic regularization subproblems," Computational Optimization and Applications, Springer, vol. 68(3), pages 533-554, December.
    2. Nicholas Gould & Dominique Orban & Philippe Toint, 2015. "CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization," Computational Optimization and Applications, Springer, vol. 60(3), pages 545-557, April.
    3. J. M. Martínez & M. Raydan, 2017. "Cubic-regularization counterpart of a variable-norm trust-region method for unconstrained minimization," Journal of Global Optimization, Springer, vol. 68(2), pages 367-385, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. J. M. Martínez & L. T. Santos, 2022. "On large-scale unconstrained optimization and arbitrary regularization," Computational Optimization and Applications, Springer, vol. 81(1), pages 1-30, January.
    2. Seonho Park & Seung Hyun Jung & Panos M. Pardalos, 2020. "Combining Stochastic Adaptive Cubic Regularization with Negative Curvature for Nonconvex Optimization," Journal of Optimization Theory and Applications, Springer, vol. 184(3), pages 953-971, March.
    3. C. P. Brás & J. M. Martínez & M. Raydan, 2020. "Large-scale unconstrained optimization using separable cubic modeling and matrix-free subspace minimization," Computational Optimization and Applications, Springer, vol. 75(1), pages 169-205, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yonggang Pei & Shaofang Song & Detong Zhu, 2023. "A sequential adaptive regularisation using cubics algorithm for solving nonlinear equality constrained optimization," Computational Optimization and Applications, Springer, vol. 84(3), pages 1005-1033, April.
    2. E. G. Birgin & J. M. Martínez, 2019. "A Newton-like method with mixed factorizations and cubic regularization for unconstrained minimization," Computational Optimization and Applications, Springer, vol. 73(3), pages 707-753, July.
    3. J. M. Martínez & L. T. Santos, 2022. "On large-scale unconstrained optimization and arbitrary regularization," Computational Optimization and Applications, Springer, vol. 81(1), pages 1-30, January.
    4. C. P. Brás & J. M. Martínez & M. Raydan, 2020. "Large-scale unconstrained optimization using separable cubic modeling and matrix-free subspace minimization," Computational Optimization and Applications, Springer, vol. 75(1), pages 169-205, January.
    5. Bo Jiang & Tianyi Lin & Shiqian Ma & Shuzhong Zhang, 2019. "Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis," Computational Optimization and Applications, Springer, vol. 72(1), pages 115-157, January.
    6. Yutao Zheng & Bing Zheng, 2017. "Two New Dai–Liao-Type Conjugate Gradient Methods for Unconstrained Optimization Problems," Journal of Optimization Theory and Applications, Springer, vol. 175(2), pages 502-509, November.
    7. Giovanni Fasano & Massimo Roma, 2016. "A novel class of approximate inverse preconditioners for large positive definite linear systems in optimization," Computational Optimization and Applications, Springer, vol. 65(2), pages 399-429, November.
    8. Mehiddin Al-Baali & Andrea Caliciotti & Giovanni Fasano & Massimo Roma, 2017. "Exploiting damped techniques for nonlinear conjugate gradient methods," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 86(3), pages 501-522, December.
    9. V. S. Amaral & R. Andreani & E. G. Birgin & D. S. Marcondes & J. M. Martínez, 2022. "On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimization," Journal of Global Optimization, Springer, vol. 84(3), pages 527-561, November.
    10. Nicholas I. M. Gould & Daniel P. Robinson, 2017. "A dual gradient-projection method for large-scale strictly convex quadratic problems," Computational Optimization and Applications, Springer, vol. 67(1), pages 1-38, May.
    11. Charles Audet & Kwassi Joseph Dzahini & Michael Kokkolaras & Sébastien Le Digabel, 2021. "Stochastic mesh adaptive direct search for blackbox optimization using probabilistic estimates," Computational Optimization and Applications, Springer, vol. 79(1), pages 1-34, May.
    12. Caliciotti, Andrea & Fasano, Giovanni & Roma, Massimo, 2018. "Preconditioned Nonlinear Conjugate Gradient methods based on a modified secant equation," Applied Mathematics and Computation, Elsevier, vol. 318(C), pages 196-214.
    13. Jianjun Liu & Xiangmin Xu & Xuehui Cui, 2018. "An accelerated nonmonotone trust region method with adaptive trust region for unconstrained optimization," Computational Optimization and Applications, Springer, vol. 69(1), pages 77-97, January.
    14. Rujun Jiang & Man-Chung Yue & Zhishuo Zhou, 2021. "An accelerated first-order method with complexity analysis for solving cubic regularization subproblems," Computational Optimization and Applications, Springer, vol. 79(2), pages 471-506, June.
    15. Andrea Cristofari & Marianna Santis & Stefano Lucidi & Francesco Rinaldi, 2017. "A Two-Stage Active-Set Algorithm for Bound-Constrained Optimization," Journal of Optimization Theory and Applications, Springer, vol. 172(2), pages 369-401, February.
    16. S. Gratton & C. W. Royer & L. N. Vicente & Z. Zhang, 2019. "Direct search based on probabilistic feasible descent for bound and linearly constrained problems," Computational Optimization and Applications, Springer, vol. 72(3), pages 525-559, April.
    17. Sven Leyffer & Charlie Vanaret, 2020. "An augmented Lagrangian filter method," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 92(2), pages 343-376, October.
    18. David Ek & Anders Forsgren, 2021. "Approximate solution of system of equations arising in interior-point methods for bound-constrained optimization," Computational Optimization and Applications, Springer, vol. 79(1), pages 155-191, May.
    19. David J. Eckman & Shane G. Henderson & Sara Shashaani, 2023. "Diagnostic Tools for Evaluating and Comparing Simulation-Optimization Algorithms," INFORMS Journal on Computing, INFORMS, vol. 35(2), pages 350-367, March.
    20. Brian Irwin & Eldad Haber, 2023. "Secant penalized BFGS: a noise robust quasi-Newton method via penalizing the secant condition," Computational Optimization and Applications, Springer, vol. 84(3), pages 651-702, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:178:y:2018:i:3:d:10.1007_s10957-018-1341-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.