IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i17p3032-d895159.html
   My bibliography  Save this article

A Hybrid Stochastic Deterministic Algorithm for Solving Unconstrained Optimization Problems

Author

Listed:
  • Ahmad M. Alshamrani

    (Statistics and Operations Research Department, College of Science, King Saud University, P.O. Box 2455, Riyadh 11451, Saudi Arabia)

  • Adel Fahad Alrasheedi

    (Statistics and Operations Research Department, College of Science, King Saud University, P.O. Box 2455, Riyadh 11451, Saudi Arabia)

  • Khalid Abdulaziz Alnowibet

    (Statistics and Operations Research Department, College of Science, King Saud University, P.O. Box 2455, Riyadh 11451, Saudi Arabia)

  • Salem Mahdi

    (Department of Mathematics & Computer Science, Faculty of Science, Alexandria University, Alexandria 21545, Egypt
    Educational Research and Development Center Sanaa, Sanaa 00967, Yemen)

  • Ali Wagdy Mohamed

    (Operations Research Department, Faculty of Graduate Studies for Statistical Research, Cairo University, Giza 12613, Egypt)

Abstract

In this paper, a new deterministic method is proposed. This method depends on presenting (suggesting) some modifications to existing parameters of some conjugate gradient methods. The parameters of our suggested method contain a mix of deterministic and stochastic parameters. The proposed method is added to a line search algorithm to make it a globally convergent method. The convergence analysis of the method is established. The gradient vector is estimated by a finite difference approximation approach, and a new step-size h of this approach is generated randomly. In addition, a set of stochastic parameter formulas is constructed from which some solutions are generated randomly for an unconstrained problem. This stochastic technique is hybridized with the new deterministic method to obtain a new hybrid algorithm that finds an approximate solution for the global minimization problem. The performance of the suggested hybrid algorithm is tested in two sets of benchmark optimization test problems containing convex and non-convex functions. Comprehensive comparisons versus four other hybrid algorithms are listed in this study. The performance profiles are utilized to evaluate and compare the performance of the five hybrid algorithms. The numerical results show that our proposed hybrid algorithm is promising and competitive for finding the global optimum point. The comparison results between the performance of our suggested hybrid algorithm and the other four hybrid algorithms indicate that the proposed algorithm is competitive with, and in all cases superior to, the four algorithms in terms of the efficiency, reliability, and effectiveness for finding the global minimizers of non-convex functions.

Suggested Citation

  • Ahmad M. Alshamrani & Adel Fahad Alrasheedi & Khalid Abdulaziz Alnowibet & Salem Mahdi & Ali Wagdy Mohamed, 2022. "A Hybrid Stochastic Deterministic Algorithm for Solving Unconstrained Optimization Problems," Mathematics, MDPI, vol. 10(17), pages 1-26, August.
    • Handle: RePEc:gam:jmathe:v:10:y:2022:i:17:p:3032-:d:895159
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/17/3032/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/10/17/3032/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Gonglin Yuan & Zehong Meng & Yong Li, 2016. "A Modified Hestenes and Stiefel Conjugate Gradient Algorithm for Large-Scale Nonsmooth Minimizations and Nonlinear Equations," Journal of Optimization Theory and Applications, Springer, vol. 168(1), pages 129-152, January.
    2. Yingjie Zhou & Yulun Wu & Xiangrong Li, 2020. "A New Hybrid PRPFR Conjugate Gradient Method for Solving Nonlinear Monotone Equations and Image Restoration Problems," Mathematical Problems in Engineering, Hindawi, vol. 2020, pages 1-13, September.
    3. Y.H. Dai & Y. Yuan, 2001. "An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization," Annals of Operations Research, Springer, vol. 103(1), pages 33-47, March.
    4. Zhenhua Su & Min Li, 2020. "A Derivative-Free Liu–Storey Method for Solving Large-Scale Nonlinear Systems of Equations," Mathematical Problems in Engineering, Hindawi, vol. 2020, pages 1-10, October.
    5. Khalid Abdulaziz Alnowibet & Salem Mahdi & Mahmoud El-Alem & Mohamed Abdelawwad & Ali Wagdy Mohamed, 2022. "Guided Hybrid Modified Simulated Annealing Algorithm for Solving Constrained Global Optimization Problems," Mathematics, MDPI, vol. 10(8), pages 1-25, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Eltiyeb Ali & Salem Mahdi, 2023. "Adaptive Hybrid Mixed Two-Point Step Size Gradient Algorithm for Solving Non-Linear Systems," Mathematics, MDPI, vol. 11(9), pages 1-35, April.
    2. Khalid Abdulaziz Alnowibet & Salem Mahdi & Ahmad M. Alshamrani & Karam M. Sallam & Ali Wagdy Mohamed, 2022. "A Family of Hybrid Stochastic Conjugate Gradient Algorithms for Local and Global Minimization Problems," Mathematics, MDPI, vol. 10(19), pages 1-37, October.
    3. Manuel Jaramillo & Diego Carrión, 2022. "An Adaptive Strategy for Medium-Term Electricity Consumption Forecasting for Highly Unpredictable Scenarios: Case Study Quito, Ecuador during the Two First Years of COVID-19," Energies, MDPI, vol. 15(22), pages 1-19, November.
    4. Ghareeb Moustafa & Ali M. El-Rifaie & Idris H. Smaili & Ahmed Ginidi & Abdullah M. Shaheen & Ahmed F. Youssef & Mohamed A. Tolba, 2023. "An Enhanced Dwarf Mongoose Optimization Algorithm for Solving Engineering Problems," Mathematics, MDPI, vol. 11(15), pages 1-26, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Khalid Abdulaziz Alnowibet & Salem Mahdi & Ahmad M. Alshamrani & Karam M. Sallam & Ali Wagdy Mohamed, 2022. "A Family of Hybrid Stochastic Conjugate Gradient Algorithms for Local and Global Minimization Problems," Mathematics, MDPI, vol. 10(19), pages 1-37, October.
    2. Qi Tian & Xiaoliang Wang & Liping Pang & Mingkun Zhang & Fanyun Meng, 2021. "A New Hybrid Three-Term Conjugate Gradient Algorithm for Large-Scale Unconstrained Problems," Mathematics, MDPI, vol. 9(12), pages 1-13, June.
    3. Elena Tovbis & Vladimir Krutikov & Predrag Stanimirović & Vladimir Meshechkin & Aleksey Popov & Lev Kazakovtsev, 2023. "A Family of Multi-Step Subgradient Minimization Methods," Mathematics, MDPI, vol. 11(10), pages 1-24, May.
    4. Hiroyuki Sakai & Hideaki Iiduka, 2020. "Hybrid Riemannian conjugate gradient methods with global convergence properties," Computational Optimization and Applications, Springer, vol. 77(3), pages 811-830, December.
    5. Serge Gratton & Vincent Malmedy & Philippe Toint, 2012. "Using approximate secant equations in limited memory methods for multilevel unconstrained optimization," Computational Optimization and Applications, Springer, vol. 51(3), pages 967-979, April.
    6. Kin Keung Lai & Shashi Kant Mishra & Bhagwat Ram & Ravina Sharma, 2023. "A Conjugate Gradient Method: Quantum Spectral Polak–Ribiére–Polyak Approach for Unconstrained Optimization Problems," Mathematics, MDPI, vol. 11(23), pages 1-14, December.
    7. Priester, C. Robert & Melbourne-Thomas, Jessica & Klocker, Andreas & Corney, Stuart, 2017. "Abrupt transitions in dynamics of a NPZD model across Southern Ocean fronts," Ecological Modelling, Elsevier, vol. 359(C), pages 372-382.
    8. Xiaoliang Wang & Liping Pang & Qi Wu & Mingkun Zhang, 2021. "An Adaptive Proximal Bundle Method with Inexact Oracles for a Class of Nonconvex and Nonsmooth Composite Optimization," Mathematics, MDPI, vol. 9(8), pages 1-27, April.
    9. Gonglin Yuan & Zhou Sheng & Wenjie Liu, 2016. "The Modified HZ Conjugate Gradient Algorithm for Large-Scale Nonsmooth Optimization," PLOS ONE, Public Library of Science, vol. 11(10), pages 1-15, October.
    10. B. Sellami & Y. Chaib, 2016. "A new family of globally convergent conjugate gradient methods," Annals of Operations Research, Springer, vol. 241(1), pages 497-513, June.
    11. Xiaoyu Wu & Hu Shao & Pengjie Liu & Yue Zhuo, 2023. "An Inertial Spectral CG Projection Method Based on the Memoryless BFGS Update," Journal of Optimization Theory and Applications, Springer, vol. 198(3), pages 1130-1155, September.
    12. Yong Li & Gonglin Yuan & Zhou Sheng, 2018. "An active-set algorithm for solving large-scale nonsmooth optimization models with box constraints," PLOS ONE, Public Library of Science, vol. 13(1), pages 1-16, January.
    13. Savin Treanţă, 2022. "Variational Problems and Applications," Mathematics, MDPI, vol. 11(1), pages 1-4, December.
    14. XiaoLiang Dong & Deren Han & Zhifeng Dai & Lixiang Li & Jianguang Zhu, 2018. "An Accelerated Three-Term Conjugate Gradient Method with Sufficient Descent Condition and Conjugacy Condition," Journal of Optimization Theory and Applications, Springer, vol. 179(3), pages 944-961, December.
    15. Neculai Andrei, 2013. "Another Conjugate Gradient Algorithm with Guaranteed Descent and Conjugacy Conditions for Large-scale Unconstrained Optimization," Journal of Optimization Theory and Applications, Springer, vol. 159(1), pages 159-182, October.
    16. N. Andrei, 2009. "Hybrid Conjugate Gradient Algorithm for Unconstrained Optimization," Journal of Optimization Theory and Applications, Springer, vol. 141(2), pages 249-264, May.
    17. Hiroyuki Sakai & Hideaki Iiduka, 2021. "Sufficient Descent Riemannian Conjugate Gradient Methods," Journal of Optimization Theory and Applications, Springer, vol. 190(1), pages 130-150, July.
    18. Morteza Kimiaei & Farzad Rahpeymaii, 2019. "A new nonmonotone line-search trust-region approach for nonlinear systems," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(2), pages 199-232, July.
    19. Eltiyeb Ali & Salem Mahdi, 2023. "Adaptive Hybrid Mixed Two-Point Step Size Gradient Algorithm for Solving Non-Linear Systems," Mathematics, MDPI, vol. 11(9), pages 1-35, April.
    20. Jose Giovany Babativa-Márquez & José Luis Vicente-Villardón, 2021. "Logistic Biplot by Conjugate Gradient Algorithms and Iterated SVD," Mathematics, MDPI, vol. 9(16), pages 1-19, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:17:p:3032-:d:895159. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.