IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v13y2025i4p627-d1591536.html
   My bibliography  Save this article

Stochastic Zeroth-Order Multi-Gradient Algorithm for Multi-Objective Optimization

Author

Listed:
  • Zhihao Li

    (School of Information Engineering, Henan University of Science and Technology, Luoyang 471023, China)

  • Qingtao Wu

    (School of Information Engineering, Henan University of Science and Technology, Luoyang 471023, China
    Intelligent System Science and Technology Innovation Center, Longmen Laboratory, Luoyang 471023, China)

  • Moli Zhang

    (School of Information Engineering, Henan University of Science and Technology, Luoyang 471023, China)

  • Lin Wang

    (School of Information Engineering, Henan University of Science and Technology, Luoyang 471023, China
    Intelligent System Science and Technology Innovation Center, Longmen Laboratory, Luoyang 471023, China)

  • Youming Ge

    (School of Information Engineering, Henan University of Science and Technology, Luoyang 471023, China)

  • Guoyong Wang

    (School of Computer and Information Engineering, Luoyang Institute of Science and Technology, Luoyang 471023, China)

Abstract

Multi-objective optimization (MOO) has become an important method in machine learning, which involves solving multiple competing objective problems simultaneously. Nowadays, many MOO algorithms assume that gradient information is easily available and use this information to optimize functions. However, when encountering situations where gradients are not available, such as black-box functions or non-differentiable functions, these algorithms become ineffective. In this paper, we propose a zeroth-order MOO algorithm named SZMG (stochastic zeroth-order multi-gradient algorithm), which approximates the gradient of functions by finite difference methods. Meanwhile, to avoid conflicting gradients between functions and reduce stochastic multi-gradient direction bias caused by stochastic gradients, an SGD-type method is adopted to acquire weight parameters. Under the non-convex setting and mild assumptions, the convergence rate is established for the SZMG algorithm. Simulation results demonstrate the effectiveness of the SZMG algorithm.

Suggested Citation

  • Zhihao Li & Qingtao Wu & Moli Zhang & Lin Wang & Youming Ge & Guoyong Wang, 2025. "Stochastic Zeroth-Order Multi-Gradient Algorithm for Multi-Objective Optimization," Mathematics, MDPI, vol. 13(4), pages 1-31, February.
    • Handle: RePEc:gam:jmathe:v:13:y:2025:i:4:p:627-:d:1591536
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/13/4/627/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/13/4/627/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mercier, Quentin & Poirion, Fabrice & Désidéri, Jean-Antoine, 2018. "A stochastic multiple gradient descent algorithm," European Journal of Operational Research, Elsevier, vol. 271(3), pages 808-817.
    2. S. Liu & L. N. Vicente, 2024. "The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning," Annals of Operations Research, Springer, vol. 339(3), pages 1119-1148, August.
    3. Jörg Fliege & Benar Fux Svaiter, 2000. "Steepest descent methods for multicriteria optimization," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 51(3), pages 479-494, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Suyun Liu & Luis Nunes Vicente, 2022. "Accuracy and fairness trade-offs in machine learning: a stochastic multi-objective approach," Computational Management Science, Springer, vol. 19(3), pages 513-537, July.
    2. Chen, Jian & Tang, Liping & Yang, Xinmin, 2023. "A Barzilai-Borwein descent method for multiobjective optimization problems," European Journal of Operational Research, Elsevier, vol. 311(1), pages 196-209.
    3. Kazemi, Abolghasem & Moreno, Jovita & Iribarren, Diego, 2023. "Economic optimization and comparative environmental assessment of natural gas combined cycle power plants with CO2 capture," Energy, Elsevier, vol. 277(C).
    4. Yulian Gao & Gehao Lu & Jimei Gao & Jinggang Li, 2023. "A High-Performance Federated Learning Aggregation Algorithm Based on Learning Rate Adjustment and Client Sampling," Mathematics, MDPI, vol. 11(20), pages 1-29, October.
    5. Ellen H. Fukuda & L. M. Graña Drummond & Fernanda M. P. Raupp, 2016. "An external penalty-type method for multicriteria," TOP: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(2), pages 493-513, July.
    6. Thai Chuong, 2013. "Newton-like methods for efficient solutions in vector optimization," Computational Optimization and Applications, Springer, vol. 54(3), pages 495-516, April.
    7. Morovati, Vahid & Pourkarimi, Latif, 2019. "Extension of Zoutendijk method for solving constrained multiobjective optimization problems," European Journal of Operational Research, Elsevier, vol. 273(1), pages 44-57.
    8. Konstantin Sonntag & Bennet Gebken & Georg Müller & Sebastian Peitz & Stefan Volkwein, 2024. "A Descent Method for Nonsmooth Multiobjective Optimization in Hilbert Spaces," Journal of Optimization Theory and Applications, Springer, vol. 203(1), pages 455-487, October.
    9. Miglierina, E. & Molho, E. & Recchioni, M.C., 2008. "Box-constrained multi-objective optimization: A gradient-like method without "a priori" scalarization," European Journal of Operational Research, Elsevier, vol. 188(3), pages 662-682, August.
    10. Erik Alex Papa Quiroz & Nancy Baygorrea Cusihuallpa & Nelson Maculan, 2020. "Inexact Proximal Point Methods for Multiobjective Quasiconvex Minimization on Hadamard Manifolds," Journal of Optimization Theory and Applications, Springer, vol. 186(3), pages 879-898, September.
    11. Kely D. V. Villacorta & Paulo R. Oliveira & Antoine Soubeyran, 2014. "A Trust-Region Method for Unconstrained Multiobjective Problems with Applications in Satisficing Processes," Journal of Optimization Theory and Applications, Springer, vol. 160(3), pages 865-889, March.
    12. Kanako Mita & Ellen H. Fukuda & Nobuo Yamashita, 2019. "Nonmonotone line searches for unconstrained multiobjective optimization problems," Journal of Global Optimization, Springer, vol. 75(1), pages 63-90, September.
    13. Miglierina Enrico & Molho Elena & Recchioni Maria Cristina, 2006. "Box-constrained vector optimization: a steepest descent method without “a priori” scalarization," Economics and Quantitative Methods qf0603, Department of Economics, University of Insubria.
    14. M. L. N. Gonçalves & F. S. Lima & L. F. Prudente, 2022. "Globally convergent Newton-type methods for multiobjective optimization," Computational Optimization and Applications, Springer, vol. 83(2), pages 403-434, November.
    15. Brian Dandurand & Margaret M. Wiecek, 2016. "Quadratic scalarization for decomposed multiobjective optimization," OR Spectrum: Quantitative Approaches in Management, Springer;Gesellschaft für Operations Research e.V., vol. 38(4), pages 1071-1096, October.
    16. Gonçalves, M.L.N. & Lima, F.S. & Prudente, L.F., 2022. "A study of Liu-Storey conjugate gradient methods for vector optimization," Applied Mathematics and Computation, Elsevier, vol. 425(C).
    17. J. Y. Bello Cruz & G. Bouza Allende, 2014. "A Steepest Descent-Like Method for Variable Order Vector Optimization Problems," Journal of Optimization Theory and Applications, Springer, vol. 162(2), pages 371-391, August.
    18. X. M. Wang & J. H. Wang & C. Li, 2023. "Convergence of Inexact Steepest Descent Algorithm for Multiobjective Optimizations on Riemannian Manifolds Without Curvature Constraints," Journal of Optimization Theory and Applications, Springer, vol. 198(1), pages 187-214, July.
    19. P. B. Assunção & O. P. Ferreira & L. F. Prudente, 2021. "Conditional gradient method for multiobjective optimization," Computational Optimization and Applications, Springer, vol. 78(3), pages 741-768, April.
    20. Qu, Shaojian & Ji, Ying & Jiang, Jianlin & Zhang, Qingpu, 2017. "Nonmonotone gradient methods for vector optimization with a portfolio optimization application," European Journal of Operational Research, Elsevier, vol. 263(2), pages 356-366.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:13:y:2025:i:4:p:627-:d:1591536. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.