IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v329y2026i1p63-78.html

A hybrid stochastic alternating direction method of multipliers for nonconvex and nonsmooth composite optimization

Author

Listed:
  • Zeng, Yuxuan
  • Bai, Jianchao
  • Wang, Shengjia
  • Wang, Zhiguo
  • Shen, Xiaojing

Abstract

Nonconvex and nonsmooth composite optimization problems with linear constraints have gained significant attention in practical applications. This paper proposes a hybrid stochastic Alternating Direction Method of Multipliers (ADMM) leveraging a novel hybrid estimator to solve such problems with expectation or finite-sum objective functions. Compared to existing double-loop stochastic ADMMs, our method features simpler updates enabled by a single-loop, single-sample framework, while avoiding the need for checkpoint selection. Under mild conditions, we analyze the explicit relationships between key parameters using refined Lyapunov functions and rigorously establish the sublinear convergence. To the best of our knowledge, our work is the first single-loop stochastic ADMM for solving both expectation and finite-sum problems while matching the best-known oracle complexity bound comparable to state-of-the-art double-loop stochastic ADMMs. Numerical experiments on several different nonconvex minimization tasks demonstrate the superior performance of the proposed method.

Suggested Citation

  • Zeng, Yuxuan & Bai, Jianchao & Wang, Shengjia & Wang, Zhiguo & Shen, Xiaojing, 2026. "A hybrid stochastic alternating direction method of multipliers for nonconvex and nonsmooth composite optimization," European Journal of Operational Research, Elsevier, vol. 329(1), pages 63-78.
    • Handle: RePEc:eee:ejores:v:329:y:2026:i:1:p:63-78
    DOI: 10.1016/j.ejor.2025.10.024
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221725008446
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2025.10.024?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Jianchao Bai & Jicheng Li & Fengmin Xu & Hongchao Zhang, 2018. "Generalized symmetric ADMM for separable convex optimization," Computational Optimization and Applications, Springer, vol. 70(1), pages 129-170, May.
    2. Le Cadre, Hélène & Mou, Yuting & Höschle, Hanspeter, 2022. "Parametrized Inexact-ADMM based coordination games: A normalized Nash equilibrium approach," European Journal of Operational Research, Elsevier, vol. 296(2), pages 696-716.
    3. Jianchao Bai & Miao Zhang & Hongchao Zhang, 2025. "An inexact ADMM for separable nonconvex and nonsmooth optimization," Computational Optimization and Applications, Springer, vol. 90(2), pages 445-479, March.
    4. Mercier, Quentin & Poirion, Fabrice & Désidéri, Jean-Antoine, 2018. "A stochastic multiple gradient descent algorithm," European Journal of Operational Research, Elsevier, vol. 271(3), pages 808-817.
    5. Jianchao Bai & William W. Hager & Hongchao Zhang, 2022. "An inexact accelerated stochastic ADMM for separable convex optimization," Computational Optimization and Applications, Springer, vol. 81(2), pages 479-518, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhongming Wu & Liu Yang & Valentina De Simone, 2026. "Behavioral portfolio optimization via cumulative prospect theory with a symmetric alternating direction method of multipliers," Computational Optimization and Applications, Springer, vol. 93(1), pages 303-334, January.
    2. Fanke Kong & Zheng-Fen Jin & Youlin Shang & Yunhai Xiao & Deren Han, 2026. "Semi-proximal ADMM for fused Lasso penalized least absolute deviation in partially linear model," Computational Statistics, Springer, vol. 41(2), pages 1-36, February.
    3. Yulian Gao & Gehao Lu & Jimei Gao & Jinggang Li, 2023. "A High-Performance Federated Learning Aggregation Algorithm Based on Learning Rate Adjustment and Client Sampling," Mathematics, MDPI, vol. 11(20), pages 1-29, October.
    4. Maryam Yashtini, 2022. "Convergence and rate analysis of a proximal linearized ADMM for nonconvex nonsmooth optimization," Journal of Global Optimization, Springer, vol. 84(4), pages 913-939, December.
    5. Jianchao Bai & William W. Hager & Hongchao Zhang, 2022. "An inexact accelerated stochastic ADMM for separable convex optimization," Computational Optimization and Applications, Springer, vol. 81(2), pages 479-518, March.
    6. Peixuan Li & Yuan Shen & Suhong Jiang & Zehua Liu & Caihua Chen, 2021. "Convergence study on strictly contractive Peaceman–Rachford splitting method for nonseparable convex minimization models with quadratic coupling terms," Computational Optimization and Applications, Springer, vol. 78(1), pages 87-124, January.
    7. Jianchao Bai & Miao Zhang & Hongchao Zhang, 2025. "An inexact ADMM for separable nonconvex and nonsmooth optimization," Computational Optimization and Applications, Springer, vol. 90(2), pages 445-479, March.
    8. Albert S. Berahas & Jiahao Shi & Zihong Yi & Baoyu Zhou, 2023. "Accelerating stochastic sequential quadratic programming for equality constrained optimization using predictive variance reduction," Computational Optimization and Applications, Springer, vol. 86(1), pages 79-116, September.
    9. Zhao, Wei & Liao, Qi & Qiu, Rui & Liu, Chunying & Xu, Ning & Yu, Xiao & Liang, Yongtu, 2024. "Pipe sharing: A bilevel optimization model for the optimal capacity allocation of natural gas network," Applied Energy, Elsevier, vol. 359(C).
    10. Suyun Liu & Luis Nunes Vicente, 2022. "Accuracy and fairness trade-offs in machine learning: a stochastic multi-objective approach," Computational Management Science, Springer, vol. 19(3), pages 513-537, July.
    11. Ruyu Wang & Cong Liu & Quanwei Gao, 2025. "A unified mini-batch stochastic accelerated method for nonconvex stochastic programming," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 102(2), pages 469-489, December.
    12. Jingjing Peng & Zhijie Wang & Siting Yu & Zengao Tang, 2025. "An Improvement of the Alternating Direction Method of Multipliers to Solve the Convex Optimization Problem," Mathematics, MDPI, vol. 13(5), pages 1-19, February.
    13. Wang, Qihui & Li, Zhengshuo, 2026. "Learning-to-optimize infused decentralized disaggregation for multi-entity technical VPP considering equity and privacy," Applied Energy, Elsevier, vol. 405(C).
    14. Shengjie Xu & Bingsheng He, 2021. "A parallel splitting ALM-based algorithm for separable convex programming," Computational Optimization and Applications, Springer, vol. 80(3), pages 831-851, December.
    15. Yibao Fan & Zheng-Fen Jin & Youlin Shang & Deren Han, 2026. "A Bregman ADMM for Robust Fused Lasso Estimation with Doubly Nonconvex Regularizers," Journal of Global Optimization, Springer, vol. 94(3), pages 755-782, March.
    16. Chen, Jian & Tang, Liping & Yang, Xinmin, 2023. "A Barzilai-Borwein descent method for multiobjective optimization problems," European Journal of Operational Research, Elsevier, vol. 311(1), pages 196-209.
    17. Zhihao Li & Qingtao Wu & Moli Zhang & Lin Wang & Youming Ge & Guoyong Wang, 2025. "Stochastic Zeroth-Order Multi-Gradient Algorithm for Multi-Objective Optimization," Mathematics, MDPI, vol. 13(4), pages 1-31, February.
    18. Yaning Jiang & Deren Han & Xingju Cai, 2022. "An efficient partial parallel method with scaling step size strategy for three-block convex optimization problems," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 96(3), pages 383-419, December.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:329:y:2026:i:1:p:63-78. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.