IDEAS home Printed from https://ideas.repec.org/a/spr/joptap/v142y2009i1d10.1007_s10957-009-9522-7.html
   My bibliography  Save this article

Subgradient Methods for Saddle-Point Problems

Author

Listed:
  • A. Nedić

    (University of Illinois at Urbana-Champaign)

  • A. Ozdaglar

    (Massachusetts Institute of Technology)

Abstract

We study subgradient methods for computing the saddle points of a convex-concave function. Our motivation comes from networking applications where dual and primal-dual subgradient methods have attracted much attention in the design of decentralized network protocols. We first present a subgradient algorithm for generating approximate saddle points and provide per-iteration convergence rate estimates on the constructed solutions. We then focus on Lagrangian duality, where we consider a convex primal optimization problem and its Lagrangian dual problem, and generate approximate primal-dual optimal solutions as approximate saddle points of the Lagrangian function. We present a variation of our subgradient method under the Slater constraint qualification and provide stronger estimates on the convergence rate of the generated primal sequences. In particular, we provide bounds on the amount of feasibility violation and on the primal objective function values at the approximate solutions. Our algorithm is particularly well-suited for problems where the subgradient of the dual function cannot be evaluated easily (equivalently, the minimum of the Lagrangian function at a dual solution cannot be computed efficiently), thus impeding the use of dual subgradient methods.

Suggested Citation

  • A. Nedić & A. Ozdaglar, 2009. "Subgradient Methods for Saddle-Point Problems," Journal of Optimization Theory and Applications, Springer, vol. 142(1), pages 205-228, July.
    • Handle: RePEc:spr:joptap:v:142:y:2009:i:1:d:10.1007_s10957-009-9522-7
    DOI: 10.1007/s10957-009-9522-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10957-009-9522-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10957-009-9522-7?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. M.J. Kallio & A. Ruszczynski, 1994. "Perturbation Methods for Saddle Point Computation," Working Papers wp94038, International Institute for Applied Systems Analysis.
    2. NESTEROV, Yu., 2005. "Primal-dual subgradient methods for convex problems," LIDAM Discussion Papers CORE 2005067, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    3. Markku Kallio & Charles H. Rosa, 1999. "Large-Scale Convex Optimization Via Saddle Point Computation," Operations Research, INFORMS, vol. 47(1), pages 93-101, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Desmond Cai & Subhonmesh Bose & Adam Wierman, 2019. "On the Role of a Market Maker in Networked Cournot Competition," Mathematics of Operations Research, INFORMS, vol. 44(3), pages 1122-1144, August.
    2. Hu, Mian & Wang, Yan-Wu & Xiao, Jiang-Wen & Lin, Xiangning, 2019. "Multi-energy management with hierarchical distributed multi-scale strategy for pelagic islanded microgrid clusters," Energy, Elsevier, vol. 185(C), pages 910-921.
    3. Denizalp Goktas & Jiayi Zhao & Amy Greenwald, 2022. "Robust No-Regret Learning in Min-Max Stackelberg Games," Papers 2203.14126, arXiv.org, revised Apr 2022.
    4. Guigues, Vincent & Shapiro, Alexander & Cheng, Yi, 2023. "Duality and sensitivity analysis of multistage linear stochastic programs," European Journal of Operational Research, Elsevier, vol. 308(2), pages 752-767.
    5. J. Y. Bello Cruz & P. S. M. Santos & S. Scheimberg, 2013. "A Two-Phase Algorithm for a Variational Inequality Formulation of Equilibrium Problems," Journal of Optimization Theory and Applications, Springer, vol. 159(3), pages 562-575, December.
    6. A. M. Bagirov & L. Jin & N. Karmitsa & A. Al Nuaimat & N. Sultanova, 2013. "Subgradient Method for Nonconvex Nonsmooth Optimization," Journal of Optimization Theory and Applications, Springer, vol. 157(2), pages 416-435, May.
    7. Andrea Simonetto & Hadi Jamali-Rad, 2016. "Primal Recovery from Consensus-Based Dual Decomposition for Distributed Convex Optimization," Journal of Optimization Theory and Applications, Springer, vol. 168(1), pages 172-197, January.
    8. Bo Wei & William B. Haskell & Sixiang Zhao, 2020. "An inexact primal-dual algorithm for semi-infinite programming," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 91(3), pages 501-544, June.
    9. Wenjie Zhao & Guangming Zhou, 2022. "Local saddle points for unconstrained polynomial optimization," Computational Optimization and Applications, Springer, vol. 82(1), pages 89-106, May.
    10. Giorgio Costa & Roy H. Kwon, 2021. "Data-driven distributionally robust risk parity portfolio optimization," Papers 2110.06464, arXiv.org.
    11. Regina S. Burachik & Alfredo N. Iusem & Jefferson G. Melo, 2013. "An Inexact Modified Subgradient Algorithm for Primal-Dual Problems via Augmented Lagrangians," Journal of Optimization Theory and Applications, Springer, vol. 157(1), pages 108-131, April.
    12. Avinash N. Madavan & Subhonmesh Bose, 2021. "A Stochastic Primal-Dual Method for Optimization with Conditional Value at Risk Constraints," Journal of Optimization Theory and Applications, Springer, vol. 190(2), pages 428-460, August.
    13. Le Thi Khanh Hien & Renbo Zhao & William B. Haskell, 2024. "An Inexact Primal-Dual Smoothing Framework for Large-Scale Non-Bilinear Saddle Point Problems," Journal of Optimization Theory and Applications, Springer, vol. 200(1), pages 34-67, January.
    14. Denizalp Goktas & Amy Greenwald, 2022. "Gradient Descent Ascent in Min-Max Stackelberg Games," Papers 2208.09690, arXiv.org.
    15. Samira Mir Mazhari Anvar & Sohrab Khanmohammadi & Javad Musevi niya, 2016. "Game theoretic power allocation for fading MIMO multiple access channels with imperfect CSIR," Telecommunication Systems: Modelling, Analysis, Design and Management, Springer, vol. 61(4), pages 875-886, April.
    16. Guangming Zhou & Qin Wang & Wenjie Zhao, 2020. "Saddle points of rational functions," Computational Optimization and Applications, Springer, vol. 75(3), pages 817-832, April.
    17. Yaohua Hu & Jiawen Li & Carisa Kwok Wai Yu, 2020. "Convergence rates of subgradient methods for quasi-convex optimization problems," Computational Optimization and Applications, Springer, vol. 77(1), pages 183-212, September.
    18. Vinayaka G. Yaji & Shalabh Bhatnagar, 2020. "Stochastic Recursive Inclusions in Two Timescales with Nonadditive Iterate-Dependent Markov Noise," Mathematics of Operations Research, INFORMS, vol. 45(4), pages 1405-1444, November.
    19. Liusheng Hou & Hongjin He & Junfeng Yang, 2016. "A partially parallel splitting method for multiple-block separable convex programming with applications to robust PCA," Computational Optimization and Applications, Springer, vol. 63(1), pages 273-303, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. NESTEROV, Yu., 2005. "Minimizing functions with bounded variation of subgradients," LIDAM Discussion Papers CORE 2005079, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    2. S.D. Flam & A. Ruszczynski, 1994. "Noncooperative Convex Games: Computing Equilibrium By Partial Regularization," Working Papers wp94042, International Institute for Applied Systems Analysis.
    3. Wenjie Zhao & Guangming Zhou, 2022. "Local saddle points for unconstrained polynomial optimization," Computational Optimization and Applications, Springer, vol. 82(1), pages 89-106, May.
    4. Flam, Sjur & Ruszczynski, A., 2006. "Computing Normalized Equilibria in Convex-Concave Games," Working Papers 2006:9, Lund University, Department of Economics.
    5. Larsson, Torbjorn & Patriksson, Michael & Stromberg, Ann-Brith, 2003. "On the convergence of conditional [var epsilon]-subgradient methods for convex programs and convex-concave saddle-point problems," European Journal of Operational Research, Elsevier, vol. 151(3), pages 461-473, December.
    6. Maarit Kallio & Markku Kallio, 2002. "Nonparametric Methods for Evaluating Economic Efficiency and Imperfect Competition," Journal of Productivity Analysis, Springer, vol. 18(2), pages 171-189, September.
    7. M.J. Kallio & C.H. Rosa, 1994. "Large-Scale Convex Optimization via Saddle Point Computation," Working Papers wp94107, International Institute for Applied Systems Analysis.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:joptap:v:142:y:2009:i:1:d:10.1007_s10957-009-9522-7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.