IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i11p2431-d1154891.html
   My bibliography  Save this article

A Modified Gradient Method for Distributionally Robust Logistic Regression over the Wasserstein Ball

Author

Listed:
  • Luyun Wang

    (College of Economics and Management, Southwest University, Chongqing 400715, China
    School of Economics, Chongqing Financial and Economic College, Chongqing 401320, China)

  • Bo Zhou

    (College of Mathematics and Statistics, Chongqing Jiaotong University, Chongqing 400074, China)

Abstract

In this paper, a modified conjugate gradient method under the forward-backward splitting framework is proposed to further improve the numerical efficiency for solving the distributionally robust Logistic regression model over the Wasserstein ball, which comprises two phases: in the first phase, a conjugate gradient descent step is performed, and in the second phase, an instantaneous optimization problem is formulated and solved with a trade-off minimization of the regularization term, while simultaneously staying in close proximity to the interim point obtained in the first phase. The modified conjugate gradient method is proven to attain the optimal solution of the Wasserstein distributionally robust Logistic regression model with nonsummable steplength at a convergence rate of 1 / T . Finally, several numerical experiments to validate the effectiveness of theoretical analysis are conducted, which demonstrate that this method outperforms the off-the-shelf solver and the existing first-order algorithmic frameworks.

Suggested Citation

  • Luyun Wang & Bo Zhou, 2023. "A Modified Gradient Method for Distributionally Robust Logistic Regression over the Wasserstein Ball," Mathematics, MDPI, vol. 11(11), pages 1-15, May.
    • Handle: RePEc:gam:jmathe:v:11:y:2023:i:11:p:2431-:d:1154891
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/11/2431/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/11/11/2431/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Gonglin Yuan & Zehong Meng & Yong Li, 2016. "A Modified Hestenes and Stiefel Conjugate Gradient Algorithm for Large-Scale Nonsmooth Minimizations and Nonlinear Equations," Journal of Optimization Theory and Applications, Springer, vol. 168(1), pages 129-152, January.
    2. Lorenzo Stella & Andreas Themelis & Panagiotis Patrinos, 2017. "Forward–backward quasi-Newton methods for nonsmooth optimization problems," Computational Optimization and Applications, Springer, vol. 67(3), pages 443-487, July.
    3. Tsegay Giday Woldu & Haibin Zhang & Xin Zhang & Yemane Hailu Fissuh, 2020. "A Modified Nonlinear Conjugate Gradient Algorithm for Large-Scale Nonsmooth Convex Optimization," Journal of Optimization Theory and Applications, Springer, vol. 185(1), pages 223-238, April.
    4. Wolfram Wiesemann & Daniel Kuhn & Melvyn Sim, 2014. "Distributionally Robust Convex Optimization," Operations Research, INFORMS, vol. 62(6), pages 1358-1376, December.
    5. M. L. N. Gonçalves & L. F. Prudente, 2020. "On the extension of the Hager–Zhang conjugate gradient method for vector optimization," Computational Optimization and Applications, Springer, vol. 76(3), pages 889-916, July.
    6. A. Chambolle & Ch. Dossal, 2015. "On the Convergence of the Iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm”," Journal of Optimization Theory and Applications, Springer, vol. 166(3), pages 968-982, September.
    7. Jose Blanchet & Karthyek Murthy, 2019. "Quantifying Distributional Model Risk via Optimal Transport," Mathematics of Operations Research, INFORMS, vol. 44(2), pages 565-600, May.
    8. Aharon Ben-Tal & Dick den Hertog & Anja De Waegenaere & Bertrand Melenberg & Gijs Rennen, 2013. "Robust Solutions of Optimization Problems Affected by Uncertain Probabilities," Management Science, INFORMS, vol. 59(2), pages 341-357, April.
    9. Luo, Fengqiao & Mehrotra, Sanjay, 2019. "Decomposition algorithm for distributionally robust optimization using Wasserstein metric with an application to a class of regression models," European Journal of Operational Research, Elsevier, vol. 278(1), pages 20-35.
    10. Dariush Khezrimotlagh & Yao Chen, 2018. "The Optimization Approach," International Series in Operations Research & Management Science, in: Decision Making and Performance Evaluation Using Data Envelopment Analysis, chapter 0, pages 107-134, Springer.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Haolin Ruan & Zhi Chen & Chin Pang Ho, 2023. "Adjustable Distributionally Robust Optimization with Infinitely Constrained Ambiguity Sets," INFORMS Journal on Computing, INFORMS, vol. 35(5), pages 1002-1023, September.
    2. Feng Liu & Zhi Chen & Shuming Wang, 2023. "Globalized Distributionally Robust Counterpart," INFORMS Journal on Computing, INFORMS, vol. 35(5), pages 1120-1142, September.
    3. Zhi Chen & Weijun Xie, 2021. "Regret in the Newsvendor Model with Demand and Yield Randomness," Production and Operations Management, Production and Operations Management Society, vol. 30(11), pages 4176-4197, November.
    4. Henry Lam, 2019. "Recovering Best Statistical Guarantees via the Empirical Divergence-Based Distributionally Robust Optimization," Operations Research, INFORMS, vol. 67(4), pages 1090-1105, July.
    5. Aleksandrina Goeva & Henry Lam & Huajie Qian & Bo Zhang, 2019. "Optimization-Based Calibration of Simulation Input Models," Operations Research, INFORMS, vol. 67(5), pages 1362-1382, September.
    6. Zhi Chen & Melvyn Sim & Huan Xu, 2019. "Distributionally Robust Optimization with Infinitely Constrained Ambiguity Sets," Operations Research, INFORMS, vol. 67(5), pages 1328-1344, September.
    7. Chassein, André & Goerigk, Marc, 2018. "Compromise solutions for robust combinatorial optimization with variable-sized uncertainty," European Journal of Operational Research, Elsevier, vol. 269(2), pages 544-555.
    8. Taozeng Zhu & Jingui Xie & Melvyn Sim, 2022. "Joint Estimation and Robustness Optimization," Management Science, INFORMS, vol. 68(3), pages 1659-1677, March.
    9. Shunichi Ohmori, 2021. "A Predictive Prescription Using Minimum Volume k -Nearest Neighbor Enclosing Ellipsoid and Robust Optimization," Mathematics, MDPI, vol. 9(2), pages 1-16, January.
    10. L. Jeff Hong & Zhiyuan Huang & Henry Lam, 2021. "Learning-Based Robust Optimization: Procedures and Statistical Guarantees," Management Science, INFORMS, vol. 67(6), pages 3447-3467, June.
    11. Pengyu Qian & Zizhuo Wang & Zaiwen Wen, 2015. "A Composite Risk Measure Framework for Decision Making under Uncertainty," Papers 1501.01126, arXiv.org.
    12. Antonio J. Conejo & Nicholas G. Hall & Daniel Zhuoyu Long & Runhao Zhang, 2021. "Robust Capacity Planning for Project Management," INFORMS Journal on Computing, INFORMS, vol. 33(4), pages 1533-1550, October.
    13. Vishal Gupta, 2019. "Near-Optimal Bayesian Ambiguity Sets for Distributionally Robust Optimization," Management Science, INFORMS, vol. 65(9), pages 4242-4260, September.
    14. Ran Ji & Miguel A. Lejeune, 2021. "Data-Driven Optimization of Reward-Risk Ratio Measures," INFORMS Journal on Computing, INFORMS, vol. 33(3), pages 1120-1137, July.
    15. Shubhechyya Ghosal & Wolfram Wiesemann, 2020. "The Distributionally Robust Chance-Constrained Vehicle Routing Problem," Operations Research, INFORMS, vol. 68(3), pages 716-732, May.
    16. Wang, Fan & Zhang, Chao & Zhang, Hui & Xu, Liang, 2021. "Short-term physician rescheduling model with feature-driven demand for mental disorders outpatients," Omega, Elsevier, vol. 105(C).
    17. Zhi Chen & Melvyn Sim & Peng Xiong, 2020. "Robust Stochastic Optimization Made Easy with RSOME," Management Science, INFORMS, vol. 66(8), pages 3329-3339, August.
    18. Jose Blanchet & Karthyek Murthy & Nian Si, 2022. "Confidence regions in Wasserstein distributionally robust estimation [Distributionally robust groupwise regularization estimator]," Biometrika, Biometrika Trust, vol. 109(2), pages 295-315.
    19. Ren, Ke & Bidkhori, Hoda, 2023. "A study of data-driven distributionally robust optimization with incomplete joint data under finite support," European Journal of Operational Research, Elsevier, vol. 305(2), pages 754-765.
    20. Postek, K.S. & den Hertog, D. & Melenberg, B., 2015. "Computationally Tractable Counterparts of Distributionally Robust Constraints on Risk Measures (revision of CentER DP 2014-031)," Discussion Paper 2015-047, Tilburg University, Center for Economic Research.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:11:p:2431-:d:1154891. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.