IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2504.13529.html
   My bibliography  Save this paper

Risk-aware black-box portfolio construction using Bayesian optimization with adaptive weighted Lagrangian estimator

Author

Listed:
  • Zinuo You
  • John Cartlidge
  • Karen Elliott
  • Menghan Ge
  • Daniel Gold

Abstract

Existing portfolio management approaches are often black-box models due to safety and commercial issues in the industry. However, their performance can vary considerably whenever market conditions or internal trading strategies change. Furthermore, evaluating these non-transparent systems is expensive, where certain budgets limit observations of the systems. Therefore, optimizing performance while controlling the potential risk of these financial systems has become a critical challenge. This work presents a novel Bayesian optimization framework to optimize black-box portfolio management models under limited observations. In conventional Bayesian optimization settings, the objective function is to maximize the expectation of performance metrics. However, simply maximizing performance expectations leads to erratic optimization trajectories, which exacerbate risk accumulation in portfolio management. Meanwhile, this can lead to misalignment between the target distribution and the actual distribution of the black-box model. To mitigate this problem, we propose an adaptive weight Lagrangian estimator considering dual objective, which incorporates maximizing model performance and minimizing variance of model observations. Extensive experiments demonstrate the superiority of our approach over five backtest settings with three black-box stock portfolio management models. Ablation studies further verify the effectiveness of the proposed estimator.

Suggested Citation

  • Zinuo You & John Cartlidge & Karen Elliott & Menghan Ge & Daniel Gold, 2025. "Risk-aware black-box portfolio construction using Bayesian optimization with adaptive weighted Lagrangian estimator," Papers 2504.13529, arXiv.org.
  • Handle: RePEc:arx:papers:2504.13529
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2504.13529
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ma, Tian & Wang, Wanwan & Chen, Yu, 2023. "Attention is all you need: An interpretable transformer-based asset allocation approach," International Review of Financial Analysis, Elsevier, vol. 90(C).
    2. Michal Kaut & Hercules Vladimirou & Stein W. Wallace & Stavros A. Zenios, 2007. "Stability analysis of portfolio management with conditional value-at-risk," Quantitative Finance, Taylor & Francis Journals, vol. 7(4), pages 397-409.
    3. Florian Herzog & Gabriel Dondi & Hans P. Geering, 2007. "Stochastic Model Predictive Control And Portfolio Optimization," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 10(02), pages 203-233.
    4. Barro, Diana & Consigli, Giorgio & Varun, Vivek, 2022. "A stochastic programming model for dynamic portfolio management with financial derivatives," Journal of Banking & Finance, Elsevier, vol. 140(C).
    5. Yuanyuan Zhang & Xiang Li & Sini Guo, 2018. "Portfolio selection problems with Markowitz’s mean–variance framework: a review of literature," Fuzzy Optimization and Decision Making, Springer, vol. 17(2), pages 125-158, June.
    6. Apichat Chaweewanchon & Rujira Chaysiri, 2022. "Markowitz Mean-Variance Portfolio Optimization with Predictive Stock Selection Using Machine Learning," IJFS, MDPI, vol. 10(3), pages 1-19, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kanwal Iqbal Khan & Syed M. Waqar Azeem Naqvi & Muhammad Mudassar Ghafoor & Rana Shahid Imdad Akash, 2020. "Sustainable Portfolio Optimization with Higher-Order Moments of Risk," Sustainability, MDPI, vol. 12(5), pages 1-14, March.
    2. Ken Kobayashi & Yuichi Takano & Kazuhide Nakata, 2021. "Bilevel cutting-plane algorithm for cardinality-constrained mean-CVaR portfolio optimization," Journal of Global Optimization, Springer, vol. 81(2), pages 493-528, October.
    3. Xiangyu Cui & Jianjun Gao & Yun Shi, 2021. "Multi-period mean–variance portfolio optimization with management fees," Operational Research, Springer, vol. 21(2), pages 1333-1354, June.
    4. Topaloglou, Nikolas & Vladimirou, Hercules & Zenios, Stavros A., 2020. "Integrated dynamic models for hedging international portfolio risks," European Journal of Operational Research, Elsevier, vol. 285(1), pages 48-65.
    5. Xiao, Xiang & Hua, Xia & Qin, Kexin, 2024. "A self-attention based cross-sectional return forecasting model with evidence from the Chinese market," Finance Research Letters, Elsevier, vol. 62(PA).
    6. Vladimir Dombrovskii & Tatyana Obedko, 2014. "Portfolio Optimization in the Financial Market with Correlated Returns under Constraints, Transaction Costs and Different Rates for Borrowing and Lending," Papers 1410.8042, arXiv.org.
    7. Yuichi Takano & Keisuke Nanjo & Noriyoshi Sukegawa & Shinji Mizuno, 2015. "Cutting plane algorithms for mean-CVaR portfolio optimization with nonconvex transaction costs," Computational Management Science, Springer, vol. 12(2), pages 319-340, April.
    8. Pejman Peykani & Mojtaba Nouri & Mir Saman Pishvaee & Camelia Oprean-Stan & Emran Mohammadi, 2023. "Credibilistic Multi-Period Mean-Entropy Rolling Portfolio Optimization Problem Based on Multi-Stage Scenario Tree," Mathematics, MDPI, vol. 11(18), pages 1-23, September.
    9. Lotfi, Somayyeh & Zeniosn, Stravros A., 2016. "Equivalence of Robust VaR and CVaR Optimization," Working Papers 16-03, University of Pennsylvania, Wharton School, Weiss Center.
    10. Yuling Max Chen & Bin Li & David Saunders, 2025. "Exploratory Mean-Variance Portfolio Optimization with Regime-Switching Market Dynamics," Papers 2501.16659, arXiv.org.
    11. Vladimir Dombrovskii & Tatyana Obyedko, 2014. "Dynamic Investment Portfolio Optimization under Constraints in the Financial Market with Regime Switching using Model Predictive Control," Papers 1410.1136, arXiv.org.
    12. Elçin Çetinkaya & Aurélie Thiele, 2016. "A moment matching approach to log-normal portfolio optimization," Computational Management Science, Springer, vol. 13(4), pages 501-520, October.
    13. Chul Jang & Andrew Clare & Iqbal Owadally, 2024. "Liability-driven investment for pension funds: stochastic optimization with real assets," Risk Management, Palgrave Macmillan, vol. 26(3), pages 1-32, September.
    14. Staino, Alessandro & Russo, Emilio, 2015. "A moment-matching method to generate arbitrage-free scenarios," European Journal of Operational Research, Elsevier, vol. 246(2), pages 619-630.
    15. Vit Prochazka & Stein W. Wallace, 2020. "Scenario tree construction driven by heuristic solutions of the optimization problem," Computational Management Science, Springer, vol. 17(2), pages 277-307, June.
    16. Zhao, Daping & Bai, Lin & Fang, Yong & Wang, Shouyang, 2022. "Multi‐period portfolio selection with investor views based on scenario tree," Applied Mathematics and Computation, Elsevier, vol. 418(C).
    17. Shanshan Guo & Lei Zhao & Xiaowei Xu, 2016. "Impact of supply risks on procurement decisions," Annals of Operations Research, Springer, vol. 241(1), pages 411-430, June.
    18. Khemka, Gaurav & Steffensen, Mogens & Warren, Geoffrey J., 2024. "A buy-hold-sell pension saving strategy," Insurance: Mathematics and Economics, Elsevier, vol. 119(C), pages 1-16.
    19. Wang, Xin & Fagerholt, Kjetil & Wallace, Stein W., 2018. "Planning for charters: A stochastic maritime fleet composition and deployment problem," Omega, Elsevier, vol. 79(C), pages 54-66.
    20. Jörgen Blomvall & Jonas Ekblom, 2018. "Corporate hedging: an answer to the “how” question," Annals of Operations Research, Springer, vol. 266(1), pages 35-69, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2504.13529. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.