IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2607.18001.html

AlphaZeroBeta: Deep Reinforcement Learning for Market-Neutral Portfolios

Author

Listed:
  • Boris Belyakov

Abstract

Market-neutral portfolios aim to generate consistent returns while offsetting systematic market risk. Traditional approaches based on factor models or convex optimization often underperform during market regime shifts or when structural assumptions break down. We propose AlphaZeroBeta, a deep reinforcement learning framework designed to deliver benchmark-relative alpha (excess returns) with near-zero beta (market neutrality). AlphaZeroBeta combines a composite reward function that balances risk-adjusted excess return, benchmark correlation, and transaction costs with a CNN-GRU policy trained end-to-end via Recurrent PPO and evaluated through a rolling walk-forward protocol. Backtests covering 2014-2024 across seven equity indices show that the model achieves higher Sharpe ratios than the baselines while maintaining near-zero benchmark correlations and competitive drawdowns.

Suggested Citation

  • Boris Belyakov, 2026. "AlphaZeroBeta: Deep Reinforcement Learning for Market-Neutral Portfolios," Papers 2607.18001, arXiv.org.
    • Handle: RePEc:arx:papers:2607.18001
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2607.18001
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Carhart, Mark M, 1997. "On Persistence in Mutual Fund Performance," Journal of Finance, American Finance Association, vol. 52(1), pages 57-82, March.
    2. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    3. Tidor-Vlad Pricope, 2021. "Deep Reinforcement Learning in Quantitative Algorithmic Trading: A Review," Papers 2106.00123, arXiv.org.
    4. Jegadeesh, Narasimhan, 1990. "Evidence of Predictable Behavior of Security Returns," Journal of Finance, American Finance Association, vol. 45(3), pages 881-898, July.
    5. Ben Hambly & Renyuan Xu & Huining Yang, 2021. "Recent Advances in Reinforcement Learning in Finance," Papers 2112.04553, arXiv.org, revised Feb 2023.
    6. Marco Avellaneda & Jeong-Hyun Lee, 2010. "Statistical arbitrage in the US equities market," Quantitative Finance, Taylor & Francis Journals, vol. 10(7), pages 761-782.
    7. Andrew J. Patton, 2009. "Are "Market Neutral" Hedge Funds Really Market Neutral?," The Review of Financial Studies, Society for Financial Studies, vol. 22(7), pages 2295-2330, July.
    8. Shiller, Robert J, 1981. "Do Stock Prices Move Too Much to be Justified by Subsequent Changes in Dividends?," American Economic Review, American Economic Association, vol. 71(3), pages 421-436, June.
    9. Alexei Chekhlov & Stanislav Uryasev & Michael Zabarankin, 2005. "Drawdown Measure In Portfolio Optimization," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 8(01), pages 13-58.
    10. Fama, Eugene F. & French, Kenneth R., 2015. "A five-factor asset pricing model," Journal of Financial Economics, Elsevier, vol. 116(1), pages 1-22.
    11. Krauss, Christopher & Do, Xuan Anh & Huck, Nicolas, 2017. "Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500," European Journal of Operational Research, Elsevier, vol. 259(2), pages 689-702.
    12. Fischer, Thomas G., 2018. "Reinforcement learning in financial markets - a survey," FAU Discussion Papers in Economics 12/2018, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    13. Clifford S. Asness & Andrea Frazzini & Lasse H. Pedersen, 2012. "Leverage Aversion and Risk Parity," Financial Analysts Journal, Taylor & Francis Journals, vol. 68(1), pages 47-59, January.
    14. Christopher Krauss & Anh Do & Nicolas Huck, 2017. "Deep neural networks, gradient-boosted trees, random forests: Statistical arbitrage on the S&P 500," Post-Print hal-01768895, HAL.
    15. Fama, Eugene F. & French, Kenneth R., 1993. "Common risk factors in the returns on stocks and bonds," Journal of Financial Economics, Elsevier, vol. 33(1), pages 3-56, February.
    16. Andrew W. Lo, 2002. "The Statistics of Sharpe Ratios," Financial Analysts Journal, Taylor & Francis Journals, vol. 58(4), pages 36-52, July.
    17. Yahui Bai & Yuhe Gao & Runzhe Wan & Sheng Zhang & Rui Song, 2024. "A Review of Reinforcement Learning in Financial Applications," Papers 2411.12746, arXiv.org.
    18. Zhengyao Jiang & Dixing Xu & Jinjun Liang, 2017. "A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem," Papers 1706.10059, arXiv.org, revised Jul 2017.
    19. Bruno Gašperov & Stjepan Begušić & Petra Posedel Šimović & Zvonko Kostanjčar, 2021. "Reinforcement Learning Approaches to Optimal Market Making," Mathematics, MDPI, vol. 9(21), pages 1-22, October.
    20. Sayat R. Baronyan & İ. İlkay Boduroğlu & Emrah Şener, 2010. "Investigation Of Stochastic Pairs Trading Strategies Under Different Volatility Regimes," Manchester School, University of Manchester, vol. 78(s1), pages 114-134, September.
    21. Zhipeng Liang & Hao Chen & Junhao Zhu & Kangkang Jiang & Yanran Li, 2018. "Adversarial Deep Reinforcement Learning in Portfolio Management," Papers 1808.09940, arXiv.org, revised Nov 2018.
    22. Matteo Barigozzi & Marc Hallin, 2026. "The Dynamic, the Static, and the Weak: Factor Models and the Analysis of High‐Dimensional Time Series," Journal of Time Series Analysis, Wiley Blackwell, vol. 47(1), pages 201-219, January.
    23. Arthur Charpentier & Romuald Élie & Carl Remlinger, 2023. "Reinforcement Learning in Economics and Finance," Computational Economics, Springer;Society for Computational Economics, vol. 62(1), pages 425-462, June.
    24. Evan Gatev & William N. Goetzmann & K. Geert Rouwenhorst, 2006. "Pairs Trading: Performance of a Relative-Value Arbitrage Rule," The Review of Financial Studies, Society for Financial Studies, vol. 19(3), pages 797-827.
    25. Ben Hambly & Renyuan Xu & Huining Yang, 2023. "Recent advances in reinforcement learning in finance," Mathematical Finance, Wiley Blackwell, vol. 33(3), pages 437-503, July.
    26. Jegadeesh, Narasimhan & Titman, Sheridan, 1993. "Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency," Journal of Finance, American Finance Association, vol. 48(1), pages 65-91, March.
    27. Junyi Ye & Bhaskar Goswami & Jingyi Gu & Ajim Uddin & Guiling Wang, 2024. "From Factor Models to Deep Learning: Machine Learning in Reshaping Empirical Asset Pricing," Papers 2403.06779, arXiv.org.
    28. Black, Fischer, 1972. "Capital Market Equilibrium with Restricted Borrowing," The Journal of Business, University of Chicago Press, vol. 45(3), pages 444-455, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Flori, Andrea & Regoli, Daniele, 2021. "Revealing Pairs-trading opportunities with long short-term memory networks," European Journal of Operational Research, Elsevier, vol. 295(2), pages 772-791.
    2. Fischer, Thomas & Krauss, Christopher, 2017. "Deep learning with long short-term memory networks for financial market predictions," FAU Discussion Papers in Economics 11/2017, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    3. Fischer, Thomas & Krauss, Christopher, 2018. "Deep learning with long short-term memory networks for financial market predictions," European Journal of Operational Research, Elsevier, vol. 270(2), pages 654-669.
    4. Adam Zaremba & Jacob Koby Shemer, 2018. "Price-Based Investment Strategies," Springer Books, Springer, number 978-3-319-91530-2, March.
    5. Fernando Rubio, 2005. "Eficiencia De Mercado, Administracion De Carteras De Fondos Y Behavioural Finance," Finance 0503028, University Library of Munich, Germany, revised 23 Jul 2005.
    6. Bradrania, Reza & Veron, Jose Francisco, 2023. "The beta anomaly in the Australian stock market and the lottery demand," Pacific-Basin Finance Journal, Elsevier, vol. 77(C).
    7. Kentaro Imajo & Kentaro Minami & Katsuya Ito & Kei Nakagawa, 2020. "Deep Portfolio Optimization via Distributional Prediction of Residual Factors," Papers 2012.07245, arXiv.org.
    8. Tobias Wiest, 2023. "Momentum: what do we know 30 years after Jegadeesh and Titman’s seminal paper?," Financial Markets and Portfolio Management, Springer;Swiss Society for Financial Market Research, vol. 37(1), pages 95-114, March.
    9. Stefan Nagel, 2013. "Empirical Cross-Sectional Asset Pricing," Annual Review of Financial Economics, Annual Reviews, vol. 5(1), pages 167-199, November.
    10. Knoll, Julian & Stübinger, Johannes & Grottke, Michael, 2017. "Exploiting social media with higher-order Factorization Machines: Statistical arbitrage on high-frequency data of the S&P 500," FAU Discussion Papers in Economics 13/2017, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    11. van Cappelle, Tjeerd & Pokidin, Dmytro & Zwinkels, Remco C.J., 2025. "The cross section of stock returns in an artificial stock market," Journal of Economic Behavior & Organization, Elsevier, vol. 239(C).
    12. Stübinger, Johannes & Endres, Sylvia, 2017. "Pairs trading with a mean-reverting jump-diffusion model on high-frequency data," FAU Discussion Papers in Economics 10/2017, Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
    13. Sebastien Valeyre & Sofiane Aboura & Denis Grebenkov, 2019. "The Reactive Beta Model," Journal of Financial Research, Southern Finance Association;Southwestern Finance Association, vol. 42(1), pages 71-113, March.
    14. Amit Goyal, 2012. "Empirical cross-sectional asset pricing: a survey," Financial Markets and Portfolio Management, Springer;Swiss Society for Financial Market Research, vol. 26(1), pages 3-38, March.
    15. Pedro M. Mirete-Ferrer & Alberto Garcia-Garcia & Juan Samuel Baixauli-Soler & Maria A. Prats, 2022. "A Review on Machine Learning for Asset Management," Risks, MDPI, vol. 10(4), pages 1-46, April.
    16. Didier SORNETTE, 2014. "Physics and Financial Economics (1776-2014): Puzzles, Ising and Agent-Based Models," Swiss Finance Institute Research Paper Series 14-25, Swiss Finance Institute.
    17. Sebastien Valeyre, 2020. "Refined model of the covariance/correlation matrix between securities," Papers 2001.08911, arXiv.org.
    18. Nobel Prize Committee, 2013. "Understanding Asset Prices," Nobel Prize in Economics documents 2013-1, Nobel Prize Committee.
    19. Kotaro Miwa, 2020. "Market Closures and Cross-sectional Stock Returns," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 27(1), pages 1-33, March.
    20. D. Sornette, 2014. "Physics and Financial Economics (1776-2014): Puzzles, Ising and Agent-Based models," Papers 1404.0243, arXiv.org.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2607.18001. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.