IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v41y2025i4p1450-1460.html

Single-stage portfolio optimization with automated machine learning for M6

Author

Listed:
  • Huang, Xinyu
  • Newton, David P.
  • Platanakis, Emmanouil
  • Sutcliffe, Charles

Abstract

The goal of the M6 forecasting competition was to shed light on the efficient market hypothesis by evaluating the forecasting abilities of participants and performance of their investment strategies. In this paper, we challenge the ‘estimate-then-optimize’ approach with one that directly optimizes portfolio weights from data. We frame portfolio selection as a constrained penalized regression problem. We present a data-driven approach that automatically performs model selection and hyperparameter tuning to maximize the objective without noisy or potentially misspecified intermediate steps. Finally, we show how the portfolio weights can be optimized using the Method of Moving Asymptotes. Testing on the M6 competition data, our approach achieves a global rate of return of 9.5% and an information ratio of 5.045, which is in stark contrast to the mean IR of the M6 competition teams of −3.421 and the IR of 0.453 for the M6 benchmark.

Suggested Citation

  • Huang, Xinyu & Newton, David P. & Platanakis, Emmanouil & Sutcliffe, Charles, 2025. "Single-stage portfolio optimization with automated machine learning for M6," International Journal of Forecasting, Elsevier, vol. 41(4), pages 1450-1460.
    • Handle: RePEc:eee:intfor:v:41:y:2025:i:4:p:1450-1460
    DOI: 10.1016/j.ijforecast.2024.08.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207024000918
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijforecast.2024.08.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Daniele Bianchi & Matthias Büchner & Andrea Tamoni, 2021. "Bond Risk Premiums with Machine Learning [Quadratic term structure models: Theory and evidence]," The Review of Financial Studies, Society for Financial Studies, vol. 34(2), pages 1046-1089.
    2. Victor DeMiguel & Lorenzo Garlappi & Raman Uppal, 2009. "Optimal Versus Naive Diversification: How Inefficient is the 1-N Portfolio Strategy?," The Review of Financial Studies, Society for Financial Studies, vol. 22(5), pages 1915-1953, May.
    3. Harry Markowitz, 1952. "The Utility of Wealth," Journal of Political Economy, University of Chicago Press, vol. 60(2), pages 151-151.
    4. Cakici, Nusret & Fieberg, Christian & Metko, Daniel & Zaremba, Adam, 2023. "Machine learning goes global: Cross-sectional return predictability in international stock markets," Journal of Economic Dynamics and Control, Elsevier, vol. 155(C).
    5. Turan G Bali & Heiner Beckmeyer & Mathis Mörke & Florian Weigert & Stefano Giglio, 2023. "Option Return Predictability with Machine Learning and Big Data," The Review of Financial Studies, Society for Financial Studies, vol. 36(9), pages 3548-3602.
    6. Platanakis, Emmanouil & Urquhart, Andrew, 2019. "Portfolio management with cryptocurrencies: The role of estimation risk," Economics Letters, Elsevier, vol. 177(C), pages 76-80.
    7. Green, Richard C & Hollifield, Burton, 1992. "When Will Mean-Variance Efficient Portfolios Be Well Diversified?," Journal of Finance, American Finance Association, vol. 47(5), pages 1785-1809, December.
    8. DeMiguel, Victor & Gil-Bazo, Javier & Nogales, Francisco J. & Santos, André A.P., 2023. "Machine learning and fund characteristics help to select mutual funds with positive alpha," Journal of Financial Economics, Elsevier, vol. 150(3).
    9. Doron Avramov & Si Cheng & Lior Metzker, 2023. "Machine Learning vs. Economic Restrictions: Evidence from Stock Return Predictability," Management Science, INFORMS, vol. 69(5), pages 2587-2619, May.
    10. Nicolas Huck, 2019. "Large data sets and machine learning: Applications to statistical arbitrage," Post-Print hal-02143971, HAL.
    11. Wenbo Wu & Jiaqi Chen & Zhibin (Ben) Yang & Michael L. Tindall, 2021. "A Cross-Sectional Machine Learning Approach for Hedge Fund Return Prediction and Selection," Management Science, INFORMS, vol. 67(7), pages 4577-4601, July.
    12. Platanakis, Emmanouil & Sutcliffe, Charles & Ye, Xiaoxia, 2021. "Horses for courses: Mean-variance for asset allocation and 1/N for stock selection," European Journal of Operational Research, Elsevier, vol. 288(1), pages 302-317.
    13. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    14. Michael W. Brandt, 1999. "Estimating Portfolio and Consumption Choice: A Conditional Euler Equations Approach," Journal of Finance, American Finance Association, vol. 54(5), pages 1609-1645, October.
    15. John L. G. Board & Charles M. S. Sutcliffe, 1994. "Estimation Methods in Portfolio Selection and the Effectiveness of Short Sales Restrictions: UK Evidence," Management Science, INFORMS, vol. 40(4), pages 516-534, April.
    16. Daniel Kinn, 2018. "Reducing Estimation Risk in Mean-Variance Portfolios with Machine Learning," Papers 1804.01764, arXiv.org, revised Jul 2018.
    17. Ledoit, Olivier & Wolf, Michael, 2003. "Improved estimation of the covariance matrix of stock returns with an application to portfolio selection," Journal of Empirical Finance, Elsevier, vol. 10(5), pages 603-621, December.
    18. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    19. Fischer, Thomas & Krauss, Christopher, 2018. "Deep learning with long short-term memory networks for financial market predictions," European Journal of Operational Research, Elsevier, vol. 270(2), pages 654-669.
    20. Jorion, Philippe, 1986. "Bayes-Stein Estimation for Portfolio Analysis," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 21(3), pages 279-292, September.
    21. Best, Michael J & Grauer, Robert R, 1991. "On the Sensitivity of Mean-Variance-Efficient Portfolios to Changes in Asset Means: Some Analytical and Computational Results," The Review of Financial Studies, Society for Financial Studies, vol. 4(2), pages 315-342.
    22. Huck, Nicolas, 2019. "Large data sets and machine learning: Applications to statistical arbitrage," European Journal of Operational Research, Elsevier, vol. 278(1), pages 330-342.
    23. Grauer, Robert R. & Shen, Frederick C., 2000. "Do constraints improve portfolio performance?," Journal of Banking & Finance, Elsevier, vol. 24(8), pages 1253-1274, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Conlon, Thomas & Cotter, John & Kynigakis, Iason, 2025. "Asset allocation with factor-based covariance matrices," European Journal of Operational Research, Elsevier, vol. 325(1), pages 189-203.
    2. Cirulli, Antonello & Walker, Patrick S., 2025. "Outperforming equal weighting," Economics Letters, Elsevier, vol. 255(C).
    3. Rubesam, Alexandre, 2022. "Machine learning portfolios with equal risk contributions: Evidence from the Brazilian market," Emerging Markets Review, Elsevier, vol. 51(PB).
    4. Kellerer, Belinda, 2019. "Portfolio Optimization and Ambiguity Aversion," Junior Management Science (JUMS), Junior Management Science e. V., vol. 4(3), pages 305-338.
    5. Wu, Haoran & Gao, Zhiwei & Nie, Boyang & Zhao, Binru, 2025. "Can machines learn Chinese mutual funds?," Pacific-Basin Finance Journal, Elsevier, vol. 94(C).
    6. Victor DeMiguel & Lorenzo Garlappi & Francisco J. Nogales & Raman Uppal, 2009. "A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms," Management Science, INFORMS, vol. 55(5), pages 798-812, May.
    7. Cakici, Nusret & Zaremba, Adam, 2025. "Accounting vs technical information: what matters more for stock return predictability?," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 104(C).
    8. Díaz, Juan D. & Hansen, Erwin & Cabrera, Gabriel, 2024. "Machine-learning stock market volatility: Predictability, drivers, and economic value," International Review of Financial Analysis, Elsevier, vol. 94(C).
    9. Rad, Hossein & Low, Rand Kwong Yew & Miffre, Joëlle & Faff, Robert, 2023. "The commodity risk premium and neural networks," Journal of Empirical Finance, Elsevier, vol. 74(C).
    10. DeMiguel, Victor & Gil-Bazo, Javier & Nogales, Francisco J. & Santos, André A.P., 2023. "Machine learning and fund characteristics help to select mutual funds with positive alpha," Journal of Financial Economics, Elsevier, vol. 150(3).
    11. Yilie Huang & Yanwei Jia & Xun Yu Zhou, 2024. "Mean--Variance Portfolio Selection by Continuous-Time Reinforcement Learning: Algorithms, Regret Analysis, and Empirical Study," Papers 2412.16175, arXiv.org, revised Mar 2026.
    12. Kolm, Petter N. & Tütüncü, Reha & Fabozzi, Frank J., 2014. "60 Years of portfolio optimization: Practical challenges and current trends," European Journal of Operational Research, Elsevier, vol. 234(2), pages 356-371.
    13. Ghaemi Asl, Mahdi & Rashidi, Muhammad Mahdi & Tavakkoli, Hamid Raza & Rezgui, Hichem, 2024. "Does Islamic investing modify portfolio performance? Time-varying optimization strategies for conventional and Shariah energy-ESG-utilities portfolio," The Quarterly Review of Economics and Finance, Elsevier, vol. 94(C), pages 37-57.
    14. Constantinos Kardaras & Hyeng Keun Koo & Johannes Ruf, 2022. "Estimation of growth in fund models," Papers 2208.02573, arXiv.org.
    15. Lin William Cong & Guanhao Feng & Jingyu He & Xin He, 2022. "Growing the Efficient Frontier on Panel Trees," NBER Working Papers 30805, National Bureau of Economic Research, Inc.
    16. Thomas Conlon & John Cotter & Iason Kynigakis, 2021. "Machine Learning and Factor-Based Portfolio Optimization," Papers 2107.13866, arXiv.org.
    17. Kourtis, Apostolos & Dotsis, George & Markellos, Raphael N., 2012. "Parameter uncertainty in portfolio selection: Shrinking the inverse covariance matrix," Journal of Banking & Finance, Elsevier, vol. 36(9), pages 2522-2531.
    18. Behr, Patrick & Guettler, Andre & Miebs, Felix, 2013. "On portfolio optimization: Imposing the right constraints," Journal of Banking & Finance, Elsevier, vol. 37(4), pages 1232-1242.
    19. James DiLellio, 2015. "A Kalman filter control technique in mean-variance portfolio management," Journal of Economics and Finance, Springer;Academy of Economics and Finance, vol. 39(2), pages 235-261, April.
    20. Hanauer, Matthias X. & Kalsbach, Tobias, 2023. "Machine learning and the cross-section of emerging market stock returns," Emerging Markets Review, Elsevier, vol. 55(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:41:y:2025:i:4:p:1450-1460. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.