IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v319y2024i3p983-998.html
   My bibliography  Save this article

Interactive preference analysis: A reinforcement learning framework

Author

Listed:
  • Hu, Xiao
  • Kang, Siqin
  • Ren, Long
  • Zhu, Shaokeng

Abstract

Automated investment managers are increasingly popular in personal wealth management due to their cost effectiveness, objectivity, and accessibility. However, it still suffers from several dilemmas, e.g., cold start, over-specialization, and black boxes. To solve these issues, we develop an online reinforcement learning framework based on the multi-armed bandit algorithm to offer personalized investment advice. We provide a comprehensive theoretical procedure for developing this framework. This framework not only enables us to capture the evolving preferences of investors effectively but also has a strong explainability power to provide more implications regarding why one financial product is preferred. We further evaluate our basic model through a large-scale, real-world data set from a leading wealth management platform. The results show a stronger effectiveness of the proposed framework compared to other well-recognized benchmark models. Furthermore, we extend our basic model to address the potential agency problem between the robo-advisor and the investors. Another extension is also provided through an optimization scheme to account for the investors’ demands for diversification in multiple aspects.

Suggested Citation

  • Hu, Xiao & Kang, Siqin & Ren, Long & Zhu, Shaokeng, 2024. "Interactive preference analysis: A reinforcement learning framework," European Journal of Operational Research, Elsevier, vol. 319(3), pages 983-998.
    • Handle: RePEc:eee:ejores:v:319:y:2024:i:3:p:983-998
    DOI: 10.1016/j.ejor.2024.06.033
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221724005125
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2024.06.033?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Xun Li & Zhenyu Wu, 2008. "Reputation entrenchment or risk minimization?: Early stop and investor-manager agency conflict in fund management," Journal of Risk Finance, Emerald Group Publishing, vol. 9(2), pages 125-150, February.
    2. Humoud Alsabah & Agostino Capponi & Octavio Ruiz Lacedelli & Matt Stern, 2021. "Robo-Advising: Learning Investors’ Risk Preferences via Portfolio Choices [Mean-variance versus Full-scale Optimisation: In and out of Sample]," Journal of Financial Econometrics, Oxford University Press, vol. 19(2), pages 369-392.
    3. Wang, Jiang-Jiang & Jing, You-Yin & Zhang, Chun-Fa & Zhao, Jun-Hong, 2009. "Review on multi-criteria decision analysis aid in sustainable energy decision-making," Renewable and Sustainable Energy Reviews, Elsevier, vol. 13(9), pages 2263-2278, December.
    4. Chen, Junlin & Xiong, Jinghong & Chen, Guobao & Liu, Xin & Yan, Peng & Jiang, Hai, 2024. "Optimal instant discounts of multiple ride options at a ride-hailing aggregator," European Journal of Operational Research, Elsevier, vol. 314(2), pages 718-734.
    5. Kallestad, Jakob & Hasibi, Ramin & Hemmati, Ahmad & Sörensen, Kenneth, 2023. "A general deep reinforcement learning hyperheuristic framework for solving combinatorial optimization problems," European Journal of Operational Research, Elsevier, vol. 309(1), pages 446-468.
    6. Philip H. Dybvig & Heber K. Farnsworth & Jennifer N. Carpenter, 2010. "Portfolio Performance and Agency," The Review of Financial Studies, Society for Financial Studies, vol. 23(1), pages 1-23, January.
    7. Agostino Capponi & Sveinn Ólafsson & Thaleia Zariphopoulou, 2022. "Personalized Robo-Advising: Enhancing Investment Through Client Interaction," Management Science, INFORMS, vol. 68(4), pages 2485-2512, April.
    8. William Breffle & Edward Morey & Jennifer Thacher, 2011. "A Joint Latent-Class Model: Combining Likert-Scale Preference Statements With Choice Data to Harvest Preference Heterogeneity," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 50(1), pages 83-110, September.
    9. Kou, Gang & Lin, Changsheng, 2014. "A cosine maximization method for the priority vector derivation in AHP," European Journal of Operational Research, Elsevier, vol. 235(1), pages 225-232.
    10. Scott, Robert C & Horvath, Philip A, 1980. "On the Direction of Preference for Moments of Higher Order Than the Variance," Journal of Finance, American Finance Association, vol. 35(4), pages 915-919, September.
    11. Xiong, Yingqiu & Liu, Yezheng & Qian, Yang & Jiang, Yuanchun & Chai, Yidong & Ling, Haifeng, 2024. "Review-based recommendation under preference uncertainty: An asymmetric deep learning framework," European Journal of Operational Research, Elsevier, vol. 316(3), pages 1044-1057.
    12. Koen W. de Bock & Kristof Coussement & Arno De Caigny & Roman Slowiński & Bart Baesens & Robert N Boute & Tsan-Ming Choi & Dursun Delen & Mathias Kraus & Stefan Lessmann & Sebastián Maldonado & David , 2023. "Explainable AI for Operational Research: A Defining Framework, Methods, Applications, and a Research Agenda," Post-Print hal-04219546, HAL.
    13. Stoughton, Neal M, 1993. "Moral Hazard and the Portfolio Management Problem," Journal of Finance, American Finance Association, vol. 48(5), pages 2009-2028, December.
    14. Nicholas Evangelopoulos & Xiaoni Zhang & Victor R Prybutok, 2012. "Latent Semantic Analysis: five methodological recommendations," European Journal of Information Systems, Taylor & Francis Journals, vol. 21(1), pages 70-86, January.
    15. Jin, Jiahuan & Cui, Tianxiang & Bai, Ruibin & Qu, Rong, 2024. "Container port truck dispatching optimization using Real2Sim based deep reinforcement learning," European Journal of Operational Research, Elsevier, vol. 315(1), pages 161-175.
    16. Fischer, Thomas & Krauss, Christopher, 2018. "Deep learning with long short-term memory networks for financial market predictions," European Journal of Operational Research, Elsevier, vol. 270(2), pages 654-669.
    17. Brenner, Lukas & Meyll, Tobias, 2020. "Robo-advisors: A substitute for human financial advice?," Journal of Behavioral and Experimental Finance, Elsevier, vol. 25(C).
    18. Tongxin Zhou & Yingfei Wang & Lu (Lucy) Yan & Yong Tan, 2023. "Spoiled for Choice? Personalized Recommendation for Healthcare Decisions: A Multiarmed Bandit Approach," Information Systems Research, INFORMS, vol. 34(4), pages 1493-1512, December.
    19. Hui Ou-Yang, 2003. "Optimal Contracts in a Continuous-Time Delegated Portfolio Management Problem," The Review of Financial Studies, Society for Financial Studies, vol. 16(1), pages 173-208.
    20. Francesco D'Acunto & Alberto G. Rossi, 2023. "Robo-Advice: Transforming Households into Rational Economic Agents," Annual Review of Financial Economics, Annual Reviews, vol. 15(1), pages 543-563, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrea M. Buffa & Dimitri Vayanos & Paul Woolley, 2022. "Asset Management Contracts and Equilibrium Prices," Journal of Political Economy, University of Chicago Press, vol. 130(12), pages 3146-3201.
    2. Elena Asparouhova & Peter Bossaerts & Jernej Čopič & Brad Cornell & Jakša Cvitanić & Debrah Meloso, 2015. "Competition in Portfolio Management: Theory and Experiment," Management Science, INFORMS, vol. 61(8), pages 1868-1888, August.
    3. Sotes-Paladino, Juan & Zapatero, Fernando, 2022. "Carrot and stick: A role for benchmark-adjusted compensation in active fund management," Journal of Financial Intermediation, Elsevier, vol. 52(C).
    4. Michael Sockin & Mindy Z Xiaolan, 2023. "Delegated Learning and Contract Commonality in Asset Management," Review of Finance, European Finance Association, vol. 27(6), pages 1931-1975.
    5. Sheng, Jiliang & Wang, Jian & Wang, Xiaoting & Yang, Jun, 2014. "Asymmetric contracts, cash flows and risk taking of mutual funds," Economic Modelling, Elsevier, vol. 38(C), pages 435-442.
    6. Agarwal, Vikas & Gómez, Juan-Pedro & Priestley, Richard, 2012. "Management compensation and market timing under portfolio constraints," Journal of Economic Dynamics and Control, Elsevier, vol. 36(10), pages 1600-1625.
    7. Cardillo, Giovanni & Chiappini, Helen, 2024. "Robo-advisors: A systematic literature review," Finance Research Letters, Elsevier, vol. 62(PA).
    8. Cuoco, Domenico & Kaniel, Ron, 2011. "Equilibrium prices in the presence of delegated portfolio management," Journal of Financial Economics, Elsevier, vol. 101(2), pages 264-296, August.
    9. Markus Ibert & Ron Kaniel & Stijn Van Nieuwerburgh & Roine Vestman, 2018. "Are Mutual Fund Managers Paid for Investment Skill?," The Review of Financial Studies, Society for Financial Studies, vol. 31(2), pages 715-772.
    10. Jakša Cvitanić & Julien Hugonnier, 2022. "Optimal fund menus," Mathematical Finance, Wiley Blackwell, vol. 32(2), pages 455-516, April.
    11. Egil Matsen, 2005. "Portfolio choice when managers control returns," Working Paper 2005/15, Norges Bank.
    12. Luo, Haohan & Liu, Xin & Lv, Xingyang & Hu, Yubei & Ahmad, Ali J., 2024. "Investors’ willingness to use robo-advisors: Extrapolating influencing factors based on the fiduciary duty of investment advisors," International Review of Economics & Finance, Elsevier, vol. 94(C).
    13. He, Zhiguo & Xiong, Wei, 2013. "Delegated asset management, investment mandates, and capital immobility," Journal of Financial Economics, Elsevier, vol. 107(2), pages 239-258.
    14. Han, Min-Yeon & Jun, Sang-Gyung & Oh, Ji Yeol Jimmy & Kang, Hyoung-Goo, 2023. "Who should choose the money managers? Institutional sponsors' equity manager performance," Pacific-Basin Finance Journal, Elsevier, vol. 78(C).
    15. Haoyang Cao & Zhengqi Wu & Renyuan Xu, 2024. "Inference of Utilities and Time Preference in Sequential Decision-Making," Papers 2405.15975, arXiv.org, revised Jun 2024.
    16. Donaldson, Jason Roderick & Piacentino, Giorgia, 2018. "Contracting to compete for flows," Journal of Economic Theory, Elsevier, vol. 173(C), pages 289-319.
    17. Nasini, Stefano & Labbé, Martine & Brotcorne, Luce, 2022. "Multi-market portfolio optimization with conditional value at risk," European Journal of Operational Research, Elsevier, vol. 300(1), pages 350-365.
    18. Guillermo Alonso Alvarez & Sergey Nadtochiy & Kevin Webster, 2022. "Optimal brokerage contracts in Almgren-Chriss model with multiple clients," Papers 2204.05403, arXiv.org.
    19. Abada, Ibrahim & Lambin, Xavier & Tchakarov, Nikolay, 2024. "Collusion by mistake: Does algorithmic sophistication drive supra-competitive profits?," European Journal of Operational Research, Elsevier, vol. 318(3), pages 927-953.
    20. Lagziel, David & Lehrer, Ehud, 2018. "Reward schemes," Games and Economic Behavior, Elsevier, vol. 107(C), pages 21-40.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:319:y:2024:i:3:p:983-998. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.