Optimizing portfolios with surrender variable annuities: A deep reinforcement learning approach

This paper investigates a portfolio optimization problem for an investor on asset allocation among risk-free asset, risky asset, and surrender variable annuity contracts featuring guaranteed minimum death benefit and guaranteed minimum maturity benefit subject to mortality and surrender risk. The investor’s objective is to maximize the expected utility of the bequest at death or the expected utility of assets at contract maturity. On each trading day before the investor’s death, the investor can adjust the allocation between risk-free and risky assets, invest in a new surrender variable annuity product. Especially, the policyholder may exercise partial or full surrender options for any existing variable annuity contract. This dynamic adjustment creates a high-dimensional state and action space, making traditional optimization methods inadequate. To address this, we utilize the Lee-Carter model to analyze Australian demographic data, predict mortality risk, simulate surrender risk based on market changes, and estimate the fair pricing of variable annuity contracts in the portfolio. Subsequently, we introduce a deep reinforcement learning algorithm within a simulated trading environment that independently models the dynamic behavior of various assets and underlying indices. The algorithm utilizes neural networks to analyze high-dimensional state variables and leverages the interactive capabilities of the agent to flexibly adapt to asset fluctuations, dynamically optimizing investment allocation. Additionally, we prove the global convergence of the algorithm under standard assumptions and validate its effectiveness in managing the complexities of high-dimensional portfolios, particularly in capturing mortality, surrender, and financial risks. Numerical experiments further demonstrate the stability and robustness of the algorithm, showcasing its advantages in complex insurance and financial scenarios.