Competitive optimal portfolio selection under mean-variance criterion

We investigate a portfolio selection problem involving multi competitive agents, each exhibiting mean-variance preferences. Unlike classical models, each agent's utility is determined by their relative wealth compared to the average wealth of all agents, introducing a competitive dynamic into the optimization framework. To address this game-theoretic problem, we first reformulate the mean-variance criterion as a constrained, non-homogeneous stochastic linear-quadratic control problem and derive the corresponding optimal feedback strategies. The existence of Nash equilibria is shown to depend on the well-posedness of a complex, coupled system of equations. Employing decoupling techniques, we reduce the well-posedness analysis to the solvability of a novel class of multi-dimensional linear backward stochastic differential equations (BSDEs). We solve a new type of nonlinear BSDEs (including the above linear one as a special case) using fixed-point theory. Depending on the interplay between market and competition parameters, three distinct scenarios arise: (i) the existence of a unique Nash equilibrium, (ii) the absence of any Nash equilibrium, and (iii) the existence of infinitely many Nash equilibria. These scenarios are rigorously characterized and discussed in detail.