Det-MASO: Diversity-Regularized Multi-Agent Sequential Optimization for Factor Generation

Quantitative investment requires alpha factors that are both predictive and diverse. However, existing factor generation methods, including formulaic mining and neural end-to-end models, are mostly based on a single-agent design, which limits exploration of the factor space and reduces adaptability to changing markets. Multi-agent approaches can broaden exploration, but agents trained under a shared objective often collapse to similar outputs, while existing diversity techniques either scale poorly with asset-universe size or address redundancy only after training. We propose a multi-agent sequential optimization (Det-MASO) framework for generating diversified, context-adaptive alpha factors. It introduces a differentiable log-determinant penalty on the inter-agent correlation matrix directly into the joint objective, discouraging correlated agent behavior during training with computational cost that scales linearly with the number of assets. We further show that this penalty prevents homogeneous equilibria and preserves factor diversity throughout optimization, while a context-conditioned scheduler dynamically adjusts agent emphasis across market regimes and Shapley interaction index analysis is used to characterize the cooperation structure that emerges among them. Experiments show that Det-MASO outperforms baselines, improving the information coefficient by 1.1\%, the annualized return by 6.4\%, and the Sharpe ratio by 0.33.