Noise-robust orthogonal clustering and applications to equity markets

The financial sector relies increasingly on advanced statistical techniques to extract meaningful signals from large and noisy datasets. Traditional classification frameworks offer an initial structure for analysing and grouping financial assets based on their primary business activities, focusing on sector-specific risk factors. However, they can fail to capture dynamic relationships, leaving practitioners with incomplete perspectives of market co-movements. In this paper, we apply clustering algorithms to the return time series of S&P 500 constituents to derive stock groupings that are temporally adaptive, statistically non-random, and explicitly deviating from traditional sector classifications. Our hypothesis is that data-driven methods better highlight true underlying structures in financial markets, thereby improving diversification and facilitating the discovery of new arbitrage opportunities. We address key challenges such as the absence of ground truth, clustering instability, noise, and hyperparameter selection. Our approach first removes GICS-driven components through orthogonal projections to ensure that the resulting representation deviates from the predefined sector structure. Then, we enhance noise robustness and stability using PCA-based eigenvector cleaning and ensemble clustering. Empirical evaluations using market data demonstrate that clustering can outperform conventional classification systems in detecting relevant patterns and mitigating artificially inflated correlations. We demonstrate the economic benefits, both in terms of Sharpe ratio and P&L, of our proposed methodology when incorporating it into a trading strategy. We further analyse the residual P&L relative to alternative clustering schemes to quantify the additional alpha captured by our approach. These findings underscore the potential for noise-robust, data-driven approaches to rethink portfolio construction and improve the efficiency of algorithmic trading strategies.