High-Dimensional Weighted K-Means with Serial Dependence

In this paper, we propose a new K-means approach for high-dimensional panel data with unknown group memberships. We highlight that the standard K-means algorithm using Euclidean distance can su¤er from misclassi cation in nite samples due to serial correlation and heteroskedasticity in the panel data. Our proposed weighted K-means algorithm addresses this issue by weighting the Euclidean distance using the full covari-ance structure of idiosyncratic shocks. Assuming that both the cross-sectional and time dimensions of the panel grow large, we develop an asymptotic theory for the weighted K-means algorithm that establishes the consistency of the estimated group centroids and the oracle property for group membership estimation. For practical implemen-tation, we propose a feasible weighted K-means method that employs a regularized estimation of the high-dimensional covariance matrix in the K-means objective func-tion. Monte Carlo simulation results demonstrate the e¤ectiveness of our weighted K-means algorithm in estimating grouped xed-e¤ects models for large panels, partic-ularly when strong serial dependencies exist in both group-level trends and idiosyncratic components.