The role of the isotonizing algorithm in Stein’s covariance matrix estimator

Covariance matrix estimation is central to many applications in statistics and allied fields. A useful estimator in this context was proposed by Stein which regularizes the sample covariance matrix by shrinking its eigenvalues together. This estimator can sometimes yield estimates of the eigenvalues that are negative or differ in order from the observed eigenvalues. In order to rectify this problem, Stein also proposed an ad hoc “isotonizing” procedure which pools together eigenvalue estimates in such a way that the original ordering and positivity of the estimates are enforced. From numerical studies, Stein’s “isotonized” estimator is known to have good risk properties in comparison with the maximum likelihood estimator. However, it remains unclear what role is played by the isotonizing procedure in the remarkable risk reductions achieved by Stein’s estimator. Through two distinct lines of investigations, it is established that Stein’s estimator without the isotonizing algorithm gives only modest risk reductions. In cases where the isotonizing algorithm is frequently used, however, Stein’s estimator can lead to significant risk reductions for certain domains of the parameter. In other cases, Stein’s estimator can even yield risk increases, such as when (1) the theoretical eigenvalues are well separated, and/or (2) when the sample size is moderate to large, leading to over-shrinkage.