Why estimation alone causes Markowitz portfolio selection to fail and what we might do about it

Markowitz optimisation is well known to work poorly in practice, but it has not been clear why this happens. We show both theoretically and empirically that Markowitz optimisation is likely to fail badly, even with normally-distributed data, with no time series or correlation effects, and even with shrinkage estimators to reduce estimation risk. A core problem is that very often we cannot confidently distinguish between the mean returns of most assets. We develop a method, based on a sequentially rejective test procedure, to help remedy this problem by identifying subsets of assets indistinguishable in mean or variance. We test our method against naive Markowitz and compare it to other methods, including bootstrap aggregation, proposed to remedy the poor practical performance of Markowitz optimisation. We use out-of-sample and bootstrap tests on data from several market indices and hedge funds. We find our method is more robust than naive Markowitz and outperforms equally weighted portfolios but bootstrap aggregation works, as expected, better when we cannot distinguish among means. We also find evidence that covariance shrinkage improves performance.