My bibliography  Save this paper

# The best estimation for high-dimensional Markowitz mean-variance optimization

## Author

Listed:
• Bai, Zhidong
• Li, Hua
• Wong, Wing-Keung

## Abstract

The traditional(plug-in) return for the Markowitz mean-variance (MV) optimization has been demonstrated to seriously overestimate the theoretical optimal return, especially when the dimension to sample size ratio $p/n$ is large. The newly developed bootstrap-corrected estimator corrects the overestimation, but it incurs the "under-prediction problem," it does not do well on the estimation of the corresponding allocation, and it has bigger risk. To circumvent these limitations and to improve the optimal return estimation further, this paper develops the theory of spectral-corrected estimation. We first establish a theorem to explain why the plug-in return greatly overestimates the theoretical optimal return. We prove that under some situations the plug-in return is $\sqrt{\gamma}\$\ times bigger than the theoretical optimal return, while under other situations, the plug-in return is bigger than but may not be $\sqrt{\gamma}\$\ times larger than its theoretic counterpart where $\gamma = \frac 1{1-y}$ with $y$ being the limit of the ratio $p/n$. Thereafter, we develop the spectral-corrected estimation for the Markowitz MV model which performs much better than both the plug-in estimation and the bootstrap-corrected estimation not only in terms of the return but also in terms of the allocation and the risk. We further develop properties for our proposed estimation and conduct a simulation to examine the performance of our proposed estimation. Our simulation shows that our proposed estimation not only overcomes the problem of "over-prediction," but also circumvents the "under-prediction," "allocation estimation," and "risk" problems. Our simulation also shows that our proposed spectral-corrected estimation is stable for different values of sample size $n$, dimension $p$, and their ratio $p/n$. In addition, we relax the normality assumption in our proposed estimation so that our proposed spectral-corrected estimators could be obtained when the returns of the assets being studied could follow any distribution under the condition of the existence of the fourth moments.

## Suggested Citation

• Bai, Zhidong & Li, Hua & Wong, Wing-Keung, 2013. "The best estimation for high-dimensional Markowitz mean-variance optimization," MPRA Paper 43862, University Library of Munich, Germany.
• Handle: RePEc:pra:mprapa:43862
as

File URL: https://mpra.ub.uni-muenchen.de/43862/1/MPRA_paper_43862.pdf
File Function: original version

## References listed on IDEAS

as
1. Ledoit, Olivier & Wolf, Michael, 2004. "A well-conditioned estimator for large-dimensional covariance matrices," Journal of Multivariate Analysis, Elsevier, vol. 88(2), pages 365-411, February.
2. Jansen, Dennis W. & Li, Qi & Wang, Zijun & Yang, Jian, 2008. "Fiscal policy and asset markets: A semiparametric analysis," Journal of Econometrics, Elsevier, vol. 147(1), pages 141-150, November.
3. Fu, Chenpeng & Lari-Lavassani, Ali & Li, Xun, 2010. "Dynamic mean-variance portfolio selection with borrowing constraint," European Journal of Operational Research, Elsevier, vol. 200(1), pages 312-319, January.
4. Andrew W. Lo, A. Craig MacKinlay, 1988. "Stock Market Prices do not Follow Random Walks: Evidence from a Simple Specification Test," Review of Financial Studies, Society for Financial Studies, vol. 1(1), pages 41-66.
5. Elton, Edwin J & Gruber, Martin J & Padberg, Manfred W, 1976. "Simple Criteria for Optimal Portfolio Selection," Journal of Finance, American Finance Association, vol. 31(5), pages 1341-1357, December.
6. Ross, Stephen A., 1976. "The arbitrage theory of capital asset pricing," Journal of Economic Theory, Elsevier, vol. 13(3), pages 341-360, December.
7. Zymler, Steve & Rustem, Berç & Kuhn, Daniel, 2011. "Robust portfolio optimization with derivative insurance guarantees," European Journal of Operational Research, Elsevier, vol. 210(2), pages 410-424, April.
8. Zhidong Bai & Yongchang Hui & Wing-Keung Wong & Ričardas Zitikis, 2012. "Prospect Performance Evaluation: Making a Case for a Non-asymptotic UMPU Test," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 10(4), pages 703-732, September.
9. Clark, Ephraim & Jokung, Octave & Kassimatis, Konstantinos, 2011. "Making inefficient market indices efficient," European Journal of Operational Research, Elsevier, vol. 209(1), pages 83-93, February.
10. Soyer, Refik & Tanyeri, Kadir, 2006. "Bayesian portfolio selection with multi-variate random variance models," European Journal of Operational Research, Elsevier, vol. 171(3), pages 977-990, June.
11. Blattberg, Robert C & Gonedes, Nicholas J, 1974. "A Comparison of the Stable and Student Distributions as Statistical Models for Stock Prices," The Journal of Business, University of Chicago Press, vol. 47(2), pages 244-280, April.
12. Sharpe, William F., 1971. "A Linear Programming Approximation for the General Portfolio Analysis Problem," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 6(05), pages 1263-1275, December.
13. Yu, Mei & Takahashi, Satoru & Inoue, Hiroshi & Wang, Shouyang, 2010. "Dynamic portfolio optimization with risk control for absolute deviation model," European Journal of Operational Research, Elsevier, vol. 201(2), pages 349-364, March.
14. Elton, Edwin J & Gruber, Martin J & Padberg, Manfred W, 1978. "Simple Criteria for Optimal Portfolio Selection: Tracing out the Efficient Frontier," Journal of Finance, American Finance Association, vol. 33(1), pages 296-302, March.
15. Kan, Raymond & Zhou, Guofu, 2007. "Optimal Portfolio Choice with Parameter Uncertainty," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 42(03), pages 621-656, September.
16. Markowitz, Harry M & Perold, Andre F, 1981. "Portfolio Analysis with Factors and Scenarios," Journal of Finance, American Finance Association, vol. 36(4), pages 871-877, September.
17. Simone Manganelli, 2004. "Asset Allocation by Variance Sensitivity Analysis," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 2(3), pages 370-389.
18. Wing-Keung Wong & Chenghu Ma, 2005. "Preferences over Meyer’s Location-Scale Family," Departmental Working Papers wp0506, National University of Singapore, Department of Economics.
19. Chan, Chia-Ying & de Peretti, Christian & Qiao, Zhuo & Wong, Wing-Keung, 2012. "Empirical test of the efficiency of the UK covered warrants market: Stochastic dominance and likelihood ratio test approach," Journal of Empirical Finance, Elsevier, vol. 19(1), pages 162-174.
20. Zhao, Yonggan & Ziemba, William T., 2008. "Calculating risk neutral probabilities and optimal portfolio policies in a dynamic investment model with downside risk control," European Journal of Operational Research, Elsevier, vol. 185(3), pages 1525-1540, March.
21. Bai, Zhidong & Wang, Keyan & Wong, Wing-Keung, 2011. "The mean-variance ratio test--A complement to the coefficient of variation test and the Sharpe ratio test," Statistics & Probability Letters, Elsevier, vol. 81(8), pages 1078-1085, August.
22. Wai Fong & Wing Wong, 2006. "The modified mixture of distributions model: a revisit," Annals of Finance, Springer, vol. 2(2), pages 167-178, March.
23. Zhidong Bai & Hua Li & Huixia Liu & Wing‐Keung Wong, 2011. "Test statistics for prospect and Markowitz stochastic dominances with applications," Econometrics Journal, Royal Economic Society, vol. 14(2), pages 278-303, July.
24. Chan, Joshua C.C. & Kroese, Dirk P., 2010. "Efficient estimation of large portfolio loss probabilities in t-copula models," European Journal of Operational Research, Elsevier, vol. 205(2), pages 361-367, September.
25. Egozcue, Martín & García, Luis Fuentes & Wong, Wing-Keung & Zitikis, Ricardas, 2011. "Do investors like to diversify? A study of Markowitz preferences," European Journal of Operational Research, Elsevier, vol. 215(1), pages 188-193, November.
26. Dominic Gasbarro & Wing-Keung Wong & J. Kenton Zumwalt, 2007. "Stochastic Dominance Analysis of iShares," The European Journal of Finance, Taylor & Francis Journals, vol. 13(1), pages 89-101.
27. Jorion, Philippe, 1985. "International Portfolio Diversification with Estimation Risk," The Journal of Business, University of Chicago Press, vol. 58(3), pages 259-278, July.
28. Luciano, Elisa & Peccati, Lorenzo & Cifarelli, Donato M., 2003. "VaR as a risk measure for multiperiod static inventory models," International Journal of Production Economics, Elsevier, vol. 81(1), pages 375-384, January.
29. Harry Markowitz, 1952. "Portfolio Selection," Journal of Finance, American Finance Association, vol. 7(1), pages 77-91, March.
30. Andre F. Perold, 1984. "Large-Scale Portfolio Optimization," Management Science, INFORMS, vol. 30(10), pages 1143-1160, October.
31. Hiroshi Konno & Hiroaki Yamazaki, 1991. "Mean-Absolute Deviation Portfolio Optimization Model and Its Applications to Tokyo Stock Market," Management Science, INFORMS, vol. 37(5), pages 519-531, May.
32. Bob Korkie & Harry J. Turtle, 2002. "A Mean-Variance Analysis of Self-Financing Portfolios," Management Science, INFORMS, vol. 48(3), pages 427-443, March.
33. Silverstein, J. W. & Choi, S. I., 1995. "Analysis of the Limiting Spectral Distribution of Large Dimensional Random Matrices," Journal of Multivariate Analysis, Elsevier, vol. 54(2), pages 295-309, August.
34. William F. Sharpe, 1964. "Capital Asset Prices: A Theory Of Market Equilibrium Under Conditions Of Risk," Journal of Finance, American Finance Association, vol. 19(3), pages 425-442, September.
35. Eugene F. Fama, 1965. "Portfolio Analysis in a Stable Paretian Market," Management Science, INFORMS, vol. 11(3), pages 404-419, January.
36. Egozcue, Martin & Wong, Wing-Keung, 2010. "Gains from diversification on convex combinations: A majorization and stochastic dominance approach," European Journal of Operational Research, Elsevier, vol. 200(3), pages 893-900, February.
37. Frederick Wong, 2003. "Efficient estimation of covariance selection models," Biometrika, Biometrika Trust, vol. 90(4), pages 809-830, December.
38. Wong, Wing-Keung, 2007. "Stochastic dominance and mean-variance measures of profit and loss for business planning and investment," European Journal of Operational Research, Elsevier, vol. 182(2), pages 829-843, October.
39. Clark, Peter K, 1973. "A Subordinated Stochastic Process Model with Finite Variance for Speculative Prices," Econometrica, Econometric Society, vol. 41(1), pages 135-155, January.
40. Wong, Wing-Keung & Phoon, Kok Fai & Lean, Hooi Hooi, 2008. "Stochastic dominance analysis of Asian hedge funds," Pacific-Basin Finance Journal, Elsevier, vol. 16(3), pages 204-223, June.
41. Ma, Chenghu & Wong, Wing-Keung, 2010. "Stochastic dominance and risk measure: A decision-theoretic foundation for VaR and C-VaR," European Journal of Operational Research, Elsevier, vol. 207(2), pages 927-935, December.
42. Lam, Kin & Liu, Taisheng & Wong, Wing-Keung, 2010. "A pseudo-Bayesian model in financial decision making with implications to market volatility, under- and overreaction," European Journal of Operational Research, Elsevier, vol. 203(1), pages 166-175, May.
43. Leung, Pui-Lam & Ng, Hon-Yip & Wong, Wing-Keung, 2012. "An improved estimation to make Markowitz’s portfolio optimization theory users friendly and estimation accurate with application on the US stock market investment," European Journal of Operational Research, Elsevier, vol. 222(1), pages 85-95.
44. William F. Sharpe, 1967. "A Linear Programming Algorithm for Mutual Fund Portfolio Selection," Management Science, INFORMS, vol. 13(7), pages 499-510, March.
45. Yusif Simaan, 1997. "Estimation Risk in Portfolio Selection: The Mean Variance Model Versus the Mean Absolute Deviation Model," Management Science, INFORMS, vol. 43(10), pages 1437-1446, October.
46. Kroll, Yoram & Levy, Haim & Markowitz, Harry M, 1984. " Mean-Variance versus Direct Utility Maximization," Journal of Finance, American Finance Association, vol. 39(1), pages 47-61, March.
47. Eugene F. Fama, 1963. "Mandelbrot and the Stable Paretian Hypothesis," The Journal of Business, University of Chicago Press, vol. 36, pages 420-420.
48. Markowitz, Harry M, 1991. " Foundations of Portfolio Theory," Journal of Finance, American Finance Association, vol. 46(2), pages 469-477, June.
Full references (including those not matched with items on IDEAS)

## Citations

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as

Cited by:

1. Chang, C-L. & McAleer, M.J. & Wong, W.-K., 2015. "Informatics, Data Mining, Econometrics and Financial Economics: A Connection," Econometric Institute Research Papers EI2015-34, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
2. Bai, Zhidong & Liu, Huixia & Wong, Wing-Keung, 2016. "Making Markowitz's Portfolio Optimization Theory Practically Useful," MPRA Paper 74360, University Library of Munich, Germany.

### Keywords

Markowitz mean-variance optimization; Optimal Return; Optimal Portfolio Allocation; Large Random Matrix; Bootstrap Method;

### JEL classification:

• G11 - Financial Economics - - General Financial Markets - - - Portfolio Choice; Investment Decisions
• C3 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables

### NEP fields

This paper has been announced in the following NEP Reports:

## Corrections

All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:43862. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Joachim Winter) or (Rebekah McClure). General contact details of provider: http://edirc.repec.org/data/vfmunde.html .

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.