IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2507.04663.html

Model-Estimation-Free, Dense, and High Dimensional Consistent Precision Matrix Estimators

Author

Listed:
  • Mehmet Caner Agostino Capponi Mihailo Stojnic

Abstract

Precision matrix estimation is a cornerstone concept in statistics, economics, and finance. Despite advances in recent years, estimation methods that are simultaneously (i) dense, (ii) consistent, and (iii) model-free are lacking. While each of these targets can be met separately, achieving them together is challenging.We address this gap by introducing a general class of estimators that unifies these features within a nonasymptotic framework, allowing for explicit characterization of the computational complexity, signal-to-noise ratio trade-off. Our analysis identifies three fundamental random quantities, complexity, signal magnitude, and method bias that jointly determine estimation error. A particularly striking result is that ridgeless regression, a tuning-free special case within our class, exhibits the double descent phenomenon. This establishes the first formal precision matrix analogue to the well-known double descent behavior in linear regression. Our theoretical analysis is supported by a thorough empirical study of the S\&P 500 index, where we observe a doubly ascending Sharpe ratio pattern, which complements the double descent phenomenon.

Suggested Citation

  • Mehmet Caner Agostino Capponi Mihailo Stojnic, 2025. "Model-Estimation-Free, Dense, and High Dimensional Consistent Precision Matrix Estimators," Papers 2507.04663, arXiv.org, revised Dec 2025.
  • Handle: RePEc:arx:papers:2507.04663
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2507.04663
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Olivier Ledoit & Michael Wolf, 2017. "Nonlinear Shrinkage of the Covariance Matrix for Portfolio Selection: Markowitz Meets Goldilocks," The Review of Financial Studies, Society for Financial Studies, vol. 30(12), pages 4349-4388.
    2. Ledoit, Oliver & Wolf, Michael, 2008. "Robust performance hypothesis testing with the Sharpe ratio," Journal of Empirical Finance, Elsevier, vol. 15(5), pages 850-859, December.
    3. Susan Athey & Mohsen Bayati & Nikolay Doudchenko & Guido Imbens & Khashayar Khosravi, 2021. "Matrix Completion Methods for Causal Panel Data Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1716-1730, October.
    4. Cai, Tony & Liu, Weidong, 2011. "Adaptive Thresholding for Sparse Covariance Matrix Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 672-684.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sven Husmann & Antoniya Shivarova & Rick Steinert, 2022. "Sparsity and stability for minimum-variance portfolios," Risk Management, Palgrave Macmillan, vol. 24(3), pages 214-235, September.
    2. Sven Husmann & Antoniya Shivarova & Rick Steinert, 2021. "Cross-validated covariance estimators for high-dimensional minimum-variance portfolios," Financial Markets and Portfolio Management, Springer;Swiss Society for Financial Market Research, vol. 35(3), pages 309-352, September.
    3. Sven Husmann & Antoniya Shivarova & Rick Steinert, 2019. "Sparsity and Stability for Minimum-Variance Portfolios," Papers 1910.11840, arXiv.org.
    4. Caner, Mehmet & Daniele, Maurizio, 2025. "Deep learning based residuals in non-linear factor models: Precision matrix estimation of returns with low signal-to-noise ratio," Journal of Econometrics, Elsevier, vol. 251(C).
    5. Weilong Liu & Yanchu Liu, 2025. "Covariance Matrix Estimation for Positively Correlated Assets," Papers 2507.01545, arXiv.org.
    6. Sven Husmann & Antoniya Shivarova & Rick Steinert, 2019. "Cross-validated covariance estimators for high-dimensional minimum-variance portfolios," Papers 1910.13960, arXiv.org, revised Oct 2020.
    7. Andreas Thomann, 2021. "Multi-asset scenario building for trend-following trading strategies," Annals of Operations Research, Springer, vol. 299(1), pages 293-315, April.
    8. Hiraki, Kazuhiro & Sun, Chuanping, 2022. "A toolkit for exploiting contemporaneous stock correlations," Journal of Empirical Finance, Elsevier, vol. 65(C), pages 99-124.
    9. Santos, André A.P. & Torrent, Hudson S., 2022. "Markowitz meets technical analysis: Building optimal portfolios by exploiting information in trend-following signals," Finance Research Letters, Elsevier, vol. 49(C).
    10. Gianluca De Nard & Olivier Ledoit & Michael Wolf, 2021. "Factor Models for Portfolio Selection in Large Dimensions: The Good, the Better and the Ugly [Using Principal Component Analysis to Estimate a High Dimensional Factor Model with High-frequency Data]," Journal of Financial Econometrics, Oxford University Press, vol. 19(2), pages 236-257.
    11. Chen, Jia & Li, Degui & Linton, Oliver, 2019. "A new semiparametric estimation approach for large dynamic covariance matrices with multiple conditioning variables," Journal of Econometrics, Elsevier, vol. 212(1), pages 155-176.
    12. Lam, Clifford, 2020. "High-dimensional covariance matrix estimation," LSE Research Online Documents on Economics 101667, London School of Economics and Political Science, LSE Library.
    13. Tu, Xueyong & Li, Bin, 2024. "Robust portfolio selection with smart return prediction," Economic Modelling, Elsevier, vol. 135(C).
    14. Cyril Bachelard & Apostolos Chalkis & Vissarion Fisikopoulos & Elias Tsigaridas, 2023. "Randomized geometric tools for anomaly detection in stock markets," Post-Print hal-04223511, HAL.
    15. Long Zhao & Deepayan Chakrabarti & Kumar Muthuraman, 2019. "Portfolio Construction by Mitigating Error Amplification: The Bounded-Noise Portfolio," Operations Research, INFORMS, vol. 67(4), pages 965-983, July.
    16. Thomas Conlon & John Cotter & Iason Kynigakis, 2021. "Machine Learning and Factor-Based Portfolio Optimization," Papers 2107.13866, arXiv.org.
    17. Jungjun Choi & Hyukjun Kwon & Yuan Liao, 2023. "Inference for Low-rank Models without Estimating the Rank," Papers 2311.16440, arXiv.org, revised Oct 2024.
    18. De Nard, Gianluca & Engle, Robert F. & Ledoit, Olivier & Wolf, Michael, 2022. "Large dynamic covariance matrices: Enhancements based on intraday data," Journal of Banking & Finance, Elsevier, vol. 138(C).
    19. Lassance, Nathan & Vanderveken, Rodolphe & Vrins, Frédéric, 2025. "Shrink with Purpose: Optimal Covariance Matrix Estimation for Portfolio Selection," LIDAM Discussion Papers LFIN 2025002, Université catholique de Louvain, Louvain Finance (LFIN).
    20. Gianluca De Nard & Robert F. Engle & Bryan Kelly, 2024. "Factor-Mimicking Portfolios for Climate Risk," Financial Analysts Journal, Taylor & Francis Journals, vol. 80(3), pages 37-58, July.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2507.04663. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.