IDEAS home Printed from https://ideas.repec.org/a/gam/jstats/v8y2025i3p60-d1700683.html
   My bibliography  Save this article

Some Useful Techniques for High-Dimensional Statistics

Author

Listed:
  • David J. Olive

    (School of Mathematical & Statistical Sciences, Southern Illinois University, Carbondale, IL 62901-4408, USA)

Abstract

High-dimensional statistics are used when n < 5 p , where n is the sample size and p is the number of predictors. Useful techniques include (a) the use of a sparse fitted model, (b) use of principal component analysis for dimension reduction, (c) use of alternative multivariate dispersion estimators instead of the sample covariance matrix, (d) eliminate weak predictors, and (e) stack low-dimensional estimators into a vector. Some variants and theory for these techniques will be given or reviewed.

Suggested Citation

  • David J. Olive, 2025. "Some Useful Techniques for High-Dimensional Statistics," Stats, MDPI, vol. 8(3), pages 1-15, July.
    • Handle: RePEc:gam:jstats:v:8:y:2025:i:3:p:60-:d:1700683
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2571-905X/8/3/60/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2571-905X/8/3/60/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ledoit, Olivier & Wolf, Michael, 2004. "A well-conditioned estimator for large-dimensional covariance matrices," Journal of Multivariate Analysis, Elsevier, vol. 88(2), pages 365-411, February.
    2. Zhihua Su & R. Dennis Cook, 2012. "Inner envelopes: efficient estimation in multivariate linear regression," Biometrika, Biometrika Trust, vol. 99(3), pages 687-702.
    3. Warton, David I., 2008. "Penalized Normal Likelihood and Ridge Regularization of Correlation and Covariance Matrices," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 340-349, March.
    4. Marks R. Nester, 1996. "An Applied Statistician's Creed," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 45(4), pages 401-410, December.
    5. Hyonho Chun & Sündüz Keleş, 2010. "Sparse partial least squares regression for simultaneous dimension reduction and variable selection," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(1), pages 3-25, January.
    6. Kosman W. G. D. H. Rajapaksha & David J. Olive, 2024. "Wald type tests with the wrong dispersion matrix," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 53(6), pages 2236-2251, March.
    7. David J. Olive & Lingling Zhang, 2025. "One component partial least squares, high dimensional regression, data splitting, and the multitude of models," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 54(1), pages 130-145, January.
    8. R. D. Cook & I. S. Helland & Z. Su, 2013. "Envelopes and partial least squares regression," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(5), pages 851-877, November.
    9. Ke-Hai Yuan & Wai Chan, 2011. "Biases and Standard Errors of Standardized Regression Coefficients," Psychometrika, Springer;The Psychometric Society, vol. 76(4), pages 670-690, October.
    10. Andrew Gelman & John Carlin, 2017. "Some Natural Solutions to the -Value Communication Problem—and Why They Won’t Work," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 899-901, July.
    11. Jianqing Fan & Jinchi Lv, 2008. "Sure independence screening for ultrahigh dimensional feature space," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(5), pages 849-911, November.
    12. Hui Zou & Trevor Hastie, 2005. "Addendum: Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(5), pages 768-768, November.
    13. Hui Zou & Trevor Hastie, 2005. "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 67(2), pages 301-320, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lasanthi C. R. Pelawa Watagoda & David J. Olive, 2021. "Comparing six shrinkage estimators with large sample theory and asymptotically optimal prediction intervals," Statistical Papers, Springer, vol. 62(5), pages 2407-2431, October.
    2. Firuz Kamalov & Hana Sulieman & Ayman Alzaatreh & Maher Emarly & Hasna Chamlal & Murodbek Safaraliev, 2025. "Mathematical Methods in Feature Selection: A Review," Mathematics, MDPI, vol. 13(6), pages 1-29, March.
    3. Bousebata, Meryem & Enjolras, Geoffroy & Girard, Stéphane, 2023. "Extreme partial least-squares," Journal of Multivariate Analysis, Elsevier, vol. 194(C).
    4. Bai, Ray & Ghosh, Malay, 2018. "High-dimensional multivariate posterior consistency under global–local shrinkage priors," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 157-170.
    5. Sihan Tu & Zhaoxing Gao, 2025. "A Supervised Screening and Regularized Factor-Based Method for Time Series Forecasting," Papers 2502.15275, arXiv.org.
    6. Christis Katsouris, 2023. "High Dimensional Time Series Regression Models: Applications to Statistical Learning Methods," Papers 2308.16192, arXiv.org.
    7. Candelon, B. & Hurlin, C. & Tokpavi, S., 2012. "Sampling error and double shrinkage estimation of minimum variance portfolios," Journal of Empirical Finance, Elsevier, vol. 19(4), pages 511-527.
    8. Shuichi Kawano, 2014. "Selection of tuning parameters in bridge regression models via Bayesian information criterion," Statistical Papers, Springer, vol. 55(4), pages 1207-1223, November.
    9. Wang, Christina Dan & Chen, Zhao & Lian, Yimin & Chen, Min, 2022. "Asset selection based on high frequency Sharpe ratio," Journal of Econometrics, Elsevier, vol. 227(1), pages 168-188.
    10. Borup, Daniel & Christensen, Bent Jesper & Mühlbach, Nicolaj Søndergaard & Nielsen, Mikkel Slot, 2023. "Targeting predictors in random forest regression," International Journal of Forecasting, Elsevier, vol. 39(2), pages 841-868.
    11. Caroline Jardet & Baptiste Meunier, 2022. "Nowcasting world GDP growth with high‐frequency data," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(6), pages 1181-1200, September.
    12. Peter Bühlmann & Jacopo Mandozzi, 2014. "High-dimensional variable screening and bias in subsequent inference, with an empirical comparison," Computational Statistics, Springer, vol. 29(3), pages 407-430, June.
    13. Kozak, Serhiy & Nagel, Stefan & Santosh, Shrihari, 2020. "Shrinking the cross-section," Journal of Financial Economics, Elsevier, vol. 135(2), pages 271-292.
    14. Kawano, Shuichi & Fujisawa, Hironori & Takada, Toyoyuki & Shiroishi, Toshihiko, 2015. "Sparse principal component regression with adaptive loading," Computational Statistics & Data Analysis, Elsevier, vol. 89(C), pages 192-203.
    15. Loann David Denis Desboulets, 2018. "A Review on Variable Selection in Regression Analysis," Econometrics, MDPI, vol. 6(4), pages 1-27, November.
    16. Lee, Ji Hyung & Shi, Zhentao & Gao, Zhan, 2022. "On LASSO for predictive regression," Journal of Econometrics, Elsevier, vol. 229(2), pages 322-349.
    17. Christidis, Anthony-Alexander & Van Aelst, Stefan & Zamar, Ruben, 2025. "Multi-model subset selection," Computational Statistics & Data Analysis, Elsevier, vol. 203(C).
    18. Ian W. McKeague & Min Qian, 2015. "An Adaptive Resampling Test for Detecting the Presence of Significant Predictors," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1422-1433, December.
    19. Victor Chernozhukov & Christian Hansen & Yuan Liao, 2015. "A lava attack on the recovery of sums of dense and sparse signals," CeMMAP working papers CWP56/15, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    20. Jingxuan Luo & Lili Yue & Gaorong Li, 2023. "Overview of High-Dimensional Measurement Error Regression Models," Mathematics, MDPI, vol. 11(14), pages 1-22, July.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jstats:v:8:y:2025:i:3:p:60-:d:1700683. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.