IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v62y2021i4d10.1007_s00362-019-01148-1.html
   My bibliography  Save this article

Multivariate outlier detection based on a robust Mahalanobis distance with shrinkage estimators

Author

Listed:
  • Elisa Cabana

    (University Carlos III of Madrid)

  • Rosa E. Lillo

    (University Carlos III of Madrid)

  • Henry Laniado

    (University EAFIT)

Abstract

A collection of robust Mahalanobis distances for multivariate outlier detection is proposed, based on the notion of shrinkage. Robust intensity and scaling factors are optimally estimated to define the shrinkage. Some properties are investigated, such as affine equivariance and breakdown value. The performance of the proposal is illustrated through the comparison to other techniques from the literature, in a simulation study and with a real dataset. The behavior when the underlying distribution is heavy-tailed or skewed, shows the appropriateness of the method when we deviate from the common assumption of normality. The resulting high true positive rates and low false positive rates in the vast majority of cases, as well as the significantly smaller computation time show the advantages of our proposal.

Suggested Citation

  • Elisa Cabana & Rosa E. Lillo & Henry Laniado, 2021. "Multivariate outlier detection based on a robust Mahalanobis distance with shrinkage estimators," Statistical Papers, Springer, vol. 62(4), pages 1583-1609, August.
    • Handle: RePEc:spr:stpapr:v:62:y:2021:i:4:d:10.1007_s00362-019-01148-1
    DOI: 10.1007/s00362-019-01148-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-019-01148-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00362-019-01148-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. DeMiguel, Victor & Martin-Utrera, Alberto & Nogales, Francisco J., 2013. "Size matters: Optimal calibration of shrinkage estimators for portfolio selection," Journal of Banking & Finance, Elsevier, vol. 37(8), pages 3018-3034.
    2. Ledoit, Olivier & Wolf, Michael, 2004. "A well-conditioned estimator for large-dimensional covariance matrices," Journal of Multivariate Analysis, Elsevier, vol. 88(2), pages 365-411, February.
    3. Davy Paindaveine & Germain Van bever, 2013. "From Depth to Local Depth: A Focus on Centrality," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(503), pages 1105-1119, September.
    4. Arup Bose, 1995. "Estimating the asymptotic dispersion of theL 1 median," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 47(2), pages 267-271, June.
    5. Chen, Song Xi & Qin, Yingli, 2010. "A Two Sample Test for High Dimensional Data with Applications to Gene-set Testing," MPRA Paper 59642, University Library of Munich, Germany.
    6. Ansgar Steland, 2018. "Shrinkage for covariance estimation: asymptotics, confidence intervals, bounds and applications in sensor monitoring and finance," Statistical Papers, Springer, vol. 59(4), pages 1441-1462, December.
    7. Couillet, Romain & McKay, Matthew, 2014. "Large dimensional analysis and optimization of robust shrinkage covariance matrix estimators," Journal of Multivariate Analysis, Elsevier, vol. 131(C), pages 99-120.
    8. Michael Falk, 1997. "On Mad and Comedians," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 49(4), pages 615-644, December.
    9. Arup Bose & Probal Chaudhuri, 1993. "On the dispersion of multivariate median," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 45(3), pages 541-550, September.
    10. Dodge, Yadolah, 1987. "An introduction to L1-norm based statistical data analysis," Computational Statistics & Data Analysis, Elsevier, vol. 5(4), pages 239-253, September.
    11. Tarr, G. & Müller, S. & Weber, N.C., 2016. "Robust estimation of precision matrices under cellwise contamination," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 404-420.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Laifa Tao & Haifei Liu & Jiqing Zhang & Xuanyuan Su & Shangyu Li & Jie Hao & Chen Lu & Mingliang Suo & Chao Wang, 2022. "Associated Fault Diagnosis of Power Supply Systems Based on Graph Matching: A Knowledge and Data Fusion Approach," Mathematics, MDPI, vol. 10(22), pages 1-28, November.
    2. Brenton R. Clarke & Andrew Grose, 2023. "A further study comparing forward search multivariate outlier methods including ATLA with an application to clustering," Statistical Papers, Springer, vol. 64(2), pages 395-420, April.
    3. Moezza Nabeel & Sajid Ali & Ismail Shah & Mohammed M. A. Almazah & Fuad S. Al-Duais, 2023. "Robust Surveillance Schemes Based on Proportional Hazard Model for Monitoring Reliability Data," Mathematics, MDPI, vol. 11(11), pages 1-21, May.
    4. Guo, Peng & Gan, Yu & Infield, David, 2022. "Wind turbine performance degradation monitoring using DPGMM and Mahalanobis distance," Renewable Energy, Elsevier, vol. 200(C), pages 1-9.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cabana Garceran del Vall, Elisa & Laniado Rodas, Henry & Lillo Rodríguez, Rosa Elvira, 2017. "Multivariate outlier detection based on a robust Mahalanobis distance with shrinkage estimators," DES - Working Papers. Statistics and Econometrics. WS 24613, Universidad Carlos III de Madrid. Departamento de Estadística.
    2. Jan Kalina & Jan Tichavský, 2022. "The minimum weighted covariance determinant estimator for high-dimensional data," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 16(4), pages 977-999, December.
    3. Ding, Wenliang & Shu, Lianjie & Gu, Xinhua, 2023. "A robust Glasso approach to portfolio selection in high dimensions," Journal of Empirical Finance, Elsevier, vol. 70(C), pages 22-37.
    4. Li, Weiming & Xu, Yangchang, 2022. "Asymptotic properties of high-dimensional spatial median in elliptical distributions with application," Journal of Multivariate Analysis, Elsevier, vol. 190(C).
    5. Füss, Roland & Miebs, Felix & Trübenbach, Fabian, 2014. "A jackknife-type estimator for portfolio revision," Journal of Banking & Finance, Elsevier, vol. 43(C), pages 14-28.
    6. Couillet, Romain & Kammoun, Abla & Pascal, Frédéric, 2016. "Second order statistics of robust estimators of scatter. Application to GLRT detection for elliptical signals," Journal of Multivariate Analysis, Elsevier, vol. 143(C), pages 249-274.
    7. Lassance, Nathan & Vanderveken, Rodolphe & Vrins, Frédéric, 2022. "On the optimal combination of naive and mean-variance portfolio strategies," LIDAM Discussion Papers LFIN 2022006, Université catholique de Louvain, Louvain Finance (LFIN).
    8. Taras Bodnar & Nestor Parolya & Erik Thors'en, 2022. "Two is better than one: Regularized shrinkage of large minimum variance portfolio," Papers 2202.06666, arXiv.org.
    9. Liusha Yang & Matthew R. Mckay & Romain Couillet, 2018. "High-Dimensional MVDR Beamforming: Optimized Solutions Based on Spiked Random Matrix Models," Post-Print hal-01957672, HAL.
    10. Lassance, Nathan, 2021. "Maximizing the Out-of-Sample Sharpe Ratio," LIDAM Discussion Papers LFIN 2021013, Université catholique de Louvain, Louvain Finance (LFIN).
    11. Olivier Ledoit & Michael Wolf, 2014. "Nonlinear shrinkage of the covariance matrix for portfolio selection: Markowitz meets Goldilocks," ECON - Working Papers 137, Department of Economics - University of Zurich, revised Feb 2017.
    12. Kircher, Felix & Rösch, Daniel, 2021. "A shrinkage approach for Sharpe ratio optimal portfolios with estimation risks," Journal of Banking & Finance, Elsevier, vol. 133(C).
    13. Chen, Songxi, 2012. "Two Sample Tests for High Dimensional Covariance Matrices," MPRA Paper 46026, University Library of Munich, Germany.
    14. Yuanrong Wang & Tomaso Aste, 2022. "Sparsification and Filtering for Spatial-temporal GNN in Multivariate Time-series," Papers 2203.03991, arXiv.org.
    15. Chakrabarti, Deepayan, 2021. "Parameter-free robust optimization for the maximum-Sharpe portfolio problem," European Journal of Operational Research, Elsevier, vol. 293(1), pages 388-399.
    16. Benoit Oriol & Alexandre Miot, 2023. "Ledoit-Wolf linear shrinkage with unknown mean," Papers 2304.07045, arXiv.org.
    17. Abadir, Karim M. & Distaso, Walter & Žikeš, Filip, 2014. "Design-free estimation of variance matrices," Journal of Econometrics, Elsevier, vol. 181(2), pages 165-180.
    18. Touloumis, Anestis, 2015. "Nonparametric Stein-type shrinkage covariance matrix estimators in high-dimensional settings," Computational Statistics & Data Analysis, Elsevier, vol. 83(C), pages 251-261.
    19. Miguel, Victor de & Martín Utrera, Alberto & Nogales, Francisco J., 2013. "Parameter uncertainty in multiperiod portfolio optimization with transaction costs," DES - Working Papers. Statistics and Econometrics. WS ws132119, Universidad Carlos III de Madrid. Departamento de Estadística.
    20. Hannart, Alexis & Naveau, Philippe, 2014. "Estimating high dimensional covariance matrices: A new look at the Gaussian conjugate framework," Journal of Multivariate Analysis, Elsevier, vol. 131(C), pages 149-162.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:62:y:2021:i:4:d:10.1007_s00362-019-01148-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.