IDEAS home Printed from https://ideas.repec.org/a/gam/jstats/v4y2021i4p57-1011d686947.html
   My bibliography  Save this article

A Geometric Perspective on Functional Outlier Detection

Author

Listed:
  • Moritz Herrmann

    (Department of Statistics, Ludwig-Maximilians-University, Ludwigstr. 33, 80539 Munich, Germany)

  • Fabian Scheipl

    (Department of Statistics, Ludwig-Maximilians-University, Ludwigstr. 33, 80539 Munich, Germany)

Abstract

We consider functional outlier detection from a geometric perspective, specifically: for functional datasets drawn from a functional manifold, which is defined by the data’s modes of variation in shape, translation, and phase. Based on this manifold, we developed a conceptualization of functional outlier detection that is more widely applicable and realistic than previously proposed taxonomies. Our theoretical and experimental analyses demonstrated several important advantages of this perspective: it considerably improves theoretical understanding and allows describing and analyzing complex functional outlier scenarios consistently and in full generality, by differentiating between structurally anomalous outlier data that are off-manifold and distributionally outlying data that are on-manifold, but at its margins. This improves the practical feasibility of functional outlier detection: we show that simple manifold-learning methods can be used to reliably infer and visualize the geometric structure of functional datasets. We also show that standard outlier-detection methods requiring tabular data inputs can be applied to functional data very successfully by simply using their vector-valued representations learned from manifold learning methods as the input features. Our experiments on synthetic and real datasets demonstrated that this approach leads to outlier detection performances at least on par with existing functional-data-specific methods in a large variety of settings, without the highly specialized, complex methodology and narrow domain of application these methods often entail.

Suggested Citation

  • Moritz Herrmann & Fabian Scheipl, 2021. "A Geometric Perspective on Functional Outlier Detection," Stats, MDPI, vol. 4(4), pages 1-41, November.
  • Handle: RePEc:gam:jstats:v:4:y:2021:i:4:p:57-1011:d:686947
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2571-905X/4/4/57/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2571-905X/4/4/57/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Febrero-Bande, Manuel & de la Fuente, Manuel Oviedo, 2012. "Statistical Computing in Functional Data Analysis: The R Package fda.usc," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 51(i04).
    2. Mia Hubert & Peter Rousseeuw & Pieter Segaert, 2015. "Multivariate functional outlier detection," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 177-202, July.
    3. Weiyi Xie & Sebastian Kurtek & Karthik Bharath & Ying Sun, 2017. "A Geometric Approach to Visualization of Variability in Functional Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 979-993, July.
    4. Polonik, Wolfgang, 1997. "Minimum volume sets and generalized quantile processes," Stochastic Processes and their Applications, Elsevier, vol. 69(1), pages 1-24, July.
    5. Pallavi Sawant & Nedret Billor & Hyejin Shin, 2012. "Functional outlier detection with robust functional principal component analysis," Computational Statistics, Springer, vol. 27(1), pages 83-102, March.
    6. Ana Arribas-Gil & Juan Romo, 2015. "Discussion of “Multivariate functional outlier detection”," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 263-267, July.
    7. Guillermo Vinue & Irene Epifanio, 2021. "Robust archetypoids for anomaly detection in big functional data," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 15(2), pages 437-462, June.
    8. Mia Hubert & Peter Rousseeuw & Pieter Segaert, 2015. "Rejoinder to ‘multivariate functional outlier detection’," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 269-277, July.
    9. Dai, Wenlin & Genton, Marc G., 2019. "Directional outlyingness for multivariate functional data," Computational Statistics & Data Analysis, Elsevier, vol. 131(C), pages 50-65.
    10. Dai, Wenlin & Mrkvička, Tomáš & Sun, Ying & Genton, Marc G., 2020. "Functional outlier detection and taxonomy by sequential transformations," Computational Statistics & Data Analysis, Elsevier, vol. 149(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ojo, Oluwasegun Taiwo & Fernández Anta, Antonio & Genton, Marc G. & Lillo Rodríguez, Rosa Elvira, 2022. "Multivariate Functional Outlier Detection using the FastMUOD Indices," DES - Working Papers. Statistics and Econometrics. WS 35665, Universidad Carlos III de Madrid. Departamento de Estadística.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Boente, Graciela & Parada, Daniela, 2023. "Robust estimation for functional quadratic regression models," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).
    2. Dai, Wenlin & Mrkvička, Tomáš & Sun, Ying & Genton, Marc G., 2020. "Functional outlier detection and taxonomy by sequential transformations," Computational Statistics & Data Analysis, Elsevier, vol. 149(C).
    3. Oluwasegun Taiwo Ojo & Antonio Fernández Anta & Rosa E. Lillo & Carlo Sguera, 2022. "Detecting and classifying outliers in big functional data," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 16(3), pages 725-760, September.
    4. Jorge R. Sosa Donoso & Miguel Flores & Salvador Naya & Javier Tarrío-Saavedra, 2023. "Local Correlation Integral Approach for Anomaly Detection Using Functional Data," Mathematics, MDPI, vol. 11(4), pages 1-18, February.
    5. Ojo, Oluwasegun Taiwo & Fernández Anta, Antonio & Genton, Marc G. & Lillo Rodríguez, Rosa Elvira, 2022. "Multivariate Functional Outlier Detection using the FastMUOD Indices," DES - Working Papers. Statistics and Econometrics. WS 35665, Universidad Carlos III de Madrid. Departamento de Estadística.
    6. Archimbaud, Aurore & Boulfani, Fériel & Gendre, Xavier & Nordhausen, Klaus & Ruiz-Gazen, Anne & Virta, Joni, 2021. "ICS for multivariate functional anomaly detection with applications to predictive maintenance and quality control," TSE Working Papers 21-1182, Toulouse School of Economics (TSE), revised Mar 2022.
    7. Martínez-Hernández, Israel & Genton, Marc G. & González-Farías, Graciela, 2019. "Robust depth-based estimation of the functional autoregressive model," Computational Statistics & Data Analysis, Elsevier, vol. 131(C), pages 66-79.
    8. Graciela Boente & Matías Salibián-Barrera, 2021. "Robust functional principal components for sparse longitudinal data," METRON, Springer;Sapienza Università di Roma, vol. 79(2), pages 159-188, August.
    9. Guillermo Vinue & Irene Epifanio, 2021. "Robust archetypoids for anomaly detection in big functional data," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 15(2), pages 437-462, June.
    10. Qiu, Zhiping & Chen, Jianwei & Zhang, Jin-Ting, 2021. "Two-sample tests for multivariate functional data with applications," Computational Statistics & Data Analysis, Elsevier, vol. 157(C).
    11. Amovin-Assagba, Martial & Gannaz, Irène & Jacques, Julien, 2022. "Outlier detection in multivariate functional data through a contaminated mixture model," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    12. Francesca Ieva & Anna Paganoni, 2015. "Discussion of “multivariate functional outlier detection” by M. Hubert, P. Rousseeuw and P. Segaert," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(2), pages 217-221, July.
    13. Alvarez, Agustín & Boente, Graciela & Kudraszow, Nadia, 2019. "Robust sieve estimators for functional canonical correlation analysis," Journal of Multivariate Analysis, Elsevier, vol. 170(C), pages 46-62.
    14. Łukasz Smaga & Hidetoshi Matsui, 2018. "A note on variable selection in functional regression via random subspace method," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(3), pages 455-477, August.
    15. Francesca Ieva & Anna Maria Paganoni, 2020. "Component-wise outlier detection methods for robustifying multivariate functional samples," Statistical Papers, Springer, vol. 61(2), pages 595-614, April.
    16. Xiaohui Liu & Karl Mosler & Pavlo Mozharovskyi, 2017. "Fast computation of Tukey trimmed regions and median in dimension p > 2," Working Papers 2017-71, Center for Research in Economics and Statistics.
    17. Davy Paindaveine & Germain Van Bever, 2017. "Halfspace Depths for Scatter, Concentration and Shape Matrices," Working Papers ECARES ECARES 2017-19, ULB -- Universite Libre de Bruxelles.
    18. Diego Rivera-García & Luis A. García-Escudero & Agustín Mayo-Iscar & Joaquín Ortega, 2019. "Robust clustering for functional data based on trimming and constraints," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 13(1), pages 201-225, March.
    19. Mia Hubert & Peter Rousseeuw & Pieter Segaert, 2017. "Multivariate and functional classification using depth and distance," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 11(3), pages 445-466, September.
    20. Virta, Joni & Li, Bing & Nordhausen, Klaus & Oja, Hannu, 2020. "Independent component analysis for multivariate functional data," Journal of Multivariate Analysis, Elsevier, vol. 176(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jstats:v:4:y:2021:i:4:p:57-1011:d:686947. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.