IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v11y2023i11p2464-d1156816.html
   My bibliography  Save this article

Detection of Anomalies in Natural Complicated Data Structures Based on a Hybrid Approach

Author

Listed:
  • Oksana Mandrikova

    (Institute of Cosmophysical Research and Radio Wave Propagation, Far Eastern Branch of the Russian Academy of Sciences, Mirnayast, 7, 684034 Paratunka, Russia)

  • Bogdana Mandrikova

    (Institute of Cosmophysical Research and Radio Wave Propagation, Far Eastern Branch of the Russian Academy of Sciences, Mirnayast, 7, 684034 Paratunka, Russia)

  • Oleg Esikov

    (Institute of Cosmophysical Research and Radio Wave Propagation, Far Eastern Branch of the Russian Academy of Sciences, Mirnayast, 7, 684034 Paratunka, Russia)

Abstract

A hybrid approach is proposed to detect anomalies in natural complicated data structures with high noise levels. The approach includes the application of an autoencoder neural network and singular spectrum analysis (SSA) with an adaptive anomaly detection algorithm (AADA) developed by the authors. The autoencoder is the quintessence of the representation learning algorithm, and it projects (selects) data features. Here, under-complete autoencoders are used. They are a product of the development of the principal component method and allow one to approximate complex nonlinear dependencies. Singular spectrum analysis decomposes data through the singular decomposition of matrix trajectories and makes it possible to detect the data structure in the noise. The AADA is based on the combination of wavelet transforms with threshold functions. Combinations of different constructions of wavelet transformation with threshold functions are widely applied to tasks relating to complex data processing. However, when the noise level is high and there is no complete knowledge of a useful signal, anomaly detection is not a trivial problem and requires a complex approach. This paper considers the use of adaptive threshold functions, the parameters of which are estimated on a probabilistic basis. Adaptive thresholds and a moving time window are introduced. The efficiency of the proposed method in detecting anomalies in neutron monitor data is illustrated. Neutron monitor data record cosmic ray intensities. We used neutron monitor data from ground stations. Anomalies in cosmic rays can create serious radiation hazards for people as well as for space and ground facilities. Thus, the diagnostics of anomalies in cosmic ray parameters is quite topical, and research is being carried out by teams from different countries. A comparison of the results for the autoencoder + AADA and SSA + AADA methods showed the higher efficiency of the autoencoder + AADA method. A more flexible NN apparatus provides better detection of short-period anomalies that have complicated structures. However, the combination of SSA and the AADA is efficient in the detection of long-term anomalies in cosmic rays that occur during strong magnetic storms. Thus, cosmic ray data analysis requires a more complex approach, including the use of the autoencoder and SSA with the AADA.

Suggested Citation

  • Oksana Mandrikova & Bogdana Mandrikova & Oleg Esikov, 2023. "Detection of Anomalies in Natural Complicated Data Structures Based on a Hybrid Approach," Mathematics, MDPI, vol. 11(11), pages 1-17, May.
  • Handle: RePEc:gam:jmathe:v:11:y:2023:i:11:p:2464-:d:1156816
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/11/11/2464/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2227-7390/11/11/2464/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Oksana Mandrikova & Bogdana Mandrikova & Anastasia Rodomanskay, 2021. "Method of Constructing a Nonlinear Approximating Scheme of a Complex Signal: Application Pattern Recognition," Mathematics, MDPI, vol. 9(7), pages 1-15, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Oksana Mandrikova & Bogdana Mandrikova, 2024. "Hybrid Model of Natural Time Series with Neural Network Component and Adaptive Nonlinear Scheme: Application for Anomaly Detection," Mathematics, MDPI, vol. 12(7), pages 1-15, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Oksana Mandrikova & Yuryi Polozov & Nataly Zhukova & Yulia Shichkina, 2022. "Approximation and Analysis of Natural Data Based on NARX Neural Networks Involving Wavelet Filtering," Mathematics, MDPI, vol. 10(22), pages 1-16, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:11:y:2023:i:11:p:2464-:d:1156816. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.