IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v36y2021i4d10.1007_s00180-021-01096-1.html
   My bibliography  Save this article

Algorithm for error-free determination of the variance of all contiguous subsequences and fixed-length contiguous subsequences for a sequence of industrial measurement data

Author

Listed:
  • Andrzej Chmielowiec

    (Rzeszow University of Technology)

Abstract

The article presents an algorithm for fast and error-free determination of statistics such as the arithmetic mean and variance of all contiguous subsequences and fixed-length contiguous subsequences for a sequence of industrial measurement data. Additionally, it shows that both floating-point and integer representation can be used to perform this kind of statistical calculations. The author proves a theorem on the number of bits of precision that an arithmetic type must have to guarantee error-free determination of the arithmetic mean and variance. The article also presents the extension of Welford’s formula for determining variance for the sliding window method—determining the variance of fixed-length contiguous subsequences. The section dedicated to implementation tests shows the running times of individual algorithms depending on the arithmetic type used. The research shows that the use of integers in calculations makes the determination of the aforementioned statistics much faster.

Suggested Citation

  • Andrzej Chmielowiec, 2021. "Algorithm for error-free determination of the variance of all contiguous subsequences and fixed-length contiguous subsequences for a sequence of industrial measurement data," Computational Statistics, Springer, vol. 36(4), pages 2813-2840, December.
    • Handle: RePEc:spr:compst:v:36:y:2021:i:4:d:10.1007_s00180-021-01096-1
    DOI: 10.1007/s00180-021-01096-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00180-021-01096-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s00180-021-01096-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Douglas M. Hawkins, 1980. "Critical Values for Identifying Outliers," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 29(1), pages 95-96, March.
    2. Philippe Pébay & Timothy B. Terriberry & Hemanth Kolla & Janine Bennett, 2016. "Numerically stable, scalable formulas for parallel and online computation of higher-order multivariate central moments with arbitrary weights," Computational Statistics, Springer, vol. 31(4), pages 1305-1325, December.
    3. Katie Evans & Tanzy Love & Sally Thurston, 2015. "Outlier Identification in Model-Based Cluster Analysis," Journal of Classification, Springer;The Classification Society, vol. 32(1), pages 63-84, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Denys Baranovskyi & Maryna Bulakh & Adam Michajłyszyn & Sergey Myamlin & Leonty Muradian, 2023. "Determination of the Risk of Failures of Locomotive Diesel Engines in Maintenance," Energies, MDPI, vol. 16(13), pages 1-14, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Karol Pilot & Alicja Ganczarek-Gamrot & Krzysztof Kania, 2024. "Dealing with Anomalies in Day-Ahead Market Prediction Using Machine Learning Hybrid Model," Energies, MDPI, vol. 17(17), pages 1-20, September.
    2. Francesca Ieva & Anna Maria Paganoni, 2020. "Component-wise outlier detection methods for robustifying multivariate functional samples," Statistical Papers, Springer, vol. 61(2), pages 595-614, April.
    3. Marc Chataigner & Stéphane Crépey & Jiang Pu, 2020. "Nowcasting Networks," Post-Print hal-03910123, HAL.
    4. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2019. "Sigma-Mu efficiency analysis: A methodology for evaluating units through composite indicators," European Journal of Operational Research, Elsevier, vol. 278(3), pages 942-960.
    5. David Juárez-Varón & Victoria Tur-Viñes & Alejandro Rabasa-Dolado & Kristina Polotskaya, 2020. "An Adaptive Machine Learning Methodology Applied to Neuromarketing Analysis: Prediction of Consumer Behaviour Regarding the Key Elements of the Packaging Design of an Educational Toy," Social Sciences, MDPI, vol. 9(9), pages 1-23, September.
    6. Zhongqiu Wang & Guan Yuan & Haoran Pei & Yanmei Zhang & Xiao Liu, 2020. "Unsupervised learning trajectory anomaly detection algorithm based on deep representation," International Journal of Distributed Sensor Networks, , vol. 16(12), pages 15501477209, December.
    7. Arata, Linda & Fabrizi, Enrico & Sckokai, Paolo, 2020. "A worldwide analysis of trend in crop yields and yield variability: Evidence from FAO data," Economic Modelling, Elsevier, vol. 90(C), pages 190-208.
    8. Iskander Karibzhanov, 2020. "Towards a HANK Model for Canada: Estimating a Canadian Income Process," Discussion Papers 2020-13, Bank of Canada.
    9. Wentao Yang & Huaxi He & Dongsheng Wei & Hao Chen, 2022. "Generating pseudo-absence samples of invasive species based on outlier detection in the geographical characteristic space," Journal of Geographical Systems, Springer, vol. 24(2), pages 261-279, April.
    10. Fournier, Nicholas PhD & Farid, Yashar Zeinali PhD & Patire, Anthony David PhD, 2021. "Potential Erroneous Degradation of High Occupancy Vehicle (HOV) Facilities," Institute of Transportation Studies, Research Reports, Working Papers, Proceedings qt3z76r7tj, Institute of Transportation Studies, UC Berkeley.
    11. Stephanou, Michael & Varughese, Melvin, 2021. "Sequential estimation of Spearman rank correlation using Hermite series estimators," Journal of Multivariate Analysis, Elsevier, vol. 186(C).
    12. Puteri Paramita & Zuduo Zheng & Md Mazharul Haque & Simon Washington & Paul Hyland, 2018. "User satisfaction with train fares: A comparative analysis in five Australian cities," PLOS ONE, Public Library of Science, vol. 13(6), pages 1-26, June.
    13. Sunil Kumar & Sudeep Varshney & Usha Jain & Prashant Johri & Abdulaziz S Almazyad & Ali Wagdy Mohamed & Mehdi Hosseinzadeh & Mohammad Shokouhifar, 2025. "EODA: A three-stage efficient outlier detection approach using Boruta-RF feature selection and enhanced KNN-based clustering algorithm," PLOS ONE, Public Library of Science, vol. 20(5), pages 1-25, May.
    14. Ippel, L. & Kaptein, M.C. & Vermunt, J.K., 2019. "Online estimation of individual-level effects using streaming shrinkage factors," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 16-32.
    15. Liqun Diao & Grace Y. Yi, 2023. "Classification Trees with Mismeasured Responses," Journal of Classification, Springer;The Classification Society, vol. 40(1), pages 168-191, April.
    16. Gasser, Patrick, 2020. "A review on energy security indices to compare country performances," Energy Policy, Elsevier, vol. 139(C).
    17. Nirpeksh Kumar, 2019. "Exact distributions of tests of outliers for exponential samples," Statistical Papers, Springer, vol. 60(6), pages 2031-2061, December.
    18. Stanley Munamato Mbiva & Fabio Mathias Correa, 2024. "Machine Learning to Enhance the Detection of Terrorist Financing and Suspicious Transactions in Migrant Remittances," JRFM, MDPI, vol. 17(5), pages 1-19, April.
    19. Taha Yehia & Ali Wahba & Sondos Mostafa & Omar Mahmoud, 2022. "Suitability of Different Machine Learning Outlier Detection Algorithms to Improve Shale Gas Production Data for Effective Decline Curve Analysis," Energies, MDPI, vol. 15(23), pages 1-25, November.
    20. Beata Gavurova & Jaroslav Belas & Katarina Zvarikova & Martin Rigelsky & Viera Ivankova, 2021. "The Effect of Education and R&D on Tourism Spending in OECD Countries: An Empirical Study," The AMFITEATRU ECONOMIC journal, Academy of Economic Studies - Bucharest, Romania, vol. 23(58), pages 806-806, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:36:y:2021:i:4:d:10.1007_s00180-021-01096-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.