IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2018-16.html
   My bibliography  Save this paper

On normalization and algorithm selection for unsupervised outlier detection

Author

Listed:
  • Sevvandi Kandanaarachchi
  • Mario A Munoz
  • Rob J Hyndman
  • Kate Smith-Miles

Abstract

This paper demonstrates that the performance of various outlier detection methods depends sensitively on both the data normalization schemes employed, as well as characteristics of the datasets. Recasting the challenge of understanding these dependencies as an algorithm selection problem, we perform the first instance space analysis of outlier detection methods. Such analysis enables the strengths and weaknesses of unsupervised outlier detection methods to be visualized and insights gained into which method and normalization scheme should be selected to obtain the most likely best performance for a given dataset.

Suggested Citation

  • Sevvandi Kandanaarachchi & Mario A Munoz & Rob J Hyndman & Kate Smith-Miles, 2018. "On normalization and algorithm selection for unsupervised outlier detection," Monash Econometrics and Business Statistics Working Papers 16/18, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2018-16
    as

    Download full text from publisher

    File URL: https://www.monash.edu/business/ebs/research/publications/ebs/wp16-2018.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Kang, Yanfei & Hyndman, Rob J. & Smith-Miles, Kate, 2017. "Visualising forecasting algorithm performance using time series instance spaces," International Journal of Forecasting, Elsevier, vol. 33(2), pages 345-358.
    2. Billor, Nedret & Hadi, Ali S. & Velleman, Paul F., 2000. "BACON: blocked adaptive computationally efficient outlier nominators," Computational Statistics & Data Analysis, Elsevier, vol. 34(3), pages 279-298, September.
    3. David H. Wolpert & William G. Macready, 1995. "No Free Lunch Theorems for Search," Working Papers 95-02-010, Santa Fe Institute.
    4. Priyanga Dilini Talagala & Rob J Hyndman & Kate Smith-Miles & Sevvandi Kandanaarachchi & Mario A Munoz, 2018. "Anomaly detection in streaming nonstationary temporal data," Monash Econometrics and Business Statistics Working Papers 4/18, Monash University, Department of Econometrics and Business Statistics.
    5. Markus Goldstein & Seiichi Uchida, 2016. "A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mohamad Alissa & Kevin Sim & Emma Hart, 2023. "Automated Algorithm Selection: from Feature-Based to Feature-Free Approaches," Journal of Heuristics, Springer, vol. 29(1), pages 1-38, February.
    2. Sevvandi Kandanaarachchi & Rob J Hyndman, 2019. "Dimension Reduction For Outlier Detection Using DOBIN," Monash Econometrics and Business Statistics Working Papers 17/19, Monash University, Department of Econometrics and Business Statistics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Priyanga Dilini Talagala & Rob J Hyndman & Kate Smith-Miles, 2019. "Anomaly Detection in High Dimensional Data," Monash Econometrics and Business Statistics Working Papers 20/19, Monash University, Department of Econometrics and Business Statistics.
    2. Priyanga Dilini Talagala & Rob J Hyndman & Catherine Leigh & Kerrie Mengersen & Kate Smith-Miles, 2019. "A Feature-Based Framework for Detecting Technical Outliers in Water-Quality Data from In Situ Sensors," Monash Econometrics and Business Statistics Working Papers 1/19, Monash University, Department of Econometrics and Business Statistics.
    3. Sevvandi Kandanaarachchi & Rob J Hyndman, 2019. "Dimension Reduction For Outlier Detection Using DOBIN," Monash Econometrics and Business Statistics Working Papers 17/19, Monash University, Department of Econometrics and Business Statistics.
    4. Smyl, Slawek, 2020. "A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting," International Journal of Forecasting, Elsevier, vol. 36(1), pages 75-85.
    5. Catherine Fuss & Angelos Theodorakopoulos, 2018. "Compositional Changes in Aggregate Productivity in an Era of Globalisation and Financial Crisis," Working Papers of VIVES - Research Centre for Regional Economics 627696, KU Leuven, Faculty of Economics and Business (FEB), VIVES - Research Centre for Regional Economics.
    6. L. Pitsoulis & G. Zioutas, 2010. "A fast algorithm for robust regression with penalised trimmed squares," Computational Statistics, Springer, vol. 25(4), pages 663-689, December.
    7. Yin, Sihua & Yang, Haidong & Xu, Kangkang & Zhu, Chengjiu & Zhang, Shaqing & Liu, Guosheng, 2022. "Dynamic real–time abnormal energy consumption detection and energy efficiency optimization analysis considering uncertainty," Applied Energy, Elsevier, vol. 307(C).
    8. Cristian BARRA & Roberto ZOTTI, 2019. "Bank Performance, Financial Stability And Market Concentration: Evidence From Cooperative And Non‐Cooperative Banks," Annals of Public and Cooperative Economics, Wiley Blackwell, vol. 90(1), pages 103-139, March.
    9. Hong Choon Ong & Ekele Alih, 2015. "A Control Chart Based on Cluster-Regression Adjustment for Retrospective Monitoring of Individual Characteristics," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-30, April.
    10. Stefani, Gianluca & Gadanakis, Yiorgos & Lombardi, Ginevra Virginia & Tiberti, Marco, 2017. "The impact of financial leverage on farms capacity to react in market shocks," 2017 International Congress, August 28-September 1, 2017, Parma, Italy 261156, European Association of Agricultural Economists.
    11. Batalla-Bejerano, Joan & Costa-Campi, Maria Teresa & Trujillo-Baute, Elisa, 2016. "Collateral effects of liberalisation: Metering, losses, load profiles and cost settlement in Spain’s electricity system," Energy Policy, Elsevier, vol. 94(C), pages 421-431.
    12. Thiyanga S. Talagala & Feng Li & Yanfei Kang, 2019. "Feature-based Forecast-Model Performance Prediction," Monash Econometrics and Business Statistics Working Papers 21/19, Monash University, Department of Econometrics and Business Statistics.
    13. A.A.M. Nurunnabi & M. Nasser & A.H.M.R. Imon, 2016. "Identification and classification of multiple outliers, high leverage points and influential observations in linear regression," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(3), pages 509-525, March.
    14. Jui-Sheng Chou & Dinh-Nhat Truong & Chih-Fong Tsai, 2021. "Solving Regression Problems with Intelligent Machine Learner for Engineering Informatics," Mathematics, MDPI, vol. 9(6), pages 1-25, March.
    15. Bojer, Casper Solheim & Meldgaard, Jens Peder, 2021. "Kaggle forecasting competitions: An overlooked learning opportunity," International Journal of Forecasting, Elsevier, vol. 37(2), pages 587-603.
    16. Jiří Schwarz & Martin Pospíšil, 2018. "Bankruptcy, Investment, and Financial Constraints: Evidence from the Czech Republic," Eastern European Economics, Taylor & Francis Journals, vol. 56(2), pages 99-121, March.
    17. Huber, Jakob & Stuckenschmidt, Heiner, 2020. "Daily retail demand forecasting using machine learning with emphasis on calendric special days," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1420-1438.
    18. Li, Li & Kang, Yanfei & Li, Feng, 2023. "Bayesian forecast combination using time-varying features," International Journal of Forecasting, Elsevier, vol. 39(3), pages 1287-1302.
    19. AMENDOLA, Adalgiso & BARRA, Cristian & BOCCIA, Marinella & PAPACCIO, Anna, 2018. "Market Structure and Financial Stability: Theory and Evidence," CELPE Discussion Papers 156, CELPE - CEnter for Labor and Political Economics, University of Salerno, Italy.
    20. Fratianni, Michele & Marchionne, Francesco, 2017. "Bank asset reallocation and sovereign debt," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 47(C), pages 15-32.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2018-16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.