IDEAS home Printed from https://ideas.repec.org/p/msh/ebswps/2019-17.html
   My bibliography  Save this paper

Dimension Reduction For Outlier Detection Using DOBIN

Author

Listed:
  • Sevvandi Kandanaarachchi
  • Rob J Hyndman

Abstract

This paper introduces DOBIN, a new approach to select a set of basis vectors tailored for outlier detection. DOBIN has a solid mathematical foundation and can be used as a dimension reduction tool for outlier detection tasks. We demonstrate the effectiveness of DOBIN on an extensive data repository, by comparing the performance of outlier detection methods using DOBIN and other bases. We further illustrate the utility of DOBIN as an outlier visualization tool. The R package dobin implements this basis construction.

Suggested Citation

  • Sevvandi Kandanaarachchi & Rob J Hyndman, 2019. "Dimension Reduction For Outlier Detection Using DOBIN," Monash Econometrics and Business Statistics Working Papers 17/19, Monash University, Department of Econometrics and Business Statistics.
    • Handle: RePEc:msh:ebswps:2019-17
    as

    Download full text from publisher

    File URL: https://www.monash.edu/business/ebs/research/publications/ebs/wp17-2019.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Vakili, Kaveh & Schmitt, Eric, 2014. "Finding multivariate outliers with FastPCS," Computational Statistics & Data Analysis, Elsevier, vol. 69(C), pages 54-66.
    2. Billor, Nedret & Hadi, Ali S. & Velleman, Paul F., 2000. "BACON: blocked adaptive computationally efficient outlier nominators," Computational Statistics & Data Analysis, Elsevier, vol. 34(3), pages 279-298, September.
    3. Sevvandi Kandanaarachchi & Mario A Munoz & Rob J Hyndman & Kate Smith-Miles, 2018. "On normalization and algorithm selection for unsupervised outlier detection," Monash Econometrics and Business Statistics Working Papers 16/18, Monash University, Department of Econometrics and Business Statistics.
    4. Markus Goldstein & Seiichi Uchida, 2016. "A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sevvandi Kandanaarachchi & Mario A Munoz & Rob J Hyndman & Kate Smith-Miles, 2018. "On normalization and algorithm selection for unsupervised outlier detection," Monash Econometrics and Business Statistics Working Papers 16/18, Monash University, Department of Econometrics and Business Statistics.
    2. Catherine Fuss & Angelos Theodorakopoulos, 2018. "Compositional Changes in Aggregate Productivity in an Era of Globalisation and Financial Crisis," Working Papers of VIVES - Research Centre for Regional Economics 627696, KU Leuven, Faculty of Economics and Business (FEB), VIVES - Research Centre for Regional Economics.
    3. L. Pitsoulis & G. Zioutas, 2010. "A fast algorithm for robust regression with penalised trimmed squares," Computational Statistics, Springer, vol. 25(4), pages 663-689, December.
    4. Yin, Sihua & Yang, Haidong & Xu, Kangkang & Zhu, Chengjiu & Zhang, Shaqing & Liu, Guosheng, 2022. "Dynamic real–time abnormal energy consumption detection and energy efficiency optimization analysis considering uncertainty," Applied Energy, Elsevier, vol. 307(C).
    5. Cristian BARRA & Roberto ZOTTI, 2019. "Bank Performance, Financial Stability And Market Concentration: Evidence From Cooperative And Non‐Cooperative Banks," Annals of Public and Cooperative Economics, Wiley Blackwell, vol. 90(1), pages 103-139, March.
    6. Hong Choon Ong & Ekele Alih, 2015. "A Control Chart Based on Cluster-Regression Adjustment for Retrospective Monitoring of Individual Characteristics," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-30, April.
    7. Stefani, Gianluca & Gadanakis, Yiorgos & Lombardi, Ginevra Virginia & Tiberti, Marco, 2017. "The impact of financial leverage on farms capacity to react in market shocks," 2017 International Congress, August 28-September 1, 2017, Parma, Italy 261156, European Association of Agricultural Economists.
    8. Schmitt, Eric & Öllerer, Viktoria & Vakili, Kaveh, 2014. "The finite sample breakdown point of PCS," Statistics & Probability Letters, Elsevier, vol. 94(C), pages 214-220.
    9. Batalla-Bejerano, Joan & Costa-Campi, Maria Teresa & Trujillo-Baute, Elisa, 2016. "Collateral effects of liberalisation: Metering, losses, load profiles and cost settlement in Spain’s electricity system," Energy Policy, Elsevier, vol. 94(C), pages 421-431.
    10. A.A.M. Nurunnabi & M. Nasser & A.H.M.R. Imon, 2016. "Identification and classification of multiple outliers, high leverage points and influential observations in linear regression," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(3), pages 509-525, March.
    11. Jiří Schwarz & Martin Pospíšil, 2018. "Bankruptcy, Investment, and Financial Constraints: Evidence from the Czech Republic," Eastern European Economics, Taylor & Francis Journals, vol. 56(2), pages 99-121, March.
    12. AMENDOLA, Adalgiso & BARRA, Cristian & BOCCIA, Marinella & PAPACCIO, Anna, 2018. "Market Structure and Financial Stability: Theory and Evidence," CELPE Discussion Papers 156, CELPE - CEnter for Labor and Political Economics, University of Salerno, Italy.
    13. Priyanga Dilini Talagala & Rob J Hyndman & Kate Smith-Miles, 2019. "Anomaly Detection in High Dimensional Data," Monash Econometrics and Business Statistics Working Papers 20/19, Monash University, Department of Econometrics and Business Statistics.
    14. Fratianni, Michele & Marchionne, Francesco, 2017. "Bank asset reallocation and sovereign debt," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 47(C), pages 15-32.
    15. Kosiorowski Daniel & Mielczarek Dominik & Rydlewski Jerzy P. & Snarska Małgorzata, 2018. "Generalized Exponential Smoothing In Prediction Of Hierarchical Time Series," Statistics in Transition New Series, Polish Statistical Association, vol. 19(2), pages 331-350, June.
    16. Sylvain Weber, 2010. "bacon: An effective way to detect outliers in multivariate data using Stata (and Mata)," Stata Journal, StataCorp LP, vol. 10(3), pages 331-338, September.
    17. Mele, Antonio & Paglialunga, Elena & Sforna, Giorgia, 2021. "Climate cooperation from Kyoto to Paris: What can be learnt from the CDM experience?," Socio-Economic Planning Sciences, Elsevier, vol. 75(C).
    18. Balcazar Salazar,Carlos Felipe, 2015. "Long-run effects of democracy on income inequality : evidence from repeated cross-sections," Policy Research Working Paper Series 7153, The World Bank.
    19. Claude Diebolt & Audrey-Rose Menard & Faustine Perrin, 2016. "Behind the Fertility-Education Nexus: What Triggered the French Development Process?," Working Papers 03-16, Association Française de Cliométrie (AFC).
    20. Philippe Kabore & Nicholas Rivers, 2023. "Manufacturing output and extreme temperature: Evidence from Canada," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 56(1), pages 191-224, February.

    More about this item

    Keywords

    outlier detection; dimension reduction; outlier visualization; basis vectors;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C38 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Classification Methdos; Cluster Analysis; Principal Components; Factor Analysis
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:msh:ebswps:2019-17. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Professor Xibin Zhang (email available below). General contact details of provider: https://edirc.repec.org/data/dxmonau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.