IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v12y2020i7p2897-d341735.html
   My bibliography  Save this article

Anomaly Detection System for Water Networks in Northern Ethiopia Using Bayesian Inference

Author

Listed:
  • Zaid Tashman

    (Accenture Labs, San Francisco, CA 94105, USA)

  • Christoph Gorder

    (Charity Water, New York City, NY 10013, USA)

  • Sonali Parthasarathy

    (Accenture Labs, San Francisco, CA 94105, USA)

  • Mohamad M. Nasr-Azadani

    (Accenture Labs, San Francisco, CA 94105, USA)

  • Rachel Webre

    (Charity Water, New York City, NY 10013, USA)

Abstract

For billions of people living in remote and rural communities in the developing countries, small water systems are the only source of clean drinking water. Due to the rural nature of such water systems, site visits may occur infrequently. This means broken water systems can remain in a malfunctioning state for months, forcing communities to return to drinking unsafe water. In this work, we present a novel two-level anomaly detection system aimed to detect malfunctioning remote sensored water hand-pumps, allowing for a proactive approach to pump maintenance. To detect anomalies, we need a model of normal water usage behavior first. We train a multilevel probabilistic model of normal usage using approximate variational Bayesian inference to obtain a conditional probability distribution over the hourly water usage data. We then use this conditional distribution to construct a level-1 scoring function for each hourly water observation and a level-2 scoring function for each pump. Probabilistic models and Bayesian inference collectively were chosen for their ability to capture the high temporal variability in the water usage data at the individual pump level as well as their ability to estimate interpretable model parameters. Experimental results in this work have demonstrated that the pump scoring function is able to detect malfunctioning sensors as well as a change in water usage behavior allowing for a more responsive and proactive pump system maintenance.

Suggested Citation

  • Zaid Tashman & Christoph Gorder & Sonali Parthasarathy & Mohamad M. Nasr-Azadani & Rachel Webre, 2020. "Anomaly Detection System for Water Networks in Northern Ethiopia Using Bayesian Inference," Sustainability, MDPI, vol. 12(7), pages 1-16, April.
  • Handle: RePEc:gam:jsusta:v:12:y:2020:i:7:p:2897-:d:341735
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/12/7/2897/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/12/7/2897/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Carpenter, Bob & Gelman, Andrew & Hoffman, Matthew D. & Lee, Daniel & Goodrich, Ben & Betancourt, Michael & Brubaker, Marcus & Guo, Jiqiang & Li, Peter & Riddell, Allen, 2017. "Stan: A Probabilistic Programming Language," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i01).
    2. Yixin Wang & David M. Blei, 2019. "The Blessings of Multiple Causes," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(528), pages 1574-1596, October.
    3. Douglas M. Hawkins, 1980. "Critical Values for Identifying Outliers," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 29(1), pages 95-96, March.
    4. David M. Blei & Alp Kucukelbir & Jon D. McAuliffe, 2017. "Variational Inference: A Review for Statisticians," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(518), pages 859-877, April.
    5. Jonathan Rougier & Michael Goldstein, 2001. "A Bayesian analysis of fluid flow in pipe‐lines," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 50(1), pages 77-93.
    6. Daniel L Wilson & Jeremy R Coyle & Evan A Thomas, 2017. "Ensemble machine learning and forecasting can achieve 99% uptime for rural handpumps," PLOS ONE, Public Library of Science, vol. 12(11), pages 1-13, November.
    7. M. Shafiqul Islam & Rehan Sadiq & Manuel J. Rodriguez & Homayoun Najjaran & Mina Hoorfar, 2016. "Integrated Decision Support System for Prognostic and Diagnostic Analyses of Water Distribution System Failures," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 30(8), pages 2831-2850, June.
    8. Zoubin Ghahramani, 2015. "Probabilistic machine learning and artificial intelligence," Nature, Nature, vol. 521(7553), pages 452-459, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gael M. Martin & David T. Frazier & Christian P. Robert, 2020. "Computing Bayes: Bayesian Computation from 1763 to the 21st Century," Monash Econometrics and Business Statistics Working Papers 14/20, Monash University, Department of Econometrics and Business Statistics.
    2. Linda S. L. Tan, 2021. "Use of model reparametrization to improve variational Bayes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(1), pages 30-57, February.
    3. Feihong Xia & Rabikar Chatterjee & Jerrold H. May, 2019. "Using Conditional Restricted Boltzmann Machines to Model Complex Consumer Shopping Patterns," Marketing Science, INFORMS, vol. 38(4), pages 711-727, July.
    4. Gael M. Martin & David T. Frazier & Christian P. Robert, 2021. "Approximating Bayes in the 21st Century," Monash Econometrics and Business Statistics Working Papers 24/21, Monash University, Department of Econometrics and Business Statistics.
    5. O’Dwyer, Edward & Pan, Indranil & Acha, Salvador & Shah, Nilay, 2019. "Smart energy systems for sustainable smart cities: Current developments, trends and future directions," Applied Energy, Elsevier, vol. 237(C), pages 581-597.
    6. Bruno Jacobs & Dennis Fok & Bas Donkers, 2021. "Understanding Large-Scale Dynamic Purchase Behavior," Marketing Science, INFORMS, vol. 40(5), pages 844-870, September.
    7. Zhang, Jincheng & Zhao, Xiaowei & Jin, Siya & Greaves, Deborah, 2022. "Phase-resolved real-time ocean wave prediction with quantified uncertainty based on variational Bayesian machine learning," Applied Energy, Elsevier, vol. 324(C).
    8. Gael M. Martin & David T. Frazier & Christian P. Robert, 2022. "Computing Bayes: From Then `Til Now," Monash Econometrics and Business Statistics Working Papers 14/22, Monash University, Department of Econometrics and Business Statistics.
    9. Azzimonti, Laura & Corani, Giorgio & Zaffalon, Marco, 2019. "Hierarchical estimation of parameters in Bayesian networks," Computational Statistics & Data Analysis, Elsevier, vol. 137(C), pages 67-91.
    10. Bansal, Prateek & Krueger, Rico & Bierlaire, Michel & Daziano, Ricardo A. & Rashidi, Taha H., 2020. "Bayesian estimation of mixed multinomial logit models: Advances and simulation-based evaluations," Transportation Research Part B: Methodological, Elsevier, vol. 131(C), pages 124-142.
    11. Caroline Jennings Saul & Heiko Gebauer, 2018. "Digital Transformation as an Enabler for Advanced Services in the Sanitation Sector," Sustainability, MDPI, vol. 10(3), pages 1-18, March.
    12. Masafumi Nakano & Akihiko Takahashi & Soichiro Takahashi, 2018. "State Space Approach to Adaptive Artificial Intelligence Modeling: Application to Financial Portfolio with Fuzzy System," CARF F-Series CARF-F-422, Center for Advanced Research in Finance, Faculty of Economics, The University of Tokyo.
    13. Francis,David C. & Kubinec ,Robert, 2022. "Beyond Political Connections : A Measurement Model Approach to Estimating Firm-levelPolitical Influence in 41 Economies," Policy Research Working Paper Series 10119, The World Bank.
    14. Martinovici, A., 2019. "Revealing attention - how eye movements predict brand choice and moment of choice," Other publications TiSEM 7dca38a5-9f78-4aee-bd81-c, Tilburg University, School of Economics and Management.
    15. Yongping Bao & Ludwig Danwitz & Fabian Dvorak & Sebastian Fehrler & Lars Hornuf & Hsuan Yu Lin & Bettina von Helversen, 2022. "Similarity and Consistency in Algorithm-Guided Exploration," CESifo Working Paper Series 10188, CESifo.
    16. Torsten Heinrich & Jangho Yang & Shuanping Dai, 2020. "Growth, development, and structural change at the firm-level: The example of the PR China," Papers 2012.14503, arXiv.org.
    17. van Kesteren Erik-Jan & Bergkamp Tom, 2023. "Bayesian analysis of Formula One race results: disentangling driver skill and constructor advantage," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 19(4), pages 273-293, December.
    18. Shen Liu & Hongyan Liu, 2021. "Tagging Items Automatically Based on Both Content Information and Browsing Behaviors," INFORMS Journal on Computing, INFORMS, vol. 33(3), pages 882-897, July.
    19. Xin Xu & Yang Lu & Yupeng Zhou & Zhiguo Fu & Yanjie Fu & Minghao Yin, 2021. "An Information-Explainable Random Walk Based Unsupervised Network Representation Learning Framework on Node Classification Tasks," Mathematics, MDPI, vol. 9(15), pages 1-14, July.
    20. Wilson, Christopher & van der Velden, Maja, 2022. "Sustainable AI: An integrated model to guide public sector decision-making," Technology in Society, Elsevier, vol. 68(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:12:y:2020:i:7:p:2897-:d:341735. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.