IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i21p14324-d961066.html
   My bibliography  Save this article

A Scalable Open-Source Framework for Machine Learning-Based Image Collection, Annotation and Classification: A Case Study for Automatic Fish Species Identification

Author

Listed:
  • Catarina N. S. Silva

    (Nature Research Centre, LT-08412 Vilnius, Lithuania)

  • Justas Dainys

    (Nature Research Centre, LT-08412 Vilnius, Lithuania)

  • Sean Simmons

    (MyCatch and Angler’s Atlas, Prince George, BC V2L 4S1, Canada)

  • Vincentas Vienožinskis

    (Deeper, LT-10312 Vilnius, Lithuania)

  • Asta Audzijonyte

    (Nature Research Centre, LT-08412 Vilnius, Lithuania
    Institute for Marine and Antarctic Studies, University of Tasmania, Hobart 7004, Australia)

Abstract

Citizen science platforms, social media and smart phone applications enable the collection of large amounts of georeferenced images. This provides a huge opportunity in biodiversity and ecological research, but also creates challenges for efficient data handling and processing. Recreational and small-scale fisheries is one of the fields that could be revolutionised by efficient, widely accessible and machine learning-based processing of georeferenced images. Most non-commercial inland and coastal fisheries are considered data poor and are rarely assessed, yet they provide multiple societal benefits and can have substantial ecological impacts. Given that large quantities of georeferenced fish images are being collected by fishers every day, artificial intelligence (AI) and computer vision applications offer a great opportunity to automate their analyses by providing species identification, and potentially also fish size estimation. This would deliver data needed for fisheries management and fisher engagement. To date, however, many AI image analysis applications in fisheries are focused on the commercial sector, limited to specific species or settings, and are not publicly available. In addition, using AI and computer vision tools often requires a strong background in programming. In this study, we aim to facilitate broader use of computer vision tools in fisheries and ecological research by compiling an open-source user friendly and modular framework for large-scale image storage, handling, annotation and automatic classification, using cost- and labour-efficient methodologies. The tool is based on TensorFlow Lite Model Maker library, and includes data augmentation and transfer learning techniques applied to different convolutional neural network models. We demonstrate the potential application of this framework using a small example dataset of fish images taken through a recreational fishing smartphone application. The framework presented here can be used to develop region-specific species identification models, which could potentially be combined into a larger hierarchical model.

Suggested Citation

  • Catarina N. S. Silva & Justas Dainys & Sean Simmons & Vincentas Vienožinskis & Asta Audzijonyte, 2022. "A Scalable Open-Source Framework for Machine Learning-Based Image Collection, Annotation and Classification: A Case Study for Automatic Fish Species Identification," Sustainability, MDPI, vol. 14(21), pages 1-13, November.
    • Handle: RePEc:gam:jsusta:v:14:y:2022:i:21:p:14324-:d:961066
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/21/14324/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/14/21/14324/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rusul Abduljabbar & Hussein Dia & Sohani Liyanage & Saeed Asadi Bagloee, 2019. "Applications of Artificial Intelligence in Transport: An Overview," Sustainability, MDPI, vol. 11(1), pages 1-24, January.
    2. Sanaz Honarmand Ebrahimi & Marinus Ossewaarde & Ariana Need, 2021. "Smart Fishery: A Systematic Review and Research Agenda for Sustainable Fisheries in the Age of AI," Sustainability, MDPI, vol. 13(11), pages 1-20, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marya Butt & Ander de Keijzer, 2022. "Using Transfer Learning to Train a Binary Classifier for Lorrca Ektacytometery Microscopic Images of Sickle Cells and Healthy Red Blood Cells," Data, MDPI, vol. 7(9), pages 1-21, September.
    2. Sohani Liyanage & Hussein Dia & Rusul Abduljabbar & Saeed Asadi Bagloee, 2019. "Flexible Mobility On-Demand: An Environmental Scan," Sustainability, MDPI, vol. 11(5), pages 1-39, February.
    3. Pamucar, Dragan & Deveci, Muhammet & Gokasar, Ilgin & Tavana, Madjid & Köppen, Mario, 2022. "A metaverse assessment model for sustainable transportation using ordinal priority approach and Aczel-Alsina norms," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    4. Eugène Loos & Maria Sourbati & Frauke Behrendt, 2020. "The Role of Mobility Digital Ecosystems for Age-Friendly Urban Public Transport: A Narrative Literature Review," IJERPH, MDPI, vol. 17(20), pages 1-16, October.
    5. De Obesso Arias, María de las Mercedes & Pérez Rivero, Carlos Alberto & Carrero Márquez, Oliver, 2023. "Artificial intelligence to manage workplace bullying," Journal of Business Research, Elsevier, vol. 160(C).
    6. Fahad Alrukaibi & Rushdi Alsaleh & Tarek Sayed, 2019. "Applying Machine Learning and Statistical Approaches for Travel Time Estimation in Partial Network Coverage," Sustainability, MDPI, vol. 11(14), pages 1-18, July.
    7. Waymond Rodgers & Mouza Al Habsi & George Gamble, 2019. "Sustainability and Firm Performance: A Review and Analysis Using Algorithmic Pathways in the Throughput Model," Sustainability, MDPI, vol. 11(14), pages 1-27, July.
    8. Alexandros Nikitas & Kalliopi Michalakopoulou & Eric Tchouamou Njoya & Dimitris Karampatzakis, 2020. "Artificial Intelligence, Transport and the Smart City: Definitions and Dimensions of a New Mobility Era," Sustainability, MDPI, vol. 12(7), pages 1-19, April.
    9. Phattarasuda Witchayaphong & Surachet Pravinvongvuth & Kunnawee Kanitpong & Kazushi Sano & Suksun Horpibulsuk, 2020. "Influential Factors Affecting Travelers’ Mode Choice Behavior on Mass Transit in Bangkok, Thailand," Sustainability, MDPI, vol. 12(22), pages 1-18, November.
    10. Przemysław Kowalik & Grzegorz Sobecki & Piotr Bawoł & Paweł Muzolf, 2023. "A Flow-Based Formulation of the Travelling Salesman Problem with Penalties on Nodes," Sustainability, MDPI, vol. 15(5), pages 1-28, February.
    11. Guimin Gong & Wenhong Lv & Qi Wang, 2023. "Research on Urban Road Traffic Network Pinning Control Based on Feedback Control," Sustainability, MDPI, vol. 15(12), pages 1-17, June.
    12. Christina Kakderi & Eleni Oikonomaki & Ilektra Papadaki, 2021. "Smart and Resilient Urban Futures for Sustainability in the Post COVID-19 Era: A Review of Policy Responses on Urban Mobility," Sustainability, MDPI, vol. 13(11), pages 1-21, June.
    13. Tan Yigitcanlar & Kevin C. Desouza & Luke Butler & Farnoosh Roozkhosh, 2020. "Contributions and Risks of Artificial Intelligence (AI) in Building Smarter Cities: Insights from a Systematic Review of the Literature," Energies, MDPI, vol. 13(6), pages 1-38, March.
    14. Wu, Min & Wang, Nanxi & Yuen, Kum Fai, 2023. "Can autonomy level and anthropomorphic characteristics affect public acceptance and trust towards shared autonomous vehicles?," Technological Forecasting and Social Change, Elsevier, vol. 189(C).
    15. Mochen Liao & Kai Lan & Yuan Yao, 2022. "Sustainability implications of artificial intelligence in the chemical industry: A conceptual framework," Journal of Industrial Ecology, Yale University, vol. 26(1), pages 164-182, February.
    16. Zulamir Hassani, Afdhal & Yusoff, Fazirah & Wan Zain, Wan Nor Aisyah, 2021. "Fair and Responsible in Logistics IR 4.0," MPRA Paper 108432, University Library of Munich, Germany.
    17. Jae-joon Chung & Hyun-Jung Kim, 2020. "An Automobile Environment Detection System Based on Deep Neural Network and its Implementation Using IoT-Enabled In-Vehicle Air Quality Sensors," Sustainability, MDPI, vol. 12(6), pages 1-17, March.
    18. Jin, Hui & Liu, Yue & Wu, Telan & Zhang, Yanpei, 2022. "Site-specific optimization of bus stop locations and designs over a corridor," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 599(C).
    19. Tahmineh Ladi & Shaghayegh Jabalameli & Ayyoob Sharifi, 2022. "Applications of machine learning and deep learning methods for climate change mitigation and adaptation," Environment and Planning B, , vol. 49(4), pages 1314-1330, May.
    20. Fosso Wamba, Samuel & Bawack, Ransome Epie & Guthrie, Cameron & Queiroz, Maciel M. & Carillo, Kevin Daniel André, 2021. "Are we preparing for a good AI society? A bibliometric review and research agenda," Technological Forecasting and Social Change, Elsevier, vol. 164(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:21:p:14324-:d:961066. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.