IDEAS home Printed from https://ideas.repec.org/a/plo/pdig00/0000229.html
   My bibliography  Save this article

An experimental machine learning study investigating the decision-making process of students and qualified radiographers when interpreting radiographic images

Author

Listed:
  • Clare Rainey
  • Angelina T Villikudathil
  • Jonathan McConnell
  • Ciara Hughes
  • Raymond Bond
  • Sonyia McFadden

Abstract

AI is becoming more prevalent in healthcare and is predicted to be further integrated into workflows to ease the pressure on an already stretched service. The National Health Service in the UK has prioritised AI and Digital health as part of its Long-Term Plan. Few studies have examined the human interaction with such systems in healthcare, despite reports of biases being present with the use of AI in other technologically advanced fields, such as finance and aviation. Understanding is needed of how certain user characteristics may impact how radiographers engage with AI systems in use in the clinical setting to mitigate against problems before they arise. The aim of this study is to determine correlations of skills, confidence in AI and perceived knowledge amongst student and qualified radiographers in the UK healthcare system. A machine learning based AI model was built to predict if the interpreter was either a student (n = 67) or a qualified radiographer (n = 39) in advance, using important variables from a feature selection technique named Boruta. A survey, which required the participant to interpret a series of plain radiographic examinations with and without AI assistance, was created on the Qualtrics survey platform and promoted via social media (Twitter/LinkedIn), therefore adopting convenience, snowball sampling This survey was open to all UK radiographers, including students and retired radiographers.Pearson’s correlation analysis revealed that males who were proficient in their profession were more likely than females to trust AI. Trust in AI was negatively correlated with age and with level of experience. A machine learning model was built, the best model predicted the image interpreter to be qualified radiographers with 0.93 area under curve and a prediction accuracy of 93%. Further testing in prospective validation cohorts using a larger sample size is required to determine the clinical utility of the proposed machine learning model.Author summary: Artificial Intelligence (AI) is becoming increasingly integrated into healthcare systems. Radiology, as a technologically advanced profession, is one area where AI has proved useful for myriad tasks. Developments in computer vision have allowed for applications in computer aided diagnosis using radiographic images. The integration and development of AI has been supported by healthcare providers and government agencies, such as the NHS in the UK. With the introduction of these systems in the clinical setting it is imperative to understand the nuances of the interaction between the clinicians using the technology and the system. Trust has been cited as a potentially significant issue in the effective integration of AI in radiology. Our research pinpoints the factors which have an impact on trust in radiographers and clarifies the strength of these associations. The means of analysis (Boruta) used provides an assessment of any correlations, allowing for intervention to be made. We found that females were less likely to trust AI that males and that trust in AI was negatively correlated with age and level of experience, indicating areas were further investigation and intervention are needed to ensure balanced trust and effective integration of AI in clinical radiography.

Suggested Citation

  • Clare Rainey & Angelina T Villikudathil & Jonathan McConnell & Ciara Hughes & Raymond Bond & Sonyia McFadden, 2023. "An experimental machine learning study investigating the decision-making process of students and qualified radiographers when interpreting radiographic images," PLOS Digital Health, Public Library of Science, vol. 2(10), pages 1-14, October.
  • Handle: RePEc:plo:pdig00:0000229
    DOI: 10.1371/journal.pdig.0000229
    as

    Download full text from publisher

    File URL: https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000229
    Download Restriction: no

    File URL: https://journals.plos.org/digitalhealth/article/file?id=10.1371/journal.pdig.0000229&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pdig.0000229?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Kursa, Miron B. & Rudnicki, Witold R., 2010. "Feature Selection with the Boruta Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 36(i11).
    2. Jérôme Allyn & Nicolas Allou & Pascal Augustin & Ivan Philip & Olivier Martinet & Myriem Belghiti & Sophie Provenchere & Philippe Montravers & Cyril Ferdynus, 2017. "A Comparison of a Machine Learning Model with EuroSCORE II in Predicting Mortality after Elective Cardiac Surgery: A Decision Curve Analysis," PLOS ONE, Public Library of Science, vol. 12(1), pages 1-12, January.
    3. Sabri Boughorbel & Fethi Jarray & Mohammed El-Anbari, 2017. "Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric," PLOS ONE, Public Library of Science, vol. 12(6), pages 1-17, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mirza Rizwan Sajid & Bader A. Almehmadi & Waqas Sami & Mansour K. Alzahrani & Noryanti Muhammad & Christophe Chesneau & Asif Hanif & Arshad Ali Khan & Ahmad Shahbaz, 2021. "Development of Nonlaboratory-Based Risk Prediction Models for Cardiovascular Diseases Using Conventional and Machine Learning Approaches," IJERPH, MDPI, vol. 18(23), pages 1-16, November.
    2. Tong, Jianfeng & Liu, Zhenxing & Zhang, Yong & Zheng, Xiujuan & Jin, Junyang, 2023. "Improved multi-gate mixture-of-experts framework for multi-step prediction of gas load," Energy, Elsevier, vol. 282(C).
    3. Asma Shaheen & Javed Iqbal, 2018. "Spatial Distribution and Mobility Assessment of Carcinogenic Heavy Metals in Soil Profiles Using Geostatistics and Random Forest, Boruta Algorithm," Sustainability, MDPI, vol. 10(3), pages 1-20, March.
    4. Ramón Ferri-García & María del Mar Rueda, 2022. "Variable selection in Propensity Score Adjustment to mitigate selection bias in online surveys," Statistical Papers, Springer, vol. 63(6), pages 1829-1881, December.
    5. Yvan Devaux & Lu Zhang & Andrew I. Lumley & Kanita Karaduzovic-Hadziabdic & Vincent Mooser & Simon Rousseau & Muhammad Shoaib & Venkata Satagopam & Muhamed Adilovic & Prashant Kumar Srivastava & Costa, 2024. "Development of a long noncoding RNA-based machine learning model to predict COVID-19 in-hospital mortality," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    6. Ghosh, Indranil & Chaudhuri, Tamal Datta & Alfaro-Cortés, Esteban & Gámez, Matías & García, Noelia, 2022. "A hybrid approach to forecasting futures prices with simultaneous consideration of optimality in ensemble feature selection and advanced artificial intelligence," Technological Forecasting and Social Change, Elsevier, vol. 181(C).
    7. Yang Zhao & Denise Gorse, 2024. "Earthquake prediction from seismic indicators using tree-based ensemble learning," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 120(3), pages 2283-2309, February.
    8. Conor Waldock & Bernhard Wegscheider & Dario Josi & Bárbara Borges Calegari & Jakob Brodersen & Luiz Jardim de Queiroz & Ole Seehausen, 2024. "Deconstructing the geography of human impacts on species’ natural distribution," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
    9. Manuel J. García Rodríguez & Vicente Rodríguez Montequín & Francisco Ortega Fernández & Joaquín M. Villanueva Balsera, 2019. "Public Procurement Announcements in Spain: Regulations, Data Analysis, and Award Price Estimator Using Machine Learning," Complexity, Hindawi, vol. 2019, pages 1-20, November.
    10. Sangjin Kim & Jong-Min Kim, 2019. "Two-Stage Classification with SIS Using a New Filter Ranking Method in High Throughput Data," Mathematics, MDPI, vol. 7(6), pages 1-16, May.
    11. Baihan Wang & Alfred Pozarickij & Mohsen Mazidi & Neil Wright & Pang Yao & Saredo Said & Andri Iona & Christiana Kartsonaki & Hannah Fry & Kuang Lin & Yiping Chen & Huaidong Du & Daniel Avery & Dan Sc, 2025. "Comparative studies of 2168 plasma proteins measured by two affinity-based platforms in 4000 Chinese adults," Nature Communications, Nature, vol. 16(1), pages 1-13, December.
    12. Arjan S. Gosal & Janine A. McMahon & Katharine M. Bowgen & Catherine H. Hoppe & Guy Ziv, 2021. "Identifying and Mapping Groups of Protected Area Visitors by Environmental Awareness," Land, MDPI, vol. 10(6), pages 1-14, May.
    13. repec:plo:pone00:0185380 is not listed on IDEAS
    14. Foutzopoulos, Giorgos & Pandis, Nikolaos & Tsagris, Michail, 2024. "Predicting full retirement attainment of NBA players," MPRA Paper 121540, University Library of Munich, Germany.
    15. Zhao-Yue Chen & Hervé Petetin & Raúl Fernando Méndez Turrubiates & Hicham Achebak & Carlos Pérez García-Pando & Joan Ballester, 2024. "Population exposure to multiple air pollutants and its compound episodes in Europe," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    16. Schrader, Silja & Graham, Sonia & Campbell, Rebecca & Height, Kaitlyn & Hawkes, Gina, 2024. "Grower attitudes and practices toward area-wide management of cropping weeds in Australia," Land Use Policy, Elsevier, vol. 137(C).
    17. Samuel Asante Gyamerah & Ning Cai, 2021. "Two-Stage Hybrid Machine Learning Model for High-Frequency Intraday Bitcoin Price Prediction Based on Technical Indicators, Variational Mode Decomposition, and Support Vector Regression," Complexity, Hindawi, vol. 2021, pages 1-15, December.
    18. Bram Janssens & Matthias Bogaert & Mathijs Maton, 2023. "Predicting the next Pogačar: a data analytical approach to detect young professional cycling talents," Annals of Operations Research, Springer, vol. 325(1), pages 557-588, June.
    19. Cooray, Upul & Watt, Richard G. & Tsakos, Georgios & Heilmann, Anja & Hariyama, Masanori & Yamamoto, Takafumi & Kuruppuarachchige, Isuruni & Kondo, Katsunori & Osaka, Ken & Aida, Jun, 2021. "Importance of socioeconomic factors in predicting tooth loss among older adults in Japan: Evidence from a machine learning analysis," Social Science & Medicine, Elsevier, vol. 291(C).
    20. Simon Besnard & Nuno Carvalhais & M Altaf Arain & Andrew Black & Benjamin Brede & Nina Buchmann & Jiquan Chen & Jan G P W Clevers & Loïc P Dutrieux & Fabian Gans & Martin Herold & Martin Jung & Yoshik, 2019. "Memory effects of climate and vegetation affecting net ecosystem CO2 fluxes in global forests," PLOS ONE, Public Library of Science, vol. 14(2), pages 1-22, February.
    21. Francesco Sartor & Jonathan P. Moore & Hans-Peter Kubis, 2021. "Plasma Interleukin-10 and Cholesterol Levels May Inform about Interdependences between Fitness and Fatness in Healthy Individuals," IJERPH, MDPI, vol. 18(4), pages 1-19, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pdig00:0000229. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: digitalhealth (email available below). General contact details of provider: https://journals.plos.org/digitalhealth .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.