Using Natural Language Processing (NLP) to assess changes in transdisciplinary understandings across a large research consortium

Researching how to create just and sustainable futures for all requires innovative and mission-oriented transdisciplinary approaches. This article sets out our approach and findings following the assessment, using Natural Language Processing (NLP), of the development in transdisciplinary understandings across a large and newly-forming research consortium as it changed over the six-year period of delivery (2019–2025). Two outcomes were sought from this assessment: i) learning to improve the transdisciplinary working in new, large research teams aiming to solve complex global challenges; and ii) improvement of use of NLP for this kind of exercise. A research-on-research workstream provided qualitative data from 63 semi-structured interviews carried out with 39 consortium members, with three rounds of interview being conducted in Years 2, 3 and 5 of the programme. Drawing on this data, a ‘dual approach’ to NLP was used to assess changes over time and by discipline: i) co-word clustering of full transcripts; and ii) analysis of vocabulary identified as transdisciplinary. The co-word clustering identified nine themes, but no clear findings. However, the analysis of transdisciplinary vocabulary indicated a clear convergence across disciplinary areas over six years revealing both a small set of useful, jargon-free words, and the time it takes for teams to build that common language. The results and analysis indicate potential efficacy if certain conditions are made possible, such as data availability, but significant limitations were identified that would need addressing. The approach was labour intensive, though future iterations should be much less so if the limitations can be addressed.Author summary: Creating just and sustainable futures requires large and often newly forming research teams that bring together a wide range of experts and stakeholders. While these types of collaborative approaches offer the potential for comprehensive problem-solving, they also present significant management and communication challenges. Evaluating the quality of inter- and transdisciplinary working has proved particularly challenging, not least in terms of enabling real time feedback. We wanted to know whether it might be possible to use Natural Language Processing to assess progress in shared understandings in real time. We drew on three rounds of interview data gathered over five years from a large UK-based research consortium by a dedicated in-house research-on-research sub-team. By feeding in a list of inter- and transdisciplinary words, the analysis show convergence over time across the different disciplines towards a smaller number of jargon free words. This suggests it takes more time than may be expected for a shared language to evolve. Co-word cluster analysis did not reveal any strong findings, but this may have been due to the relative low volume of data. Further investigation of larger data sats may prove more useful, and is worth testing given the potential time-savings this might enable.