Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impact

Despite significant technical advances in machine learning (ML) over the past several years, the tangible impact of this technology in healthcare has been limited. This is due not only to the particular complexities of healthcare, but also due to structural issues in the machine learning for healthcare (MLHC) community which broadly reward technical novelty over tangible, equitable impact. We structure our work as a healthcare-focused echo of the 2012 paper “Machine Learning that Matters”, which highlighted such structural issues in the ML community at large, and offered a series of clearly defined “Impact Challenges” to which the field should orient itself. Drawing on the expertise of a diverse and international group of authors, we engage in a narrative review and examine issues in the research background environment, training processes, evaluation metrics, and deployment protocols which act to limit the real-world applicability of MLHC. Broadly, we seek to distinguish between machine learning ON healthcare data and machine learning FOR healthcare—the former of which sees healthcare as merely a source of interesting technical challenges, and the latter of which regards ML as a tool in service of meeting tangible clinical needs. We offer specific recommendations for a series of stakeholders in the field, from ML researchers and clinicians, to the institutions in which they work, and the governments which regulate their data access.Author summary: The field of machine learning has made significant technical advancements over the past several years, but the impact of this technology on healthcare practice has remained limited. We identify issues in the structure of the field of machine learning for healthcare which incentivise work that is scientifically novel over work that ultimately impacts patients. Among others, these issues include a lack of diversity in available data, an emphasis on targets which are easy to measure but may not be clinically important, and limited funding for work focused on deployment. We offer a series of suggestions about how best to address these issues, and advocate for a distinction to be made between “machine research performed ON healthcare data” and true “machine FOR healthcare”. The latter, we argue, requires starting from the very beginning with a focus on the impact that a model will have on patients. We conclude with discussion of “impact challenges”—specific and measurable goals with an emphasis upon health equity and broad community impact—as examples of the types of goals the field should strive toward.