Fairness as an afterthought: An American perspective on fairness in model developer-clinician user collaborations

Numerous ethics guidelines have been handed down over the last few years on the ethical applications of machine learning models. Virtually every one of them mentions the importance of “fairness” in the development and use of these models. Unfortunately, though, these ethics documents omit providing a consensually adopted definition or characterization of fairness. As one group of authors observed, these documents treat fairness as an “afterthought” whose importance is undeniable but whose essence seems strikingly elusive. In this essay, which offers a distinctly American treatment of “fairness,” we comment on a number of fairness formulations and on qualitative or statistical methods that have been encouraged to achieve fairness. We argue that none of them, at least from an American moral perspective, provides a one-size-fits-all definition of or methodology for securing fairness that could inform or standardize fairness over the universe of use cases witnessing machine learning applications. Instead, we argue that because fairness comprehensions and applications reflect a vast range of use contexts, model developers and clinician users will need to engage in thoughtful collaborations that examine how fairness should be conceived and operationalized in the use case at issue. Part II of this paper illustrates key moments in these collaborations, especially when inter and intra disagreement occurs among model developer and clinician user groups over whether a model is fair or unfair. We conclude by noting that these collaborations will likely occur over the lifetime of a model if its claim to fairness is to advance beyond “afterthought” status.Author summary: This essay has two parts. The first part explains why a universal, all-inclusive definition of fairness that could ethically inform, justify, and standardize the ways machine learning models operationalize fairness has not emerged, at least in the United States. This explains to some degree why prominent healthcare groups that have offered ethical guidelines or recommendations for machine learning development seem to treat fairness as vitally important yet gloss over attempts to define it. The second part of this essay traces the implications of a failure to adopt a one-size-fits-all definition and how that failure can affect the moral contours of the model developer-clinician user relationship. The importance of this conversation is heightened by the fact that machine learning models are virtually unregulated in the United States outside of general safety considerations; no methodological framework for identifying fairness-related issues and incorporating mitigation techniques in machine learning design exists; model developers might not be particularly sensitive towards considering how fairness plays out in their model; and “honest” disagreement can exist between model developers and clinician users over whether a given model is fair or unfair. We conclude by noting that if achieving algorithmic “fairness” is as challenging as we believe it to be, then 1) conceptualizations of fairness will be highly dependent on the specific use case under scrutiny for their content, 2) model developers and clinician users will need to be keenly sensitive as to how fairness impacts patient populations in those cases, and 3) model developers, clinician users, and the populations impacted by the model will need to engage in collaborative efforts throughout the life of their models that aim at operationalizing and realizing justifiable comprehensions and applications of fairness practices.