A critical analysis of multi-criteria models for the prioritisation of health threats

Multi-criteria assessments are increasingly being employed in the prioritisation of health threats, supporting decision processes related to health risk management. The use of multi-criteria analysis in this context is welcome, as it facilitates the consideration of multiple impacts of health threats, it can encompass the use of expert judgment to complement and amalgamate the evidence available, and it permits the modelling of policy makers’ priorities. However, these assessments often lack a clear multi-criteria conceptual framework, in terms of both axiomatic rigour and adequate procedures for preference modelling. Such assessments are ad hoc from a multi-criteria decision analysis perspective, despite the strong health expertise used in constructing these models. In this paper we critically examine some key assumptions and modelling choices made in these assessments, comparing them with the best practices of multi-attribute value analysis. Furthermore, we suggest a set of guidelines on how simulation studies might be employed to assess the impact of these modelling choices. We apply these guidelines to two relevant studies available in the health threat prioritisation domain. We identify severe variability in our simulations due to poor modelling choices, which could cause changes in the ranking of threats being assessed and thus lead to alternative policy recommendations than those suggested in their reports. Our results confirm the importance of carefully designing multi-criteria evaluation models for the prioritisation of health threats.