A core multi-criteria framework for assessing the performance of policies related to risk: a case study on risk policy for high risk sites

Today most public authorities have implemented some type of risk governance framework or system, which provides structure, approaches, and methods for how to handle societal risks. One main challenge of risk-related policies and frameworks is to adequately and effectively take into account risks. To meet this challenge there is a need for knowledge about the performance of the various policies on risk. However, such knowledge is not easily derived since the performance is subject to uncertainties and is difficult to measure -especially before the occurrence of impacts, which can be significant when focusing on high-risk sites. The present paper discusses this issue of risk policy performance or "effectiveness". The main aim is to establish a set of suitable criteria for assessing the performance of such risk policies by relying on both (1) foundational theoretical and methodological studies and (2) empirical and methodological case studies. Results provide new insights into risk policy assessment by relating the discussion to current risk science knowledge and multi-criteria decision analysis. Consequently, a novel multi-criteria framework consisting of 16 assessment criteria is proposed to cover the spectrum of various improvements, degradations, or stagnations of the effectiveness of risk policies. The discussion is illustrated by a case study on a risk policy linked to a high-risk site in the chemical and petrochemical industry. Based on 25 interviews with risk actors, the study findings reveal how perceived policy improvements or degradations vary under the proposed set of criteria including efficiency, efficacy, and reputational effects. The main contribution demonstrates the practicability of the proposed risk policy assessment framework consisting of 16 criteria. Further research can include the investigation of the Multiple-Criteria Decision Aiding (MCDA) techniques to support the application of this framework.