IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-05252430.html
   My bibliography  Save this paper

A core multi-criteria framework for assessing the performance of policies related to risk: a case study on risk policy for high risk sites

Author

Listed:
  • Scarlett Tannous

    (LAMSADE - Laboratoire d'analyse et modélisation de systèmes pour l'aide à la décision - Université Paris Dauphine-PSL - PSL - Université Paris Sciences et Lettres - CNRS - Centre National de la Recherche Scientifique)

  • Terje Aven

    (University of Stavanger)

  • Myriam Merad

    (LAMSADE - Laboratoire d'analyse et modélisation de systèmes pour l'aide à la décision - Université Paris Dauphine-PSL - PSL - Université Paris Sciences et Lettres - CNRS - Centre National de la Recherche Scientifique)

Abstract

Today most public authorities have implemented some type of risk governance framework or system, which provides structure, approaches, and methods for how to handle societal risks. One main challenge of risk-related policies and frameworks is to adequately and effectively take into account risks. To meet this challenge there is a need for knowledge about the performance of the various policies on risk. However, such knowledge is not easily derived since the performance is subject to uncertainties and is difficult to measure -especially before the occurrence of impacts, which can be significant when focusing on high-risk sites. The present paper discusses this issue of risk policy performance or "effectiveness". The main aim is to establish a set of suitable criteria for assessing the performance of such risk policies by relying on both (1) foundational theoretical and methodological studies and (2) empirical and methodological case studies. Results provide new insights into risk policy assessment by relating the discussion to current risk science knowledge and multi-criteria decision analysis. Consequently, a novel multi-criteria framework consisting of 16 assessment criteria is proposed to cover the spectrum of various improvements, degradations, or stagnations of the effectiveness of risk policies. The discussion is illustrated by a case study on a risk policy linked to a high-risk site in the chemical and petrochemical industry. Based on 25 interviews with risk actors, the study findings reveal how perceived policy improvements or degradations vary under the proposed set of criteria including efficiency, efficacy, and reputational effects. The main contribution demonstrates the practicability of the proposed risk policy assessment framework consisting of 16 criteria. Further research can include the investigation of the Multiple-Criteria Decision Aiding (MCDA) techniques to support the application of this framework.

Suggested Citation

  • Scarlett Tannous & Terje Aven & Myriam Merad, 2025. "A core multi-criteria framework for assessing the performance of policies related to risk: a case study on risk policy for high risk sites," Post-Print hal-05252430, HAL.
    • Handle: RePEc:hal:journl:hal-05252430
    DOI: 10.1016/j.esr.2025.101870
    Note: View the original document on HAL open archive server: https://hal.science/hal-05252430v1
    as

    Download full text from publisher

    File URL: https://hal.science/hal-05252430v1/document
    Download Restriction: no
    File URL: https://libkey.io/10.1016/j.esr.2025.101870?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. A. Markandya & D. W. Pearce, 1998. "Environmental sustainability and cost–benefit analysis," Chapters, in: The Economics of Environment and Development, chapter 4, pages 54-64, Edward Elgar Publishing.
    2. Acheampong, Theophilus & Phimister, Euan & Kemp, Alexander, 2021. "What difference has the Cullen Report made? Empirical analysis of offshore safety regulations in the United Kingdom's oil and gas industry," Energy Policy, Elsevier, vol. 155(C).
    3. Ullah, Mirzat & Umair, Muhammad & Sohag, Kazi & Mariev, Oleg & Khan, Muhammad Asif & Sohail, Hafiz M., 2024. "The connection between disaggregate energy use and export sophistication: New insights from OECD with robust panel estimations," Energy, Elsevier, vol. 306(C).
    4. Liu, Quanlong & Li, Xinchun & Hassall, Maureen, 2021. "Regulatory regime on coal Mine Safety in China and Australia: Comparative analysis and overall findings," Resources Policy, Elsevier, vol. 74(C).
    5. Sen, Amartya Kumar, 2000. "The Discipline of Cost†Benefit Analysis," Scholarly Articles 3444801, Harvard University Department of Economics.
    6. Acheampong, Theophilus & Akumperigya, Rainer, 2018. "Offshore risk regulation: A comparative analysis of regulatory framework in Ghana, the United Kingdom and Norway," Energy Policy, Elsevier, vol. 113(C), pages 701-710.
    7. Kirsti Russell Vastveit & Kerstin Eriksson & Ove Njå, 2014. "Critical reflections on municipal risk and vulnerability analyses as decision support tools: the role of regulation regimes," Environment Systems and Decisions, Springer, vol. 34(3), pages 443-455, September.
    8. Aven, Terje & Renn, Ortwin, 2018. "Improving government policy on risk: Eight key principles," Reliability Engineering and System Safety, Elsevier, vol. 176(C), pages 230-241.
    9. Jones-Lee, M W & Hammerton, M & Philips, P R, 1985. "The Value of Safety: Results of a National Sample Survey," Economic Journal, Royal Economic Society, vol. 95(377), pages 49-72, March.
    10. Sen, Amartya, 2000. "The Discipline of Cost-Benefit Analysis," The Journal of Legal Studies, University of Chicago Press, vol. 29(2), pages 931-952, June.
    11. Corinne Vitale & Sander Meijerink, 2023. "Flood risk policies in Italy: a longitudinal institutional analysis of continuity and change," International Journal of Water Resources Development, Taylor & Francis Journals, vol. 39(2), pages 211-235, March.
    12. Mazzocchi, Mario & Ragona, Maddalena & Zanoli, Agostina, 2013. "A fuzzy multi-criteria approach for the ex-ante impact assessment of food safety policies," Food Policy, Elsevier, vol. 38(C), pages 177-189.
    13. Scarlett Tannous & Myriam Merad & Jan Hayes, 2024. "A comparative analysis of risk prevention policy tools and governance structures in Normandy (France) and Victoria (Australia): Assessing policies for high-risk sites," Post-Print hal-04877609, HAL.
    14. Terje Aven & Enrico Zio, 2014. "Foundational Issues in Risk Assessment and Risk Management," Risk Analysis, John Wiley & Sons, vol. 34(7), pages 1164-1172, July.
    15. Jones-Lee, M. & Aven, T., 2009. "The role of social cost–benefit analysis in societal decision-making under large uncertainties with application to robbery at a cash depot," Reliability Engineering and System Safety, Elsevier, vol. 94(12), pages 1954-1961.
    16. Elisabeth Paté‐Cornell & Louis Anthony Cox, 2014. "Improving Risk Management: From Lame Excuses to Principled Practice," Risk Analysis, John Wiley & Sons, vol. 34(7), pages 1228-1239, July.
    17. Nagel, Stuart S., 1990. "Bridging theory and practice in policy/program evaluation," Evaluation and Program Planning, Elsevier, vol. 13(3), pages 275-283, January.
    18. Scarlett Tannous & David Javier Castro Rodriguez & Myriam Merad & Micaela Demichela, 2025. "Risk policy tools for high-risk industrial sites in Normandy (France) and Piedmont (Italy): more hazards-focused than vulnerabilities-focused," Journal of Risk Research, Taylor & Francis Journals, vol. 28(1), pages 78-104, January.
    19. Ward, E. John & Dimitriou, Harry T. & Dean, Marco, 2016. "Theory and background of multi-criteria analysis: Toward a policy-led approach to mega transport infrastructure project appraisal," Research in Transportation Economics, Elsevier, vol. 58(C), pages 21-45.
    20. Marra, Mita, 2018. "The ambiguities of performance-based governance reforms in Italy: Reviving the fortunes of evaluation and performance measurement," Evaluation and Program Planning, Elsevier, vol. 69(C), pages 173-182.
    21. Scarlett Tannous & David Javier Castro Rodriguez & Myriam Merad & Micaela Demichela, 2025. "Risk policy tools for high-risk industrial sites in Normandy (France) and Piedmont (Italy): more hazards-focused than vulnerabilities-focused," Post-Print hal-05197567, HAL.
    22. E. Y. Wong & R. A. Ponce & S. Farrow & S. M. Bartell & R. C. Lee & E. M. Faustman, 2003. "Comparative Risk and Policy Analysis in Environmental Health," Risk Analysis, John Wiley & Sons, vol. 23(6), pages 1337-1349, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Derek Linton, 2024. "The Uncertainty Problem in Cost-Benefit Analysis Expanded: A Current Review," Journal of Economic Impact, Science Impact Publishers, vol. 6(1), pages 21-26.
    2. Adélie Ranville & Marcos Barros, 2022. "Towards Normative Theories of Social Entrepreneurship. A Review of the Top Publications of the Field," Journal of Business Ethics, Springer, vol. 180(2), pages 407-438, October.
    3. Cheney Shreve, 2016. "Economic Efficiency or Gender Equality: Conceptualizing an Equitable “Social Framing” for Economic Evaluations to Support Gender Equality in Disaster Risk- and Environmental-Management Decision-Making," Resources, MDPI, vol. 5(3), pages 1-16, July.
    4. Paul Koster, 2023. "Counting what counts: Moral considerations and market surplus," Tinbergen Institute Discussion Papers 23-008/VIII, Tinbergen Institute.
    5. Skott, Peter & Davis, Leila, 2013. "Distributional biases in the analysis of climate change," Ecological Economics, Elsevier, vol. 85(C), pages 188-197.
    6. Aven, Terje, 2020. "Three influential risk foundation papers from the 80s and 90s: Are they still state-of-the-art?," Reliability Engineering and System Safety, Elsevier, vol. 193(C).
    7. De Brucker, Klaas & Macharis, Cathy & Verbeke, Alain, 2013. "Multi-criteria analysis and the resolution of sustainable development dilemmas: A stakeholder management approach," European Journal of Operational Research, Elsevier, vol. 224(1), pages 122-131.
    8. Heidi Peterson, 2023. "Cost–Benefit Analysis (CBA) or the Highway? An Alternative Road to Investigating the Value for Money of International Development Research," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 35(2), pages 260-280, April.
    9. Mouter, Niek & Koster, Paul & Dekker, Thijs, 2021. "Contrasting the recommendations of participatory value evaluation and cost-benefit analysis in the context of urban mobility investments," Transportation Research Part A: Policy and Practice, Elsevier, vol. 144(C), pages 54-73.
    10. Revesz, Richard & Stavins, Robert, 2004. "Environmental Law and Policy," Working Paper Series rwp04-023, Harvard University, John F. Kennedy School of Government.
    11. te Boveldt, Geert & Keseru, Imre & Macharis, Cathy, 2022. "When monetarisation and ranking are not appropriate. A novel stakeholder-based appraisal method," Transportation Research Part A: Policy and Practice, Elsevier, vol. 156(C), pages 192-205.
    12. Öberg, Christina & Huge-Brodin, Maria & Björklund, Maria, 2012. "Applying a network level in environmental impact assessments," Journal of Business Research, Elsevier, vol. 65(2), pages 247-255.
    13. Will, Matthias Georg, 2011. "Technologischer Fortschritt und Vertrauen: Gefahrenproduktivität und Bindungsmechanismen zur Überwindung von Konflikten," Discussion Papers 2011-19, Martin Luther University of Halle-Wittenberg, Chair of Economic Ethics.
    14. Irene Bayiyana & Anton Bua & Alfred Ozimati & Johnny Mugisha & John Colvin & Christopher Abu Omongo, 2023. "Insecticide Use by Small-Scale Ugandan Cassava Growers: An Economic Analysis," Agriculture, MDPI, vol. 13(5), pages 1-17, May.
    15. S. Scrieciu & Valerie Belton & Zaid Chalabi & Reinhard Mechler & Daniel Puig, 2014. "Advancing methodological thinking and practice for development-compatible climate policy planning," Mitigation and Adaptation Strategies for Global Change, Springer, vol. 19(3), pages 261-288, March.
    16. Mouter, Niek & Annema, Jan Anne & van Wee, Bert, 2013. "Ranking the substantive problems in the Dutch Cost–Benefit Analysis practice," Transportation Research Part A: Policy and Practice, Elsevier, vol. 49(C), pages 241-255.
    17. Scarlett Tannous & Myriam Merad & Jan Hayes, 2024. "A comparative analysis of risk prevention policy tools and governance structures in Normandy (France) and Victoria (Australia): Assessing policies for high-risk sites," Post-Print hal-04877609, HAL.
    18. Mouter, Niek & Annema, Jan Anne & Wee, Bert van, 2013. "Attitudes towards the role of Cost–Benefit Analysis in the decision-making process for spatial-infrastructure projects: A Dutch case study," Transportation Research Part A: Policy and Practice, Elsevier, vol. 58(C), pages 1-14.
    19. Vredin Johansson, Maria & Heldt, Tobias & Johansson, Per, 2006. "The effects of attitudes and personality traits on mode choice," Transportation Research Part A: Policy and Practice, Elsevier, vol. 40(6), pages 507-525, July.
    20. Marcela Bindzarova Gergelova & Martina Zelenakova & Maria Hlinkova & Hany F. Abd-Elhamid, 2025. "Towards a Robust Framework for Navigating Flood-Related Challenges: A Comprehensive Proposal for an Advanced Flood Risk Assessment Scale in the Slovak Republic," Land, MDPI, vol. 14(9), pages 1-20, August.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-05252430. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.