IDEAS home Printed from https://ideas.repec.org/a/spr/climat/v153y2019i1d10.1007_s10584-018-2355-3.html
   My bibliography  Save this article

How to evaluate a monitoring system for adaptive policies: criteria for signposts selection and their model-based evaluation

Author

Listed:
  • Luciano Raso

    (Delft University of Technology)

  • Jan Kwakkel

    (Delft University of Technology)

  • Jos Timmermans

    (Delft University of Technology)

  • Geremy Panthou

    (Université Grenoble-Alpes)

Abstract

Adaptive policies have emerged as a valuable strategy for dealing with uncertainties by recognising the capacity of systems to adapt over time to new circumstances and surprises. The efficacy of adaptive policies hinges on detecting on-going change and ensuring that actions are indeed taken if and when necessary. This is operationalised by including a monitoring system composed of signposts and triggers in the design of the plan. A well-designed monitoring system is indispensable for the effective implementation of adaptive policies. Despite the importance of monitoring for adaptive policies, the present literature has not considered criteria enabling the a-priori evaluation of the efficacy of signposts. In this paper, we introduce criteria for the evaluation of individual signposts and the monitoring system as a whole. These criteria are relevance, observability, completeness, and parsimony. These criteria are intended to enhance the capacity to detect the need for adaptation in the presence of noisy and ambiguous observations of the real system. The criteria are identified from an analysis of the information chain, from system observations to policy success, focusing on how data becomes information. We illustrate how models, in particular, the combined use of stochastic and exploratory modelling can be used to assess individual signposts, and the whole monitoring system according to these criteria. This analysis provides significant insight into critical factors that may hinder learning from data. The proposed criteria are demonstrated using a hypothetical case, in which a monitoring system for a flood protection policy in the Niger River is designed and tested.

Suggested Citation

  • Luciano Raso & Jan Kwakkel & Jos Timmermans & Geremy Panthou, 2019. "How to evaluate a monitoring system for adaptive policies: criteria for signposts selection and their model-based evaluation," Climatic Change, Springer, vol. 153(1), pages 267-283, March.
    • Handle: RePEc:spr:climat:v:153:y:2019:i:1:d:10.1007_s10584-018-2355-3
    DOI: 10.1007/s10584-018-2355-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10584-018-2355-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10584-018-2355-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Michelle Woodward & Zoran Kapelan & Ben Gouldby, 2014. "Adaptive Flood Risk Management Under Climate Change Uncertainty Using Real Options and Optimization," Risk Analysis, John Wiley & Sons, vol. 34(1), pages 75-92, January.
    2. Marjolijn Haasnoot & Hans Middelkoop & Astrid Offermans & Eelco Beek & Willem Deursen, 2012. "Exploring pathways for sustainable water management in river deltas in a changing environment," Climatic Change, Springer, vol. 115(3), pages 795-819, December.
    3. Robert J. Lempert & David G. Groves & Steven W. Popper & Steve C. Bankes, 2006. "A General, Analytic Method for Generating Robust Strategies and Narrative Scenarios," Management Science, INFORMS, vol. 52(4), pages 514-528, April.
    4. Robert L. Ceres & Chris E. Forest & Klaus Keller, 2017. "Understanding the detectability of potential changes to the 100-year peak storm surge," Climatic Change, Springer, vol. 145(1), pages 221-235, November.
    5. Sterman, J.D., 2006. "Learning from evidence in a complex world," American Journal of Public Health, American Public Health Association, vol. 96(3), pages 505-514.
    6. Abdul Tariq & Robert Jay Lempert & John Riverson & Marla Schwartz & Neil Berg, 2017. "A climate stress test of Los Angeles’ water quality plans," Climatic Change, Springer, vol. 144(4), pages 625-639, October.
    7. Christopher A. Sims, 1982. "Policy Analysis with Econometric Models," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 13(1), pages 107-164.
    8. David McInerney & Robert Lempert & Klaus Keller, 2012. "What are robust strategies in the face of uncertain climate threshold responses?," Climatic Change, Springer, vol. 112(3), pages 547-568, June.
    9. Blyth, William & Bradley, Richard & Bunn, Derek & Clarke, Charlie & Wilson, Tom & Yang, Ming, 2007. "Investment risks under uncertain climate change policy," Energy Policy, Elsevier, vol. 35(11), pages 5766-5773, November.
    10. Steve Bankes, 1993. "Exploratory Modeling for Policy Analysis," Operations Research, INFORMS, vol. 41(3), pages 435-449, June.
    11. L. Raso & S. V. Weijs & M. Werner, 2018. "Balancing Costs and Benefits in Selecting New Information: Efficient Monitoring Using Deterministic Hydro-economic Models," Water Resources Management: An International Journal, Published for the European Water Resources Association (EWRA), Springer;European Water Resources Association (EWRA), vol. 32(1), pages 339-357, January.
    12. Kwakkel, Jan H. & Pruyt, Erik, 2013. "Exploratory Modeling and Analysis, an approach for model-based foresight under deep uncertainty," Technological Forecasting and Social Change, Elsevier, vol. 80(3), pages 419-431.
    13. Hamarat, Caner & Kwakkel, Jan H. & Pruyt, Erik, 2013. "Adaptive Robust Design under deep uncertainty," Technological Forecasting and Social Change, Elsevier, vol. 80(3), pages 408-418.
    14. Walker, Warren E. & Rahman, S. Adnan & Cave, Jonathan, 2001. "Adaptive policies, policy analysis, and policy-making," European Journal of Operational Research, Elsevier, vol. 128(2), pages 282-289, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Naomi Radke & Klaus Keller & Rasoul Yousefpour & Marc Hanewinkel, 2020. "Identifying decision-relevant uncertainties for dynamic adaptive forest management under climate change," Climatic Change, Springer, vol. 163(2), pages 891-911, November.
    2. Zamora-Pereira, Juan Carlos & Hanewinkel, Marc & Yousefpour, Rasoul, 2023. "Robust management strategies promoting ecological resilience and economic efficiency of a mixed conifer-broadleaf forest in Southwest Germany under the risk of severe drought," Ecological Economics, Elsevier, vol. 209(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luciano Raso & Jan Kwakkel & Jos Timmermans, 2019. "Assessing the Capacity of Adaptive Policy Pathways to Adapt on Time by Mapping Trigger Values to Their Outcomes," Sustainability, MDPI, vol. 11(6), pages 1-16, March.
    2. Jan Kwakkel & Marjolijn Haasnoot & Warren Walker, 2015. "Developing dynamic adaptive policy pathways: a computer-assisted approach for developing adaptive strategies for a deeply uncertain world," Climatic Change, Springer, vol. 132(3), pages 373-386, October.
    3. Erik Pruyt & Jan H. Kwakkel, 2014. "Radicalization under deep uncertainty: a multi-model exploration of activism, extremism, and terrorism," System Dynamics Review, System Dynamics Society, vol. 30(1-2), pages 1-28, January.
    4. Hidayatno, Akhmad & Jafino, Bramka Arga & Setiawan, Andri D. & Purwanto, Widodo Wahyu, 2020. "When and why does transition fail? A model-based identification of adoption barriers and policy vulnerabilities for transition to natural gas vehicles," Energy Policy, Elsevier, vol. 138(C).
    5. Lempert Robert J., 2014. "Embedding (some) benefit-cost concepts into decision support processes with deep uncertainty," Journal of Benefit-Cost Analysis, De Gruyter, vol. 5(3), pages 487-514, December.
    6. Eker, Sibel & van Daalen, Els, 2015. "A model-based analysis of biomethane production in the Netherlands and the effectiveness of the subsidization policy under uncertainty," Energy Policy, Elsevier, vol. 82(C), pages 178-196.
    7. Moallemi, Enayat A. & Elsawah, Sondoss & Ryan, Michael J., 2020. "Robust decision making and Epoch–Era analysis: A comparison of two robustness frameworks for decision-making under uncertainty," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    8. Steinmann, Patrick & Auping, Willem L. & Kwakkel, Jan H., 2020. "Behavior-based scenario discovery using time series clustering," Technological Forecasting and Social Change, Elsevier, vol. 156(C).
    9. Warren E. Walker & Marjolijn Haasnoot & Jan H. Kwakkel, 2013. "Adapt or Perish: A Review of Planning Approaches for Adaptation under Deep Uncertainty," Sustainability, MDPI, vol. 5(3), pages 1-25, March.
    10. Auping, Willem L. & Pruyt, Erik & de Jong, Sijbren & Kwakkel, Jan H., 2016. "The geopolitical impact of the shale revolution: Exploring consequences on energy prices and rentier states," Energy Policy, Elsevier, vol. 98(C), pages 390-399.
    11. Sandrine Mathy & Patrick Criqui & Katharina Knoop & Manfred Fischedick & Sascha Samadi, 2016. "Uncertainty management and the dynamic adjustment of deep decarbonization pathways," Climate Policy, Taylor & Francis Journals, vol. 16(sup1), pages 47-62, June.
    12. Dittrich, Ruth & Wreford, Anita & Moran, Dominic, 2016. "A survey of decision-making approaches for climate change adaptation: Are robust methods the way forward?," Ecological Economics, Elsevier, vol. 122(C), pages 79-89.
    13. Moallemi, Enayat A. & Elsawah, Sondoss & Ryan, Michael J., 2020. "Strengthening ‘good’ modelling practices in robust decision support: A reporting guideline for combining multiple model-based methods," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 175(C), pages 3-24.
    14. Erik Pruyt & Tushith Islam, 2015. "On generating and exploring the behavior space of complex models," System Dynamics Review, System Dynamics Society, vol. 31(4), pages 220-249, October.
    15. Jim W. Hall & Robert J. Lempert & Klaus Keller & Andrew Hackbarth & Christophe Mijere & David J. McInerney, 2012. "Robust Climate Policies Under Uncertainty: A Comparison of Robust Decision Making and Info‐Gap Methods," Risk Analysis, John Wiley & Sons, vol. 32(10), pages 1657-1672, October.
    16. Hamarat, Caner & Kwakkel, Jan H. & Pruyt, Erik, 2013. "Adaptive Robust Design under deep uncertainty," Technological Forecasting and Social Change, Elsevier, vol. 80(3), pages 408-418.
    17. Ram, Camelia, 2020. "Scenario presentation and scenario generation in multi-criteria assessments: An exploratory study," Technological Forecasting and Social Change, Elsevier, vol. 151(C).
    18. Junyu Zhang & Dafang Fu & Christian Urich & Rajendra Prasad Singh, 2018. "Accelerated Exploration for Long-Term Urban Water Infrastructure Planning through Machine Learning," Sustainability, MDPI, vol. 10(12), pages 1-16, December.
    19. Jan H. Kwakkel, 2019. "A generalized many‐objective optimization approach for scenario discovery," Futures & Foresight Science, John Wiley & Sons, vol. 1(2), June.
    20. Baustert, Paul & Othoniel, Benoit & Rugani, Benedetto & Leopold, Ulrich, 2018. "Uncertainty analysis in integrated environmental models for ecosystem service assessments: Frameworks, challenges and gaps," Ecosystem Services, Elsevier, vol. 33(PB), pages 110-123.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:climat:v:153:y:2019:i:1:d:10.1007_s10584-018-2355-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.