IDEAS home Printed from https://ideas.repec.org/a/spr/climat/v166y2021i1d10.1007_s10584-021-03099-9.html
   My bibliography  Save this article

Evaluating process-based integrated assessment models of climate change mitigation

Author

Listed:
  • Charlie Wilson

    (University of East Anglia
    International Institute for Applied Systems Analysis (IIASA))

  • Céline Guivarch

    (Centre International de Recherche sur l’Environnement et le Developpement (CIRED), Ecole des Ponts ParisTech)

  • Elmar Kriegler

    (Potsdam Institute for Climate Impact Research, Member of The Leibniz Association
    University of Potsdam)

  • Bas Ruijven

    (International Institute for Applied Systems Analysis (IIASA))

  • Detlef P. Vuuren

    (PBL Netherlands Environmental Assessment Agency
    Utrecht University)

  • Volker Krey

    (International Institute for Applied Systems Analysis (IIASA))

  • Valeria Jana Schwanitz

    (Western Norway University of Applied Sciences
    The Schumacher Institute)

  • Erica L. Thompson

    (London School of Economics and Political Science)

Abstract

Process-based integrated assessment models (IAMs) project long-term transformation pathways in energy and land-use systems under what-if assumptions. IAM evaluation is necessary to improve the models’ usefulness as scientific tools applicable in the complex and contested domain of climate change mitigation. We contribute the first comprehensive synthesis of process-based IAM evaluation research, drawing on a wide range of examples across six different evaluation methods including historical simulations, stylised facts, and model diagnostics. For each evaluation method, we identify progress and milestones to date, and draw out lessons learnt as well as challenges remaining. We find that each evaluation method has distinctive strengths, as well as constraints on its application. We use these insights to propose a systematic evaluation framework combining multiple methods to establish the appropriateness, interpretability, credibility, and relevance of process-based IAMs as useful scientific tools for informing climate policy. We also set out a programme of evaluation research to be mainstreamed both within and outside the IAM community.

Suggested Citation

  • Charlie Wilson & Céline Guivarch & Elmar Kriegler & Bas Ruijven & Detlef P. Vuuren & Volker Krey & Valeria Jana Schwanitz & Erica L. Thompson, 2021. "Evaluating process-based integrated assessment models of climate change mitigation," Climatic Change, Springer, vol. 166(1), pages 1-22, May.
  • Handle: RePEc:spr:climat:v:166:y:2021:i:1:d:10.1007_s10584-021-03099-9
    DOI: 10.1007/s10584-021-03099-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10584-021-03099-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s10584-021-03099-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Mathijs Harmsen & Detlef Vuuren & Maarten Berg & Andries Hof & Chris Hope & Volker Krey & Jean-Francois Lamarque & Adriana Marcucci & Drew Shindell & Michiel Schaeffer, 2015. "How well do integrated assessment models represent non-CO 2 radiative forcing?," Climatic Change, Springer, vol. 133(4), pages 565-582, December.
    2. Fujimori, Shinichiro & Dai, Hancheng & Masui, Toshihiko & Matsuoka, Yuzuru, 2016. "Global energy model hindcasting," Energy, Elsevier, vol. 114(C), pages 293-301.
    3. Iyer, Gokul & Hultman, Nathan & Eom, Jiyong & McJeon, Haewon & Patel, Pralit & Clarke, Leon, 2015. "Diffusion of low-carbon technologies and the feasibility of long-term climate targets," Technological Forecasting and Social Change, Elsevier, vol. 90(PA), pages 103-118.
    4. Kriegler, Elmar & Petermann, Nils & Krey, Volker & Schwanitz, Valeria Jana & Luderer, Gunnar & Ashina, Shuichi & Bosetti, Valentina & Eom, Jiyong & Kitous, Alban & Méjean, Aurélie & Paroussos, Leonida, 2015. "Diagnostic indicators for integrated assessment models of climate policy," Technological Forecasting and Social Change, Elsevier, vol. 90(PA), pages 45-61.
    5. McJeon, Haewon C. & Clarke, Leon & Kyle, Page & Wise, Marshall & Hackbarth, Andrew & Bryant, Benjamin P. & Lempert, Robert J., 2011. "Technology interactions among low-carbon energy technologies: What can we learn from a large number of scenarios?," Energy Economics, Elsevier, vol. 33(4), pages 619-631, July.
    6. Chris Hope & Mat Hope, 2013. "The social cost of CO2 in a low-growth world," Nature Climate Change, Nature, vol. 3(8), pages 722-724, August.
    7. DeCarolis, Joseph & Daly, Hannah & Dodds, Paul & Keppo, Ilkka & Li, Francis & McDowall, Will & Pye, Steve & Strachan, Neil & Trutnevyte, Evelina & Usher, Will & Winning, Matthew & Yeh, Sonia & Zeyring, 2017. "Formalizing best practice for energy system optimization modelling," Applied Energy, Elsevier, vol. 194(C), pages 184-198.
    8. Zoi Vrontisi & Gunnar Luderer & Bert Saveyn & Kimon Keramidas & Lara Aleluia Reis & Lavinia Baumstark & Christoph Bertram & Harmen Sytze de Boer & Laurent Drouet & Kostas Fragkiadakis & Oliver Fricko , 2018. "Enhancing global climate policy ambition towards a 1.5 °C stabilization: a short-term multi-model assessment," Post-Print halshs-01782274, HAL.
    9. Guivarch, Céline & Hallegatte, Stéphane & Crassous, Renaud, 2009. "The resilience of the Indian economy to rising oil prices as a validation test for a global energy-environment-economy CGE model," Energy Policy, Elsevier, vol. 37(11), pages 4259-4266, November.
    10. Gaskins, Darius W, Jr & Weyant, John P, 1993. "Model Comparisons of the Costs of Reducing CO2 Emissions," American Economic Review, American Economic Association, vol. 83(2), pages 318-323, May.
    11. Sibel Eker & Elena Rovenskaya & Michael Obersteiner & Simon Langan, 2018. "Practice and perspectives in the validation of resource management models," Nature Communications, Nature, vol. 9(1), pages 1-10, December.
    12. Richard Rosen & Edeltraud Guenther, 2014. "The Economics of Mitigating Climate Change?," Challenge, Taylor & Francis Journals, vol. 57(4), pages 57-81.
    13. Henri Waisman & Céline Guivarch & Fabio Grazi & Jean Hourcade, 2012. "The I maclim-R model: infrastructures, technical inertia and the costs of low carbon futures under imperfect foresight," Climatic Change, Springer, vol. 114(1), pages 101-120, September.
    14. Thompson, Erica L. & Smith, Leonard A., 2019. "Escape from model-land," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 13, pages 1-17.
    15. Michael Greenstone & Elizabeth Kopits & Ann Wolverton, 2013. "Developing a Social Cost of Carbon for US Regulatory Analysis: A Methodology and Interpretation," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 7(1), pages 23-46, January.
    16. Huntington, Hillard G & Weyant, John P & Sweeney, James L, 1982. "Modeling for insights, not numbers: the experiences of the energy modeling forum," Omega, Elsevier, vol. 10(5), pages 449-462.
    17. Nicholas Stern, 2016. "Economics: Current climate models are grossly misleading," Nature, Nature, vol. 530(7591), pages 407-409, February.
    18. Maurizio Gargiulo & Brian Ó Gallachóir, 2013. "Long-term energy models: Principles, characteristics, focus, and limitations," Wiley Interdisciplinary Reviews: Energy and Environment, Wiley Blackwell, vol. 2(2), pages 158-177, March.
    19. Frances C. Moore & Delavane B. Diaz, 2015. "Temperature impacts on economic growth warrant stringent mitigation policy," Nature Climate Change, Nature, vol. 5(2), pages 127-131, February.
    20. Nebojsa Nakicenovic & Robert Lempert & Anthony Janetos, 2014. "A Framework for the Development of New Socio-economic Scenarios for Climate Change Research: Introductory Essay," Climatic Change, Springer, vol. 122(3), pages 351-361, February.
    21. Alan Manne & Richard Richels, 1992. "Buying Greenhouse Insurance: The Economic Costs of CO2 Emission Limits," MIT Press Books, The MIT Press, edition 1, volume 1, number 026213280x, December.
    22. David Anthoff & Richard Tol, 2013. "The uncertainty about the social cost of carbon: A decomposition analysis using fund," Climatic Change, Springer, vol. 117(3), pages 515-530, April.
    23. Borgonovo, E., 2010. "Sensitivity analysis with finite changes: An application to modified EOQ models," European Journal of Operational Research, Elsevier, vol. 200(1), pages 127-138, January.
    24. Michael Oppenheimer & Christopher M. Little & Roger M. Cooke, 2016. "Expert judgement and uncertainty quantification for climate change," Nature Climate Change, Nature, vol. 6(5), pages 445-451, May.
    25. John Weyant, 2017. "Some Contributions of Integrated Assessment Models of Global Climate Change," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 11(1), pages 115-137.
    26. Rosen, Richard A. & Guenther, Edeltraud, 2015. "The economics of mitigating climate change: What can we know?," Technological Forecasting and Social Change, Elsevier, vol. 91(C), pages 93-106.
    27. L. Drouet & V. Bosetti & M. Tavoni, 2015. "Selection of climate policies under the uncertainties in the Fifth Assessment Report of the IPCC," Nature Climate Change, Nature, vol. 5(10), pages 937-940, October.
    28. Chaturvedi, Vaibhav & Kim, Sonny & Smith, Steven J. & Clarke, Leon & Yuyu, Zhou & Kyle, Page & Patel, Pralit, 2013. "Model evaluation and hindcasting: An experiment with an integrated assessment model," Energy, Elsevier, vol. 61(C), pages 479-490.
    29. Detlef Vuuren & Jason Lowe & Elke Stehfest & Laila Gohar & Andries Hof & Chris Hope & Rachel Warren & Malte Meinshausen & Gian-Kasper Plattner, 2011. "How well do integrated assessment models simulate climate change?," Climatic Change, Springer, vol. 104(2), pages 255-285, January.
    30. Jonathan Koomey & Paul Craig & Ashok Gadgil & David Lorenzetti, 2003. "Improving Long-Range Energy Modeling: A Plea for Historical Retrospectives," The Energy Journal, International Association for Energy Economics, vol. 0(Number 4), pages 75-92.
    31. Wilkerson, Jordan T. & Leibowicz, Benjamin D. & Turner, Delavane D. & Weyant, John P., 2015. "Comparison of integrated assessment models: Carbon price impacts on U.S. energy," Energy Policy, Elsevier, vol. 76(C), pages 18-31.
    32. Schwanitz, Valeria Jana & Wierling, August, 2016. "Offshore wind investments – Realism about cost developments is necessary," Energy, Elsevier, vol. 106(C), pages 170-181.
    33. Trutnevyte, Evelina, 2016. "Does cost optimization approximate the real-world energy transition?," Energy, Elsevier, vol. 106(C), pages 182-193.
    34. Edmonds, Jae & Reilly, John, 1983. "A long-term global energy- economic model of carbon dioxide release from fossil fuel use," Energy Economics, Elsevier, vol. 5(2), pages 74-88, April.
    35. David Anthoff & Richard Tol, 2013. "Erratum to: The uncertainty about the social cost of carbon: A decomposition analysis using fund," Climatic Change, Springer, vol. 121(2), pages 413-413, November.
    36. DeCarolis, Joseph F. & Hunter, Kevin & Sreepathi, Sarat, 2012. "The case for repeatable analysis with energy economy optimization models," Energy Economics, Elsevier, vol. 34(6), pages 1845-1853.
    37. Pye, Steve & Sabio, Nagore & Strachan, Neil, 2015. "An integrated systematic analysis of uncertainties in UK energy transition pathways," Energy Policy, Elsevier, vol. 87(C), pages 673-684.
    38. van Ruijven, Bas & Urban, Frauke & Benders, René M.J. & Moll, Henri C. & van der Sluijs, Jeroen P. & de Vries, Bert & van Vuuren, Detlef P., 2008. "Modeling Energy and Development: An Evaluation of Models and Concepts," World Development, Elsevier, vol. 36(12), pages 2801-2821, December.
    39. Thompson, Erica L. & Smith, Leonard A., 2019. "Escape from model-land," LSE Research Online Documents on Economics 103310, London School of Economics and Political Science, LSE Library.
    40. Pilavachi, P.A. & Dalamaga, Th. & Rossetti di Valdalbero, D. & Guilmot, J.-F., 2008. "Ex-post evaluation of European energy models," Energy Policy, Elsevier, vol. 36(5), pages 1726-1735, May.
    41. Rosen, Richard A. & Guenther, Edeltraud, 2016. "The energy policy relevance of the 2014 IPCC Working Group III report on the macro-economics of mitigating climate change," Energy Policy, Elsevier, vol. 93(C), pages 330-334.
    42. Thompson, Erica L. & Smith, Leonard A., 2019. "Escape from model-land," Economics Discussion Papers 2019-23, Kiel Institute for the World Economy (IfW Kiel).
    43. Frances C. Moore & Delavane B. Diaz, 2015. "Erratum: Temperature impacts on economic growth warrant stringent mitigation policy," Nature Climate Change, Nature, vol. 5(3), pages 280-280, March.
    44. Felix Creutzig & Peter Agoston & Jan Christoph Goldschmidt & Gunnar Luderer & Gregory Nemet & Robert C. Pietzcker, 2017. "The underestimated potential of solar energy to mitigate climate change," Nature Energy, Nature, vol. 2(9), pages 1-9, September.
    45. van Ruijven, Bas J. & van Vuuren, Detlef P. & de Vries, Bert J.M. & Isaac, Morna & van der Sluijs, Jeroen P. & Lucas, Paul L. & Balachandra, P., 2011. "Model projections for household energy use in India," Energy Policy, Elsevier, vol. 39(12), pages 7747-7761.
    46. Roger M. Cooke, 2015. "Messaging climate change uncertainty," Nature Climate Change, Nature, vol. 5(1), pages 8-10, January.
    47. Marisa Beck & Tobias Krueger, 2016. "The epistemic, ethical, and political dimensions of uncertainty in integrated assessment modeling," Wiley Interdisciplinary Reviews: Climate Change, John Wiley & Sons, vol. 7(5), pages 627-645, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ken Oshiro & Shinichiro Fujimori, 2024. "Limited impact of hydrogen co-firing on prolonging fossil-based power generation under low emissions scenarios," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    2. Chang, Miguel & Lund, Henrik & Thellufsen, Jakob Zinck & Østergaard, Poul Alberg, 2023. "Perspectives on purpose-driven coupling of energy system models," Energy, Elsevier, vol. 265(C).
    3. da Silva Neves, Marcus Vinicius & Szklo, Alexandre & Schaeffer, Roberto, 2023. "Fossil fuel facilities exergy return for a frontier of analysis incorporating CO2 capture: The case of a coal power plant," Energy, Elsevier, vol. 284(C).
    4. Rising, James A. & Taylor, Charlotte & Ives, Matthew C. & Ward, Robert E.T., 2022. "Challenges and innovations in the economic evaluation of the risks of climate change," Ecological Economics, Elsevier, vol. 197(C).
    5. Clare Hanmer & Charlie Wilson & Oreane Y. Edelenbosch & Detlef P. van Vuuren, 2022. "Translating Global Integrated Assessment Model Output into Lifestyle Change Pathways at the Country and Household Level," Energies, MDPI, vol. 15(5), pages 1-31, February.
    6. Heinisch, Verena & Dujardin, Jérôme & Gabrielli, Paolo & Jain, Pranjal & Lehning, Michael & Sansavini, Giovanni & Sasse, Jan-Philipp & Schaffner, Christian & Schwarz, Marius & Trutnevyte, Evelina, 2023. "Inter-comparison of spatial models for high shares of renewable electricity in Switzerland," Applied Energy, Elsevier, vol. 350(C).
    7. Rising, James A. & Taylor, Charlotte & Ives, Matthew C. & Ward, Robert E.t., 2022. "Challenges and innovations in the economic evaluation of the risks of climate change," LSE Research Online Documents on Economics 114941, London School of Economics and Political Science, LSE Library.
    8. Mohamed Elsayed Gabr & Amira Mahmoud El Shorbagy & Hamdy Badee Faheem, 2022. "Utilizing the Harvesting of Rainwater to Provide Safe Road Transportation Efficiency and Increase Water Resources in the Context of Climatic Change," Sustainability, MDPI, vol. 14(15), pages 1-20, August.
    9. Wen, Xin & Jaxa-Rozen, Marc & Trutnevyte, Evelina, 2023. "Hindcasting to inform the development of bottom-up electricity system models: The cases of endogenous demand and technology learning," Applied Energy, Elsevier, vol. 340(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ajay Gambhir & Isabela Butnar & Pei-Hao Li & Pete Smith & Neil Strachan, 2019. "A Review of Criticisms of Integrated Assessment Models and Proposed Approaches to Address These, through the Lens of BECCS," Energies, MDPI, vol. 12(9), pages 1-21, May.
    2. Huntington, Hillard G., 2021. "Model evaluation for policy insights: Reflections on the forum process," Energy Policy, Elsevier, vol. 156(C).
    3. DeCarolis, Joseph & Daly, Hannah & Dodds, Paul & Keppo, Ilkka & Li, Francis & McDowall, Will & Pye, Steve & Strachan, Neil & Trutnevyte, Evelina & Usher, Will & Winning, Matthew & Yeh, Sonia & Zeyring, 2017. "Formalizing best practice for energy system optimization modelling," Applied Energy, Elsevier, vol. 194(C), pages 184-198.
    4. Richard S.J. Tol, 2021. "Estimates of the social cost of carbon have not changed over time," Working Paper Series 0821, Department of Economics, University of Sussex Business School.
    5. Wen, Xin & Jaxa-Rozen, Marc & Trutnevyte, Evelina, 2022. "Accuracy indicators for evaluating retrospective performance of energy system models," Applied Energy, Elsevier, vol. 325(C).
    6. Richard S J Tol, 2018. "The Economic Impacts of Climate Change," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 12(1), pages 4-25.
    7. J. Farmer & Cameron Hepburn & Penny Mealy & Alexander Teytelboym, 2015. "A Third Wave in the Economics of Climate Change," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 62(2), pages 329-357, October.
    8. Richard S. J. Tol, 2021. "Estimates of the social cost of carbon have increased over time," Papers 2105.03656, arXiv.org, revised Aug 2022.
    9. Trutnevyte, Evelina, 2016. "Does cost optimization approximate the real-world energy transition?," Energy, Elsevier, vol. 106(C), pages 182-193.
    10. Tiruwork B. Tibebu & Eric Hittinger & Qing Miao & Eric Williams, 2024. "Adoption Model Choice Affects the Optimal Subsidy for Residential Solar," Energies, MDPI, vol. 17(3), pages 1-19, February.
    11. Kenneth Gillingham & William D. Nordhaus & David Anthoff & Geoffrey Blanford & Valentina Bosetti & Peter Christensen & Haewon McJeon & John Reilly & Paul Sztorc, 2015. "Modeling Uncertainty in Climate Change: A Multi-Model Comparison," NBER Working Papers 21637, National Bureau of Economic Research, Inc.
    12. Rong Wang & Juan Moreno-Cruz & Ken Caldeira, 2017. "Will the use of a carbon tax for revenue generation produce an incentive to continue carbon emissions?," Post-Print hal-03226925, HAL.
    13. Craxton, Melanie & Merrick, James & Makridis, Christos & Taggart, John, 2017. "On the climate policy implications of substitutability and flexibility in the economy: An in-depth integrated assessment model diagnostic," Technological Forecasting and Social Change, Elsevier, vol. 125(C), pages 289-298.
    14. Cotterman, Turner & Small, Mitchell J. & Wilson, Stephen & Abdulla, Ahmed & Wong-Parodi, Gabrielle, 2021. "Applying risk tolerance and socio-technical dynamics for more realistic energy transition pathways," Applied Energy, Elsevier, vol. 291(C).
    15. Glotin, David & Bourgeois, Cyril & Giraudet, Louis-Gaëtan & Quirion, Philippe, 2019. "Prediction is difficult, even when it's about the past: A hindcast experiment using Res-IRF, an integrated energy-economy model," Energy Economics, Elsevier, vol. 84(S1).
    16. John Weyant, 2017. "Some Contributions of Integrated Assessment Models of Global Climate Change," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 11(1), pages 115-137.
    17. Pablo E. Carvajal & Asami Miketa & Nadeem Goussous & Pauline Fulcheri, 2022. "Best Practice in Government Use and Development of Long-Term Energy Transition Scenarios," Energies, MDPI, vol. 15(6), pages 1-21, March.
    18. Wen, Xin & Jaxa-Rozen, Marc & Trutnevyte, Evelina, 2023. "Hindcasting to inform the development of bottom-up electricity system models: The cases of endogenous demand and technology learning," Applied Energy, Elsevier, vol. 340(C).
    19. Kollenbach, Gilbert, 2017. "On the optimal accumulation of renewable energy generation capacity," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 157-179.
    20. De Bruin, Kelly & Kiran Krishnamurthy, Chandra, 2021. "Optimal Climate Policy with Fat-tailed Uncertainty: What the Models Can Tell Us," Papers WP697, Economic and Social Research Institute (ESRI).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:climat:v:166:y:2021:i:1:d:10.1007_s10584-021-03099-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.