IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v14y2022i18p11337-d911361.html
   My bibliography  Save this article

Reliability and Validity of Widely Used International Surveys on the Environment

Author

Listed:
  • Joseph Paul Lavallee

    (International Business and Trade Program, Ming Chuan University International College, Taipei 111, Taiwan)

  • Bruno Di Giusto

    (Journalism and Mass Communication Program, Ming Chuan University International College, Taipei 111, Taiwan)

  • Tai-Yi Yu

    (Department of Risk Management & Insurance, Ming Chuan University, Taipei 111, Taiwan)

  • Su-Pin Hung

    (Center for Teacher Education, Institute of Education, National Cheng Kung University, Tainan 701, Taiwan)

Abstract

Do existing public opinion surveys provide valid and reliable measures of attitudes towards environmental sustainability? This question is critical given the importance of public support for achieving sustainability. Starting with 28 survey items about the environment drawn from the World Values Survey Waves 5 and 6 and the 2010 International Social Survey Program, we assessed reliability by checking for significant correlations between similar or identical items on different surveys. Next, to assess validity, we evaluated correlations between survey items and 22 objective environmental indicators drawn from the Environmental Performance Index (EPI). As the level of economic development is a likely confound, we also performed partial correlation analyses controlling for GDP per capita. From the initial 28 items, we identified 23 sufficiently reliable items, but many of these were found to have low predictive power in the validity analysis. Items about air and water pollution were valid predictors of objective environmental conditions in these areas. Items asking about the relative importance of environmental problems compared to other social issues were also good positive predictors of progress on perceptible environmental issues. Items asking about general sentiment with no clear referent performed poorly. When controlling for GDP, country-level attitudes were more aligned with country-specific environmental conditions. Finally, nearly half of all EPI indicators were associated with few or no survey items, indicating the existence of ‘blind spots’ in public awareness. Our findings should offer guidance to both survey developers and users, as well as to policy makers responsible for conveying information about environmental sustainability to the wider public.

Suggested Citation

  • Joseph Paul Lavallee & Bruno Di Giusto & Tai-Yi Yu & Su-Pin Hung, 2022. "Reliability and Validity of Widely Used International Surveys on the Environment," Sustainability, MDPI, vol. 14(18), pages 1-22, September.
    • Handle: RePEc:gam:jsusta:v:14:y:2022:i:18:p:11337-:d:911361
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/14/18/11337/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/14/18/11337/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Stegmueller, Daniel, 2011. "Apples and Oranges? The Problem of Equivalence in Comparative Research," Political Analysis, Cambridge University Press, vol. 19(4), pages 471-487.
    2. repec:cup:apsrev:v:98:y:2004:i:01:p:191-207_00 is not listed on IDEAS
    3. Tjernström, E. & Tietenberg, T., 2008. "Do differences in attitudes explain differences in national climate change policies?," Ecological Economics, Elsevier, vol. 65(2), pages 315-324, April.
    4. Schaffer, Lena Maria & Bernauer, Thomas, 2014. "Explaining government choices for promoting renewable energy," Energy Policy, Elsevier, vol. 68(C), pages 15-27.
    5. Michael Howes & Liana Wortley & Ruth Potts & Aysin Dedekorkut-Howes & Silvia Serrao-Neumann & Julie Davidson & Timothy Smith & Patrick Nunn, 2017. "Environmental Sustainability: A Case of Policy Implementation Failure?," Sustainability, MDPI, vol. 9(2), pages 1-17, January.
    6. Hannah Janetschek & Clara Brandi & Adis Dzebo & Bernd Hackmann, 2020. "The 2030 Agenda and the Paris Agreement: voluntary contributions towards thematic policy coherence," Climate Policy, Taylor & Francis Journals, vol. 20(4), pages 430-442, April.
    7. World Bank, 2016. "World Development Indicators 2016," World Bank Publications - Books, The World Bank Group, number 23969, December.
    8. Eric Neumayer, 2002. "Do Democracies Exhibit Stronger International Environmental Commitment? A Cross-country Analysis," Journal of Peace Research, Peace Research Institute Oslo, vol. 39(2), pages 139-164, March.
    9. Bättig, Michèle B. & Bernauer, Thomas, 2009. "National Institutions and Global Public Goods: Are Democracies More Cooperative in Climate Change Policy?," International Organization, Cambridge University Press, vol. 63(2), pages 281-308, April.
    10. Mark Elder & Simon Høiberg Olsen, 2019. "The Design of Environmental Priorities in the SDGs," Global Policy, London School of Economics and Political Science, vol. 10(S1), pages 70-82, January.
    11. Eric Neumayer, 2002. "Do We Trust the Data? On the Validity and Reliability of Cross‐National Environmental Surveys," Social Science Quarterly, Southwestern Social Science Association, vol. 83(1), pages 332-340, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Li, Xing & Hu, Zhigao & Cao, Jianhua & Xu, Xing, 2022. "The impact of environmental accountability on air pollution: A public attention perspective," Energy Policy, Elsevier, vol. 161(C).
    2. Daniel Fiorino, 2011. "Explaining national environmental performance: approaches, evidence, and implications," Policy Sciences, Springer;Society of Policy Sciences, vol. 44(4), pages 367-389, November.
    3. Marlene Kammerer & Chandreyee Namhata, 2018. "What drives the adoption of climate change mitigation policy? A dynamic network approach to policy diffusion," Policy Sciences, Springer;Society of Policy Sciences, vol. 51(4), pages 477-513, December.
    4. Haiqing Hu & Di Chen & Chun‐Ping Chang & Yin Chu, 2021. "The Political Economy Of Environmental Consequences: A Review Of The Empirical Literature," Journal of Economic Surveys, Wiley Blackwell, vol. 35(1), pages 250-306, February.
    5. Fredriksson, Per G. & Neumayer, Eric, 2013. "Democracy and climate change policies: Is history important?," Ecological Economics, Elsevier, vol. 95(C), pages 11-19.
    6. Catherine Z. Worsnop, 2017. "Domestic politics and the WHO’s International Health Regulations: Explaining the use of trade and travel barriers during disease outbreaks," The Review of International Organizations, Springer, vol. 12(3), pages 365-395, September.
    7. Stepping, Katharina M. K. & Banholzer, Lilli, 2017. "Autocratic angels? Democratic demons? The impact of regime type, state capacity and economic development on reaching environmental targets," IDOS Discussion Papers 26/2017, German Institute of Development and Sustainability (IDOS).
    8. Katarzyna Iwińska & Athanasios Kampas & Kerry Longhurst, 2019. "Interactions between Democracy and Environmental Quality: Toward a More Nuanced Understanding," Sustainability, MDPI, vol. 11(6), pages 1-17, March.
    9. Undp, 2011. "HDR 2011 - Sustainability and Equity: A Better Future for All," Human Development Report (1990 to present), Human Development Report Office (HDRO), United Nations Development Programme (UNDP), number hdr2011, September.
    10. Paul G. Harris & Taedong Lee, 2017. "Compliance with climate change agreements: the constraints of consumption," International Environmental Agreements: Politics, Law and Economics, Springer, vol. 17(6), pages 779-794, December.
    11. Todd A. Eisenstadt & Daniel J. Fiorino & Daniela Stevens, 2019. "National environmental policies as shelter from the storm: specifying the relationship between extreme weather vulnerability and national environmental performance," Journal of Environmental Studies and Sciences, Springer;Association of Environmental Studies and Sciences, vol. 9(1), pages 96-107, March.
    12. Tobias Böhmelt & Edita Butkutė, 2018. "The self-selection of democracies into treaty design: insights from international environmental agreements," International Environmental Agreements: Politics, Law and Economics, Springer, vol. 18(3), pages 351-367, June.
    13. Zhaojun Yang & Weihao Liu & Jun Sun & Yali Zhang, 2017. "Corporate Environmental Responsibility and Environmental Non-Governmental Organizations in China," Sustainability, MDPI, vol. 9(10), pages 1-12, September.
    14. Lauri Peterson, 2021. "Silver Lining to Extreme Weather Events? Democracy and Climate Change Mitigation," Global Environmental Politics, MIT Press, vol. 21(1), pages 23-53, Winter.
    15. Mohammad Badrul Muttakin & Tarek Rana & Dessalegn Getie Mihret, 2022. "Democracy, national culture and greenhouse gas emissions: An international study," Business Strategy and the Environment, Wiley Blackwell, vol. 31(7), pages 2978-2991, November.
    16. Andrew Cheon & Johannes Urpelainen, 2013. "How do Competing Interest Groups Influence Environmental Policy? The Case of Renewable Electricity in Industrialized Democracies, 1989–2007," Political Studies, Political Studies Association, vol. 61(4), pages 874-897, December.
    17. Daniel Y. Kono & Gabriella R. Montinola, 2019. "Foreign aid and climate change policy: What can('t) the data tell us?," WIDER Working Paper Series wp-2019-15, World Institute for Development Economic Research (UNU-WIDER).
    18. Tobias Böhmelt, 2013. "A closer look at the information provision rationale: Civil society participation in states’ delegations at the UNFCCC," The Review of International Organizations, Springer, vol. 8(1), pages 55-80, March.
    19. Frida Boräng & Simon Felgendreher & Niklas Harring & Åsa Löfgren, 2019. "Committing to the Climate: A Global Study of Accountable Climate Targets," Sustainability, MDPI, vol. 11(7), pages 1-11, March.
    20. Jan Fagerberg & Bengt-Åke Lundvall & Martin Srholec, 2018. "Global Value Chains, National Innovation Systems and Economic Development," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 30(3), pages 533-556, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:14:y:2022:i:18:p:11337-:d:911361. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.