IDEAS home Printed from https://ideas.repec.org/a/spr/climat/v167y2021i3d10.1007_s10584-021-03194-x.html
   My bibliography  Save this article

Public opinion on climate change in the USA: to what extent can it be nudged by questionnaire design features?

Author

Listed:
  • Catherine Chen

    (Stanford University)

  • Bo MacInnis

    (Stanford University)

  • Matthew Waltman

    (Stanford University)

  • Jon A. Krosnick

    (Stanford University)

Abstract

Policymakers may wish to take into account public opinion on climate change as they craft legislation, but if public opinion changes substantially in response to seemingly trivial changes in survey questionnaire design, perhaps such reliance would be unwise. This paper examines 110 experiments implemented in surveys of truly random samples of American adults between 2012 and 2018 (N = 4414), exploring the extent to which answers to questions were influenced by order and wording manipulations. Of 144 tests, 31 (22%) yielded statistically significant effects. Adjustments for multiple hypothesis tests reduced this percentage to between 7 and 9%. The effect sizes are routinely small. These results are consistent with the conclusion that survey results on climate change issues are relatively robust, so policymakers can take them seriously if they wish to do so.

Suggested Citation

  • Catherine Chen & Bo MacInnis & Matthew Waltman & Jon A. Krosnick, 2021. "Public opinion on climate change in the USA: to what extent can it be nudged by questionnaire design features?," Climatic Change, Springer, vol. 167(3), pages 1-18, August.
    • Handle: RePEc:spr:climat:v:167:y:2021:i:3:d:10.1007_s10584-021-03194-x
    DOI: 10.1007/s10584-021-03194-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10584-021-03194-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10584-021-03194-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Malhotra, Neil & Krosnick, Jon A., 2007. "The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples," Political Analysis, Cambridge University Press, vol. 15(3), pages 286-323, July.
    2. Tien Ming Lee & Ezra M. Markowitz & Peter D. Howe & Chia-Ying Ko & Anthony A. Leiserowitz, 2015. "Predictors of public climate change awareness and risk perception around the world," Nature Climate Change, Nature, vol. 5(11), pages 1014-1020, November.
    3. Converse, Philip E., 1974. "Comment: The Status of Nonattitudes," American Political Science Review, Cambridge University Press, vol. 68(2), pages 650-660, June.
    4. Graham Kalton & Martin Collins & Lindsay Brook, 1978. "Experiments in Wording Opinion Questions," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 27(2), pages 149-161, June.
    5. Daniele Fanelli, 2010. "Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data," PLOS ONE, Public Library of Science, vol. 5(4), pages 1-7, April.
    6. Ioana Alina Cristea & John P A Ioannidis, 2018. "P values in display items are ubiquitous and almost invariably significant: A survey of top science journals," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-15, May.
    7. Matthew Motta & Daniel Chapman & Dominik Stecula & Kathryn Haglin, 2019. "An experimental examination of measurement disparities in public climate change beliefs," Climatic Change, Springer, vol. 154(1), pages 37-47, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carroll, Eamonn & Timmons, Shane & McGinnity, Frances, 2023. "Experimental tests of public support for disability policy," Research Series, Economic and Social Research Institute (ESRI), number RS159, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Carina Cornesse & Annelies G. Blom, 2023. "Response Quality in Nonprobability and Probability-based Online Panels," Sociological Methods & Research, , vol. 52(2), pages 879-908, May.
    2. Abinash Bhattachan & Matthew D. Jurjonas & Priscilla R. Morris & Paul J. Taillie & Lindsey S. Smart & Ryan E. Emanuel & Erin L. Seekamp, 2019. "Linking residential saltwater intrusion risk perceptions to physical exposure of climate change impacts in rural coastal communities of North Carolina," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 97(3), pages 1277-1295, July.
    3. Guglielmo Zappalà, 2023. "Drought Exposure and Accuracy: Motivated Reasoning in Climate Change Beliefs," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 85(3), pages 649-672, August.
    4. A. K. Enamul Haque & Heman D. Lohano & Pranab Mukhopadhyay & Mani Nepal & Fathimath Shafeeqa & Shamen P. Vidanage, 2019. "NDC pledges of South Asia: are the stakeholders onboard?," Climatic Change, Springer, vol. 155(2), pages 237-244, July.
    5. Can Askan Mavi & Nicolas Quérou, 2020. "Common pool resource management and risk perceptions," DEM Discussion Paper Series 20-25, Department of Economics at the University of Luxembourg.
    6. Stefan, Matthias & Huber, Jürgen & Kirchler, Michael & Sutter, Matthias & Walzl, Markus, 2023. "Monetary and social incentives in multi-tasking: The ranking substitution effect," European Economic Review, Elsevier, vol. 156(C).
    7. Kevin J. Boyle & Mark Morrison & Darla Hatton MacDonald & Roderick Duncan & John Rose, 2016. "Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 64(3), pages 401-419, July.
    8. Matthias Stefan & Jürgen Huber & Michael Kirchler & Matthias Sutter & Markus Walzl, 2020. "Monetary and Social Incentives in Multi-Tasking: The Ranking Substitution Effect," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2020_10, Max Planck Institute for Research on Collective Goods.
    9. Helena Fornwagner & Oliver P. Hauser, 2022. "Climate Action for (My) Children," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 81(1), pages 95-130, January.
    10. Andrew Bieler & Randolph Haluza-Delay & Ann Dale & Marcia Mckenzie, 2017. "A National Overview of Climate Change Education Policy: Policy Coherence between Subnational Climate and Education Policies in Canada (K-12)," Journal of Education for Sustainable Development, , vol. 11(2), pages 63-85, September.
    11. Jasper Brinkerink, 2023. "When Shooting for the Stars Becomes Aiming for Asterisks: P-Hacking in Family Business Research," Entrepreneurship Theory and Practice, , vol. 47(2), pages 304-343, March.
    12. Lasse J. Jessen & Sebastian Koehne & Patrick Nüß & Jens Ruhose, 2024. "Socioeconomic Inequality in Life Expectancy: Perception and Policy Demand," CESifo Working Paper Series 10940, CESifo.
    13. repec:hal:spmain:info:hdl:2441/6d7es28iae9pjoil7092hs41h3 is not listed on IDEAS
    14. Dell'Anno, Roberto & Caferra, Rocco & Morone, Andrea, 2020. "A “Trojan Horse” in the peer-review process of fee-charging economic journals," Journal of Informetrics, Elsevier, vol. 14(3).
    15. Lars Mewes & Leonie Tuitjer & Peter Dirksmeier, 2024. "Exploring the variances of climate change opinions in Germany at a fine-grained local scale," Nature Communications, Nature, vol. 15(1), pages 1-14, December.
    16. Sam Crawley & Hilde Coffé & Ralph Chapman, 2022. "Climate Belief and Issue Salience: Comparing Two Dimensions of Public Opinion on Climate Change in the EU," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 162(1), pages 307-325, July.
    17. Aaron C. Sparks & Heather Hodges & Sarah Oliver & Eric R. A. N. Smith, 2020. "Confidence in Local, National, and International Scientists on Climate Change," Sustainability, MDPI, vol. 13(1), pages 1-13, December.
    18. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    19. Mona Nabil Demaidi & Khaled Al-Sahili, 2021. "Integrating SDGs in Higher Education—Case of Climate Change Awareness and Gender Equality in a Developing Country According to RMEI-TARGET Strategy," Sustainability, MDPI, vol. 13(6), pages 1-21, March.
    20. Donatella Baiardi, 2021. "What do you think about climate change?," Working Papers 477, University of Milano-Bicocca, Department of Economics, revised Aug 2021.
    21. Reitmann, Ann-Kristin & Goedhuys, Micheline & Grimm, Michael & Nillesen, Eleonora E.M., 2019. "Measuring attitudes on gender equality and domestic violence in the Arab context : The role of framing, priming and interviewer effects," MERIT Working Papers 2019-027, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:climat:v:167:y:2021:i:3:d:10.1007_s10584-021-03194-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.