IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v56y2022i6d10.1007_s11135-022-01354-4.html
   My bibliography  Save this article

An experimental study of countermeasures against threats: real-world effects meet treatment effects

Author

Listed:
  • Roman Chytilek

    (Masaryk University)

  • Miroslav Mareš

    (Masaryk University)

  • Jakub Drmola

    (Masaryk University)

  • Lenka Hrbková

    (Masaryk University)

  • Petra Mlejnková

    (Masaryk University)

  • Zuzana Špačková

    (University of Defence)

  • Michal Tóth

    (Masaryk University)

Abstract

The experimental study of positions on policies and measures against various new types of threat is fast becoming a mainstream research practice. In this article we argue as follows: in security studies in particular, there is a risk that the experimental treatment is contaminated by subjects’ previous experience of the real world (‘contamination’), and this may substantially complicate the assessment of the size of the experimental treatment’s causal effect. We discuss ways to decrease the risk of uncontrolled contamination. Using two experimental case studies we show two typical cases of contamination in security studies (one, where the contamination of all treatments was extremely high, and another, where the level of contamination was unknown and might have varied across the experimental groups) and consider what this implies for the substantive results of the experiments. An analysis of contamination should become a routine, especially when reporting security experiments.

Suggested Citation

  • Roman Chytilek & Miroslav Mareš & Jakub Drmola & Lenka Hrbková & Petra Mlejnková & Zuzana Špačková & Michal Tóth, 2022. "An experimental study of countermeasures against threats: real-world effects meet treatment effects," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(6), pages 4825-4840, December.
    • Handle: RePEc:spr:qualqt:v:56:y:2022:i:6:d:10.1007_s11135-022-01354-4
    DOI: 10.1007/s11135-022-01354-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-022-01354-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-022-01354-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Linos, Katerina & Twist, Kimberly, 2018. "Diverse Pre-Treatment Effects in Survey Experiments," Journal of Experimental Political Science, Cambridge University Press, vol. 5(2), pages 148-158, July.
    2. Yan, Jubo & Kniffin, Kevin M. & Kunreuther, Howard C. & Schulze, William D., 2020. "The roles of reason and emotion in private and public responses to terrorism," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 778-796.
    3. James N. Druckman & Thomas J. Leeper, 2012. "Learning More from Political Communication Experiments: Pretreatment and Its Effects," American Journal of Political Science, John Wiley & Sons, vol. 56(4), pages 875-896, October.
    4. Gaines, Brian J. & Kuklinski, James H. & Quirk, Paul J., 2007. "The Logic of the Survey Experiment Reexamined," Political Analysis, Cambridge University Press, vol. 15(1), pages 1-20, January.
    5. Paul Slovic, 2002. "Terrorism as Hazard: A New Species of Trouble," Risk Analysis, John Wiley & Sons, vol. 22(3), pages 425-426, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kevin L. Cope, 2023. "Measuring law's normative force," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 20(4), pages 1005-1044, December.
    2. Hicken, Allen & Leider, Stephen & Ravanilla, Nico & Yang, Dean, 2018. "Temptation in vote-selling: Evidence from a field experiment in the Philippines," Journal of Development Economics, Elsevier, vol. 131(C), pages 1-14.
    3. Yacov Y. Haimes, 2012. "Systems‐Based Guiding Principles for Risk Modeling, Planning, Assessment, Management, and Communication," Risk Analysis, John Wiley & Sons, vol. 32(9), pages 1451-1467, September.
    4. Yu‐Ru Lin & Drew Margolin & Xidao Wen, 2017. "Tracking and Analyzing Individual Distress Following Terrorist Attacks Using Social Media Streams," Risk Analysis, John Wiley & Sons, vol. 37(8), pages 1580-1605, August.
    5. David A. Steinberg & Yeling Tan, 2023. "Public responses to foreign protectionism: Evidence from the US-China trade war," The Review of International Organizations, Springer, vol. 18(1), pages 145-167, January.
    6. Briguglio, Marie & Delaney, Liam & Wood, Alex, 2018. "Partisanship, priming and participation in public-good schemes," European Journal of Political Economy, Elsevier, vol. 55(C), pages 136-150.
    7. Verteramo Chiu, Leslie J. & Turvey, Calum G., 2015. "Perception and Action in a Conflict Zone: a Study of Rural Economy and Rural Life amidst Narcos in Northeastern Mexico," 2015 AAEA & WAEA Joint Annual Meeting, July 26-28, San Francisco, California 205447, Agricultural and Applied Economics Association.
    8. Turvey, Calum G. & Onyango, Benjamin & Cuite, Cara & Hallman, William K., 2010. "Risk, fear, bird flu and terrorists: A study of risk perceptions and economics," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 39(1), pages 1-10, January.
    9. Kuehnhanss, Colin R. & Heyndels, Bruno, 2018. "All’s fair in taxation: A framing experiment with local politicians," Journal of Economic Psychology, Elsevier, vol. 65(C), pages 26-40.
    10. Manville, Michael & Levine, Adam Seth, 2018. "What motivates public support for public transit?," Transportation Research Part A: Policy and Practice, Elsevier, vol. 118(C), pages 567-580.
    11. John Garvey & Martin Mullins, 2008. "Contemporary Terrorism: Risk Perception in the London Options Market," Risk Analysis, John Wiley & Sons, vol. 28(1), pages 151-160, February.
    12. Angino, Siria & Secola, Stefania, 2022. "Instinctive versus reflective trust in the European Central Bank," Working Paper Series 2660, European Central Bank.
    13. Bechtel, Michael & Hainmueller, Jens & Hangartner, Dominik & Helbling, Marc, 2015. "Reality Bites: The Limits of Framing Effects for Salient and Contested Policy Issues," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 3(3), pages 683-695.
    14. Katerina Linos & Kimberly Twist, 2016. "The Supreme Court, the Media, and Public Opinion: Comparing Experimental and Observational Methods," The Journal of Legal Studies, University of Chicago Press, vol. 45(2), pages 223-254.
    15. Bruno Castanho Silva & Jens Wäckerle & Christopher Wratil, 2022. "Determinants of Public Opinion Support for a Full Embargo on Russian Energy in Germany," ECONtribute Discussion Papers Series 170, University of Bonn and University of Cologne, Germany.
    16. Ilyana Kuziemko & Michael I. Norton & Emmanuel Saez & Stefanie Stantcheva, 2015. "How Elastic Are Preferences for Redistribution? Evidence from Randomized Survey Experiments," American Economic Review, American Economic Association, vol. 105(4), pages 1478-1508, April.
    17. Adams, Ian T., 2022. "Modeling Officer Perceptions of Body-worn Cameras: A National Survey," Thesis Commons fnxbg, Center for Open Science.
    18. Patrick Haack & Michael D. Pfarrer & Andreas Georg Scherer, 2014. "Legitimacy-as-Feeling: How Affect Leads to Vertical Legitimacy Spillovers in Transnational Governance," Journal of Management Studies, Wiley Blackwell, vol. 51(4), pages 634-666, June.
    19. Aklin, Michaël & Cheng, Chao-Yo & Urpelainen, Johannes, 2018. "Social acceptance of new energy technology in developing countries: A framing experiment in rural India," Energy Policy, Elsevier, vol. 113(C), pages 466-477.
    20. Lala Muradova & Ross James Gildea, 2021. "Oil wealth and US public support for war," Conflict Management and Peace Science, Peace Science Society (International), vol. 38(1), pages 3-19, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:56:y:2022:i:6:d:10.1007_s11135-022-01354-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.