IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0286403.html
   My bibliography  Save this article

Hedges, mottes, and baileys: Causally ambiguous statistical language can increase perceived study quality and policy relevance

Author

Listed:
  • Daniela Alvarez-Vargas
  • David Braithwaite
  • Hugues Lortie-Forgues
  • Melody Moore
  • Sirui Wan
  • Elizabeth Martin
  • Drew Hal Bailey

Abstract

There is a norm in psychology to use causally ambiguous statistical language, rather than straightforward causal language, when describing methods and results of nonexperimental studies. However, causally ambiguous language may inhibit a critical examination of the study’s causal assumptions and lead to a greater acceptance of policy recommendations that rely on causal interpretations of nonexperimental findings. In a preregistered experiment, 142 psychology faculty, postdocs, and doctoral students (54% female), ages 22–67 (M = 33.20, SD = 8.96), rated the design and analysis from hypothetical studies with causally ambiguous statistical language as of higher quality (by .34-.80 SD) and as similarly or more supportive (by .16-.27 SD) of policy recommendations than studies described in straightforward causal language. Thus, using statistical rather than causal language to describe nonexperimental findings did not decrease, and may have increased, perceived support for implicitly causal conclusions.

Suggested Citation

  • Daniela Alvarez-Vargas & David Braithwaite & Hugues Lortie-Forgues & Melody Moore & Sirui Wan & Elizabeth Martin & Drew Hal Bailey, 2023. "Hedges, mottes, and baileys: Causally ambiguous statistical language can increase perceived study quality and policy relevance," PLOS ONE, Public Library of Science, vol. 18(10), pages 1-21, October.
    • Handle: RePEc:plo:pone00:0286403
    DOI: 10.1371/journal.pone.0286403
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0286403
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0286403&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0286403?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ben Weidmann & Luke Miratrix, 2021. "Lurking Inferential Monsters? Quantifying Selection Bias In Evaluations Of School Programs," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(3), pages 964-986, June.
    2. repec:plo:pone00:0196346 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gonzalo Nunez-Chaim & Henry G. Overman & Capucine Riom, 2024. "Does subsidising business advice improve firm performance? Evidence from a large RCT," CEP Discussion Papers dp1977, Centre for Economic Performance, LSE.
    2. Greta Morando & Lucinda Platt, 2022. "The Impact of Centre‐based Childcare on Non‐cognitive Skills of Young Children," Economica, London School of Economics and Political Science, vol. 89(356), pages 908-946, October.
    3. Nunez Chaim, Gonzalo Ignacio & Overman, Henry George & Riom, Capucine Anne Veronique, 2024. "Does subsidising business advice improve firm performance? Evidence from a large RCT," LSE Research Online Documents on Economics 126834, London School of Economics and Political Science, LSE Library.
    4. Sam Sims & Jake Anders & Matthew Inglis & Hugues Lortie-Forgues & Ben Styles & Ben Weidmann, 2023. "Experimental education research: rethinking why, how and when to use random assignment," CEPEO Working Paper Series 23-07, UCL Centre for Education Policy and Equalising Opportunities, revised Aug 2023.
    5. John Deke & Mariel Finucane & Daniel Thal, "undated". "The BASIE (BAyeSian Interpretation of Estimates) Framework for Interpreting Findings from Impact Evaluations: A Practical Guide for Education Researchers," Mathematica Policy Research Reports 5a0d5dff375d42048799878be, Mathematica Policy Research.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0286403. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.