IDEAS home Printed from https://ideas.repec.org/a/nat/natcom/v11y2020i1d10.1038_s41467-020-20142-y.html
   My bibliography  Save this article

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Author

Listed:
  • Alec P. Christie

    (Conservation Science Group, Department of Zoology, University of Cambridge, The David Attenborough Building)

  • David Abecasis

    (Centre of Marine Sciences (CCMar), Universidade do Algarve)

  • Mehdi Adjeroud

    (Institut de Recherche pour le Développement (IRD), UMR 9220 ENTROPIE & Laboratoire d’Excellence CORAIL, Université de Perpignan Via Domitia)

  • Juan C. Alonso

    (Museo Nacional de Ciencias Naturales, CSIC)

  • Tatsuya Amano

    (School of Biological Sciences, University of Queensland)

  • Alvaro Anton

    (Education Faculty of Bilbao, University of the Basque Country (UPV/EHU). Sarriena z/g E-48940 Leioa)

  • Barry P. Baldigo

    (U.S. Geological Survey, New York Water Science Center)

  • Rafael Barrientos

    (Universidad Complutense de Madrid, Departamento de Biodiversidad, Ecología y Evolución, Facultad de Ciencias Biológicas)

  • Jake E. Bicknell

    (Durrell Institute of Conservation and Ecology (DICE), School of Anthropology and Conservation, University of Kent)

  • Deborah A. Buhl

    (U.S. Geological Survey, Northern Prairie Wildlife Research Center)

  • Just Cebrian

    (Northern Gulf Institute, Mississippi State University, 1021 Balch Blvd, John C. Stennis Space Center)

  • Ricardo S. Ceia

    (MARE – Marine and Environmental Sciences Centre, Dept. Life Sciences, University of Coimbra
    CFE – Centre for Functional Ecology, Dept. Life Sciences, University of Coimbra)

  • Luciana Cibils-Martina

    (Universidad Nacional de Río Cuarto (UNRC)
    CONICET)

  • Sarah Clarke

    (Marine Institute, Rinville, Oranmore)

  • Joachim Claudet

    (National Center for Scientific Research, PSL Université Paris, CRIOBE, USR 3278 CNRS-EPHE-UPVD, Maison des Océans)

  • Michael D. Craig

    (School of Biological Sciences, University of Western Australia
    School of Environmental and Conservation Sciences, Murdoch University)

  • Dominique Davoult

    (Sorbonne Université, CNRS, UMR 7144, Station Biologique)

  • Annelies Backer

    (Flanders Research Institute for Agriculture, Fisheries and Food (ILVO))

  • Mary K. Donovan

    (Marine Science Institute, University of California Santa Barbara
    Hawaii Institute of Marine Biology, University of Hawaii at Manoa)

  • Tyler D. Eddy

    (Baruch Institute for Marine & Coastal Sciences, University of South Carolina
    Centre for Fisheries Ecosystems Research, Fisheries & Marine Institute, Memorial University of Newfoundland
    School of Biological Sciences, Victoria University of Wellington)

  • Filipe M. França

    (Lancaster Environment Centre, Lancaster University)

  • Jonathan P. A. Gardner

    (School of Biological Sciences, Victoria University of Wellington)

  • Bradley P. Harris

    (Fisheries, Aquatic Science and Technology Laboratory, Alaska Pacific University)

  • Ari Huusko

    (Natural Resources Institute Finland)

  • Ian L. Jones

    (Memorial University)

  • Brendan P. Kelaher

    (National Marine Science Centre and Marine Ecology Research Centre, Southern Cross University)

  • Janne S. Kotiaho

    (University of Jyväskylä
    School of Resource Wisdom, University of Jyväskylä)

  • Adrià López-Baucells

    (Centre for Ecology, Evolution and Environmental Changes – cE3c, Faculty of Sciences, University of Lisbon
    Biological Dynamics of Forest Fragments Project, National Institute for Amazonian Research and Smithsonian Tropical Research Institute
    Granollers Museum of Natural History)

  • Heather L. Major

    (University of New Brunswick)

  • Aki Mäki-Petäys

    (Voimalohi Oy
    Natural Resources Institute Finland, Paavo Havaksen tie 3, 90014 University of Oulu)

  • Beatriz Martín

    (Fundación Migres CIMA Ctra
    Intergovernmental Oceanographic Commission of UNESCO, Marine Policy and Regional Coordination Section Paris 07)

  • Carlos A. Martín

    (Universidad Complutense de Madrid, Departamento de Biodiversidad, Ecología y Evolución, Facultad de Ciencias Biológicas)

  • Philip A. Martin

    (Conservation Science Group, Department of Zoology, University of Cambridge, The David Attenborough Building
    BioRISC, St. Catharine’s College)

  • Daniel Mateos-Molina

    (Universidad de Murcia)

  • Robert A. McConnaughey

    (RACE Division, Alaska Fisheries Science Center, National Marine Fisheries Service, NOAA, 7600 Sand Point Way NE)

  • Michele Meroni

    (European Commission, Joint Research Centre (JRC))

  • Christoph F. J. Meyer

    (Centre for Ecology, Evolution and Environmental Changes – cE3c, Faculty of Sciences, University of Lisbon
    Biological Dynamics of Forest Fragments Project, National Institute for Amazonian Research and Smithsonian Tropical Research Institute
    School of Science, Engineering and Environment, University of Salford)

  • Kade Mills

    (Victorian National Park Association)

  • Monica Montefalcone

    (Environment and Life Sciences (DiSTAV), University of Genoa)

  • Norbertas Noreika

    (Swedish University of Agricultural Sciences
    Chair of Plant Health, Institute of Agricultural and Environmental Sciences, Estonian University of Life Sciences)

  • Carlos Palacín

    (Museo Nacional de Ciencias Naturales, CSIC)

  • Anjali Pande

    (School of Biological Sciences, Victoria University of Wellington
    Biosecurity New Zealand – Tiakitanga Pūtaiao Aotearoa, Ministry for Primary Industries – Manatū Ahu Matua
    National Institute of Water & Atmospheric Research Ltd (NIWA))

  • C. Roland Pitcher

    (CSIRO Oceans & Atmosphere, Queensland Biosciences Precinct)

  • Carlos Ponce

    (Museo Nacional de Ciencias Naturales, CSIC)

  • Matt Rinella

    (Fort Keogh Livestock and Range Research Laboratory)

  • Ricardo Rocha

    (Centre for Ecology, Evolution and Environmental Changes – cE3c, Faculty of Sciences, University of Lisbon
    Biological Dynamics of Forest Fragments Project, National Institute for Amazonian Research and Smithsonian Tropical Research Institute
    CIBIO-InBIO, Research Centre in Biodiversity and Genetic Resources, University of Porto)

  • María C. Ruiz-Delgado

    (Químicos y Naturales, Universidad Pablo de Olavide)

  • Juan J. Schmitter-Soto

    (El Colegio de la Frontera Sur)

  • Jill A. Shaffer

    (U.S. Geological Survey, Northern Prairie Wildlife Research Center)

  • Shailesh Sharma

    (Division of Fish and Wildlife, New York State Department of Environmental Conservation)

  • Anna A. Sher

    (University of Denver Department of Biological Sciences)

  • Doriane Stagnol

    (Sorbonne Université, CNRS, UMR 7144, Station Biologique)

  • Thomas R. Stanley

    (U.S. Geological Survey, Fort Collins Science Center)

  • Kevin D. E. Stokesbury

    (School for Marine Science and Technology, University of Massachusetts Dartmouth)

  • Aurora Torres

    (Georges Lemaître Earth and Climate Research Centre, Earth and Life Institute, Université Catholique de Louvain
    Center for Systems Integration and Sustainability, Department of Fisheries and Wildlife, 13 Michigan State University)

  • Oliver Tully

    (Marine Institute, Rinville, Oranmore)

  • Teppo Vehanen

    (Natural Resources Institute Finland)

  • Corinne Watts

    (Manaaki Whenua – Landcare Research)

  • Qingyuan Zhao

    (Statistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge)

  • William J. Sutherland

    (Conservation Science Group, Department of Zoology, University of Cambridge, The David Attenborough Building
    BioRISC, St. Catharine’s College)

Abstract

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

Suggested Citation

  • Alec P. Christie & David Abecasis & Mehdi Adjeroud & Juan C. Alonso & Tatsuya Amano & Alvaro Anton & Barry P. Baldigo & Rafael Barrientos & Jake E. Bicknell & Deborah A. Buhl & Just Cebrian & Ricardo , 2020. "Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences," Nature Communications, Nature, vol. 11(1), pages 1-11, December.
    • Handle: RePEc:nat:natcom:v:11:y:2020:i:1:d:10.1038_s41467-020-20142-y
    DOI: 10.1038/s41467-020-20142-y
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41467-020-20142-y
    File Function: Abstract
    Download Restriction: no
    File URL: https://libkey.io/10.1038/s41467-020-20142-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:natcom:v:11:y:2020:i:1:d:10.1038_s41467-020-20142-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.