IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v183y2020i1p193-209.html
   My bibliography  Save this article

Adjusting trial results for biases in meta‐analysis: combining data‐based evidence on bias with detailed trial assessment

Author

Listed:
  • K. M. Rhodes
  • J. Savović
  • R. Elbers
  • H. E. Jones
  • J. P. T. Higgins
  • J. A. C. Sterne
  • N. J. Welton
  • R. M. Turner

Abstract

Flaws in the conduct of randomized trials can lead to biased estimation of the intervention effect. Methods for adjustment of within‐trial biases in meta‐analysis include the use of empirical evidence from an external collection of meta‐analyses, and the use of expert opinion informed by the assessment of detailed trial information. Our aim is to present methods to combine these two approaches to gain the advantages of both. We make use of the risk of bias information that is routinely available in Cochrane reviews, by obtaining empirical distributions for the bias associated with particular bias profiles (combinations of risk of bias judgements). We propose three methods: a formal combination of empirical evidence and opinion in a Bayesian analysis; asking experts to give an opinion on bias informed by both summary trial information and a bias distribution from the empirical evidence, either numerically or by selecting areas of the empirical distribution. The methods are demonstrated through application to two example binary outcome meta‐analyses. Bias distributions based on opinion informed by trial information alone were most dispersed on average, and those based on opinions obtained by selecting areas of the empirical distribution were narrowest. Although the three methods for combining empirical evidence with opinion vary in ease and speed of implementation, they yielded similar results in the two examples.

Suggested Citation

  • K. M. Rhodes & J. Savović & R. Elbers & H. E. Jones & J. P. T. Higgins & J. A. C. Sterne & N. J. Welton & R. M. Turner, 2020. "Adjusting trial results for biases in meta‐analysis: combining data‐based evidence on bias with detailed trial assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(1), pages 193-209, January.
    • Handle: RePEc:bla:jorssa:v:183:y:2020:i:1:p:193-209
    DOI: 10.1111/rssa.12485
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12485
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssa.12485?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. David M. Phillippo & Sofia Dias & A. E. Ades & Vanessa Didelez & Nicky J. Welton, 2018. "Sensitivity of treatment recommendations to bias in network meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(3), pages 843-867, June.
    2. Sander Greenland, 2005. "Multiple‐bias modelling for analysis of observational data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 267-306, March.
    3. Rebecca M. Turner & David J. Spiegelhalter & Gordon C. S. Smith & Simon G. Thompson, 2009. "Bias modelling in evidence synthesis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 21-47, January.
    4. David M. Eddy, 1989. "The Confidence Profile Method: A Bayesian Method for Assessing Health Technologies," Operations Research, INFORMS, vol. 37(2), pages 210-228, April.
    5. N. J. Welton & A. E. Ades & J. B. Carlin & D. G. Altman & J. A. C. Sterne, 2009. "Models for potentially biased evidence in meta‐analysis using empirically based priors," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 119-136, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. McCandless Lawrence C., 2012. "Meta-Analysis of Observational Studies with Unmeasured Confounders," The International Journal of Biostatistics, De Gruyter, vol. 8(2), pages 1-31, January.
    2. S. Dias & N. J. Welton & V. C. C. Marinho & G. Salanti & J. P. T. Higgins & A. E. Ades, 2010. "Estimation and adjustment of bias in randomized evidence by using mixed treatment comparison meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(3), pages 613-629, July.
    3. Mathur, Maya B & VanderWeele, Tyler, 2018. "Statistical methods for evidence synthesis," Thesis Commons kd6ja, Center for Open Science.
    4. Ian Wadsworth & Lisa V. Hampson & Thomas Jaki & Graeme J. Sills & Anthony G. Marson & Richard Appleton, 2020. "A quantitative framework to inform extrapolation decisions in children," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(2), pages 515-534, February.
    5. Rebecca M Turner & Myfanwy Lloyd-Jones & Dilly O C Anumba & Gordon C S Smith & David J Spiegelhalter & Hazel Squires & John W Stevens & Michael J Sweeting & Stanislaw J Urbaniak & Robert Webster & Sim, 2012. "Routine Antenatal Anti-D Prophylaxis in Women Who Are Rh(D) Negative: Meta-Analyses Adjusted for Differences in Study Design and Quality," PLOS ONE, Public Library of Science, vol. 7(2), pages 1-10, February.
    6. Petra Schnell‐Inderst & Cynthia P. Iglesias & MARJAN Arvandi & ORIANA Ciani & Raffaella Matteucci Gothe & Jaime Peters & Ashley W. Blom & Rod S. Taylor & Uwe Siebert, 2017. "A bias‐adjusted evidence synthesis of RCT and observational data: the case of total hip replacement," Health Economics, John Wiley & Sons, Ltd., vol. 26(S1), pages 46-69, February.
    7. David M. Phillippo & Sofia Dias & A. E. Ades & Vanessa Didelez & Nicky J. Welton, 2018. "Sensitivity of treatment recommendations to bias in network meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(3), pages 843-867, June.
    8. repec:osf:thesis:kd6ja_v1 is not listed on IDEAS
    9. Desiree C Wilks & Stephen J Sharp & Ulf Ekelund & Simon G Thompson & Adrian P Mander & Rebecca M Turner & Susan A Jebb & Anna Karin Lindroos, 2011. "Objectively Measured Physical Activity and Fat Mass in Children: A Bias-Adjusted Meta-Analysis of Prospective Studies," PLOS ONE, Public Library of Science, vol. 6(2), pages 1-8, February.
    10. A. E. Ades & Karl Claxton & Mark Sculpher, 2006. "Evidence synthesis, parameter correlation and probabilistic sensitivity analysis," Health Economics, John Wiley & Sons, Ltd., vol. 15(4), pages 373-381, April.
    11. Lawrence C. McCandless & Sylvia Richardson & Nicky Best, 2012. "Adjustment for Missing Confounders Using External Validation Data and Propensity Scores," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(497), pages 40-51, March.
    12. Julian P. T. Higgins & Simon G. Thompson & David J. Spiegelhalter, 2009. "A re‐evaluation of random‐effects meta‐analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 137-159, January.
    13. Woertman, Willem & Vermeulen, Bram & Groenewoud, Hans & van der Wilt, Gert Jan, 2013. "Evidence based policy decisions through a Bayesian approach: The case of a statin appraisal in the Netherlands," Health Policy, Elsevier, vol. 112(3), pages 234-240.
    14. Douglas E. Schaubel & Guanghui Wei, 2011. "Double Inverse-Weighted Estimation of Cumulative Treatment Effects Under Nonproportional Hazards and Dependent Censoring," Biometrics, The International Biometric Society, vol. 67(1), pages 29-38, March.
    15. Carlos Díaz-Venegas, 2014. "Identifying the Confounders of Marginalization and Mortality in Mexico, 2003–2007," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 118(2), pages 851-875, September.
    16. Isabelle Albert & Emmanuelle Espié & Henriette de Valk & Jean‐Baptiste Denis, 2011. "A Bayesian Evidence Synthesis for Estimating Campylobacteriosis Prevalence," Risk Analysis, John Wiley & Sons, vol. 31(7), pages 1141-1155, July.
    17. A. Goubar & A. E. Ades & D. De Angelis & C. A. McGarrigle & C. H. Mercer & P. A. Tookey & K. Fenton & O. N. Gill, 2008. "Estimates of human immunodeficiency virus prevalence and proportion diagnosed based on Bayesian multiparameter synthesis of surveillance data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(3), pages 541-580, June.
    18. Xavier de Luna & Mathias Lundin, 2014. "Sensitivity analysis of the unconfoundedness assumption with an application to an evaluation of college choice effects on earnings," Journal of Applied Statistics, Taylor & Francis Journals, vol. 41(8), pages 1767-1784, August.
    19. A. E. Ades & A. J. Sutton, 2006. "Multiparameter evidence synthesis in epidemiology and medical decision‐making: current approaches," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(1), pages 5-35, January.
    20. C Elizabeth McCarron & Eleanor M Pullenayegum & Lehana Thabane & Ron Goeree & Jean-Eric Tarride, 2011. "Bayesian Hierarchical Models Combining Different Study Types and Adjusting for Covariate Imbalances: A Simulation Study to Assess Model Performance," PLOS ONE, Public Library of Science, vol. 6(10), pages 1-7, October.
    21. John J. Graff & Nalini Sathiakumar & Maurizio Macaluso & George Maldonado & Robert Matthews & Elizabeth Delzell, 2009. "The Effect of Uncertainty in Exposure Estimation on the Exposure-Response Relation between 1,3-Butadiene and Leukemia," IJERPH, MDPI, vol. 6(9), pages 1-20, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:183:y:2020:i:1:p:193-209. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.