IDEAS home Printed from https://ideas.repec.org/a/bla/jorssc/v71y2022i3p669-697.html
   My bibliography  Save this article

Generalizing trial evidence to target populations in non‐nested designs: Applications to AIDS clinical trials

Author

Listed:
  • Fan Li
  • Ashley L. Buchanan
  • Stephen R. Cole

Abstract

Comparative effectiveness evidence from randomized trials may not be directly generalizable to a target population of substantive interest when, as in most cases, trial participants are not randomly sampled from the target population. Motivated by the need to generalize evidence from two trials conducted in the AIDS Clinical Trials Group (ACTG), we consider weighting, regression and doubly robust estimators to estimate the causal effects of HIV interventions in a specified population of people living with HIV in the USA. We focus on a non‐nested trial design and discuss strategies for both point and variance estimation of the target population average treatment effect. Specifically in the generalizability context, we demonstrate both analytically and empirically that estimating the known propensity score in trials does not increase the variance for each of the weighting, regression and doubly robust estimators. We apply these methods to generalize the average treatment effects from two ACTG trials to specified target populations and operationalize key practical considerations. Finally, we report on a simulation study that investigates the finite‐sample operating characteristics of the generalizability estimators and their sandwich variance estimators.

Suggested Citation

  • Fan Li & Ashley L. Buchanan & Stephen R. Cole, 2022. "Generalizing trial evidence to target populations in non‐nested designs: Applications to AIDS clinical trials," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(3), pages 669-697, June.
  • Handle: RePEc:bla:jorssc:v:71:y:2022:i:3:p:669-697
    DOI: 10.1111/rssc.12550
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssc.12550
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssc.12550?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Ashley L. Buchanan & Michael G. Hudgens & Stephen R. Cole & Katie R. Mollan & Paul E. Sax & Eric S. Daar & Adaora A. Adimora & Joseph J. Eron & Michael J. Mugavero, 2018. "Generalizing evidence from randomized trials using inverse probability of sampling weights," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1193-1209, October.
    2. Hitomi, Kohtaro & Nishiyama, Yoshihiko & Okui, Ryo, 2008. "A Puzzling Phenomenon In Semiparametric Estimation Problems With Infinite-Dimensional Nuisance Parameters," Econometric Theory, Cambridge University Press, vol. 24(6), pages 1717-1728, December.
    3. Wooldridge, Jeffrey M., 2007. "Inverse probability weighted estimation for general missing data problems," Journal of Econometrics, Elsevier, vol. 141(2), pages 1281-1301, December.
    4. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    5. Kara E. Rudolph & Mark J. Laan, 2017. "Robust estimation of encouragement design intervention effects transported across sites," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1509-1525, November.
    6. Hudgens, Michael G. & Halloran, M. Elizabeth, 2008. "Toward Causal Inference With Interference," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 832-842, June.
    7. Heejung Bang & James M. Robins, 2005. "Doubly Robust Estimation in Missing Data and Causal Inference Models," Biometrics, The International Biometric Society, vol. 61(4), pages 962-973, December.
    8. Elizabeth A. Stuart & Stephen R. Cole & Catherine P. Bradshaw & Philip J. Leaf, 2011. "The use of propensity scores to assess the generalizability of results from randomized trials," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(2), pages 369-386, April.
    9. Erin Hartman & Richard Grieve & Roland Ramsahai & Jasjeet S. Sekhon, 2015. "From sample average treatment effect to population average treatment effect on the treated: combining experimental with observational studies to estimate population treatment effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(3), pages 757-778, June.
    10. White, Halbert, 1982. "Maximum Likelihood Estimation of Misspecified Models," Econometrica, Econometric Society, vol. 50(1), pages 1-25, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xinyu Li & Wang Miao & Fang Lu & Xiao‐Hua Zhou, 2023. "Improving efficiency of inference in clinical trials with external control data," Biometrics, The International Biometric Society, vol. 79(1), pages 394-403, March.
    2. Ruoxuan Xiong & Allison Koenecke & Michael Powell & Zhu Shen & Joshua T. Vogelstein & Susan Athey, 2021. "Federated Causal Inference in Heterogeneous Observational Data," Papers 2107.11732, arXiv.org, revised Apr 2023.
    3. Dasom Lee & Shu Yang & Lin Dong & Xiaofei Wang & Donglin Zeng & Jianwen Cai, 2023. "Improving trial generalizability using observational studies," Biometrics, The International Biometric Society, vol. 79(2), pages 1213-1225, June.
    4. Graham, Bryan S. & Pinto, Cristine Campos de Xavier, 2022. "Semiparametrically efficient estimation of the average linear regression function," Journal of Econometrics, Elsevier, vol. 226(1), pages 115-138.
    5. Ranjbar, Setareh & Salvati, Nicola & Pacini, Barbara, 2023. "Estimating heterogeneous causal effects in observational studies using small area predictors," Computational Statistics & Data Analysis, Elsevier, vol. 184(C).
    6. Ashley L. Buchanan & Michael G. Hudgens & Stephen R. Cole & Katie R. Mollan & Paul E. Sax & Eric S. Daar & Adaora A. Adimora & Joseph J. Eron & Michael J. Mugavero, 2018. "Generalizing evidence from randomized trials using inverse probability of sampling weights," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1193-1209, October.
    7. Prokhorov, Artem & Schmidt, Peter, 2009. "GMM redundancy results for general missing data problems," Journal of Econometrics, Elsevier, vol. 151(1), pages 47-55, July.
    8. Otávio Bartalotti, 2013. "GMM Efficiency and IPW Estimation for Nonsmooth Functions," Working Papers 1301, Tulane University, Department of Economics.
    9. Difang Huang & Jiti Gao & Tatsushi Oka, 2022. "Semiparametric Single-Index Estimation for Average Treatment Effects," Papers 2206.08503, arXiv.org, revised Oct 2022.
    10. Max H. Farrell, 2013. "Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations," Papers 1309.4686, arXiv.org, revised Feb 2018.
    11. Girma, Sourafel & Görg, Holger & Stepanok, Ignat, 2020. "Subsidies, spillovers and exports," Economics Letters, Elsevier, vol. 186(C).
    12. Rui Chen & Guanhua Chen & Menggang Yu, 2023. "Entropy balancing for causal generalization with target sample summary information," Biometrics, The International Biometric Society, vol. 79(4), pages 3179-3190, December.
    13. Meyer, Birgit, 2020. "How deep is your love? Innovation, Upgrading and the Depth of Internationalization," VfS Annual Conference 2020 (Virtual Conference): Gender Economics 224584, Verein für Socialpolitik / German Economic Association.
    14. Isaiah Andrews & Emily Oster, 2017. "A Simple Approximation for Evaluating External Validity Bias," NBER Working Papers 23826, National Bureau of Economic Research, Inc.
    15. Jun Du & Sourafel Girma & Holger Görg & Ignat Stepanok, 2023. "Who wins and who loses from state subsidies?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 56(3), pages 1007-1031, August.
    16. William Rhodes, 2010. "Heterogeneous Treatment Effects: What Does a Regression Estimate?," Evaluation Review, , vol. 34(4), pages 334-361, August.
    17. Cattaneo, Matias D., 2010. "Efficient semiparametric estimation of multi-valued treatment effects under ignorability," Journal of Econometrics, Elsevier, vol. 155(2), pages 138-154, April.
    18. Du, Jun & Girma, Sourafel & Görg, Holger & Stepanok, Ignat, 2022. "Who Wins and Who Loses from State Subsidies?," IZA Discussion Papers 15249, Institute of Labor Economics (IZA).
    19. Bryan S. Graham & Keisuke Hirano, 2011. "Robustness to Parametric Assumptions in Missing Data Models," American Economic Review, American Economic Association, vol. 101(3), pages 538-543, May.
    20. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssc:v:71:y:2022:i:3:p:669-697. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.