IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v88y2023i1d10.1007_s11336-022-09869-3.html
   My bibliography  Save this article

Bayesian Dynamic Borrowing of Historical Information with Applications to the Analysis of Large-Scale Assessments

Author

Listed:
  • David Kaplan

    (University of Wisconsin – Madison)

  • Jianshen Chen

    (The College Board)

  • Sinan Yavuz

    (University of Wisconsin – Madison)

  • Weicong Lyu

    (University of Wisconsin – Madison)

Abstract

The purpose of this paper is to demonstrate and evaluate the use of Bayesian dynamic borrowing (Viele et al, in Pharm Stat 13:41-54, 2014) as a means of systematically utilizing historical information with specific applications to large-scale educational assessments. Dynamic borrowing via Bayesian hierarchical models is a special case of a general framework of historical borrowing where the degree of borrowing depends on the heterogeneity among historical data and current data. A joint prior distribution over the historical and current data sets is specified with the degree of heterogeneity across the data sets controlled by the variance of the joint distribution. We apply Bayesian dynamic borrowing to both single-level and multilevel models and compare this approach to other historical borrowing methods such as complete pooling, Bayesian synthesis, and power priors. Two case studies using data from the Program for International Student Assessment reveal the utility of Bayesian dynamic borrowing in terms of predictive accuracy. This is followed by two simulation studies that reveal the utility of Bayesian dynamic borrowing over simple pooling and power priors in cases where the historical data is heterogeneous compared to the current data based on bias, mean squared error, and predictive accuracy. In cases of homogeneous historical data, Bayesian dynamic borrowing performs similarly to data pooling, Bayesian synthesis, and power priors. In contrast, for heterogeneous historical data, Bayesian dynamic borrowing performed at least as well, if not better, than other methods of borrowing with respect to mean squared error, percent bias, and leave-one-out cross-validation.

Suggested Citation

  • David Kaplan & Jianshen Chen & Sinan Yavuz & Weicong Lyu, 2023. "Bayesian Dynamic Borrowing of Historical Information with Applications to the Analysis of Large-Scale Assessments," Psychometrika, Springer;The Psychometric Society, vol. 88(1), pages 1-30, March.
  • Handle: RePEc:spr:psycho:v:88:y:2023:i:1:d:10.1007_s11336-022-09869-3
    DOI: 10.1007/s11336-022-09869-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-022-09869-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11336-022-09869-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jayne F Tierney & Claire Vale & Richard Riley & Catrin Tudur Smith & Lesley Stewart & Mike Clarke & Maroeska Rovers, 2015. "Individual Participant Data (IPD) Meta-analyses of Randomised Controlled Trials: Guidance on Their Use," PLOS Medicine, Public Library of Science, vol. 12(7), pages 1-16, July.
    2. Little, Roderick J., 2006. "Calibrated Bayes: A Bayes/Frequentist Roadmap," The American Statistician, American Statistical Association, vol. 60, pages 213-223, August.
    3. Lewandowski, Daniel & Kurowicka, Dorota & Joe, Harry, 2009. "Generating random correlation matrices based on vines and extended onion method," Journal of Multivariate Analysis, Elsevier, vol. 100(9), pages 1989-2001, October.
    4. Brian P. Hobbs & Bradley P. Carlin & Sumithra J. Mandrekar & Daniel J. Sargent, 2011. "Hierarchical Commensurate and Power Prior Models for Adaptive Incorporation of Historical Information in Clinical Trials," Biometrics, The International Biometric Society, vol. 67(3), pages 1047-1056, September.
    5. Rubin, Donald B, 1986. "Statistical Matching Using File Concatenation with Adjusted Weights and Multiple Imputations," Journal of Business & Economic Statistics, American Statistical Association, vol. 4(1), pages 87-94, January.
    6. Satoshi Morita & Peter F. Thall & Peter Müller, 2008. "Determining the Effective Sample Size of a Parametric Prior," Biometrics, The International Biometric Society, vol. 64(2), pages 595-602, June.
    7. Robert Mislevy, 1991. "Randomization-based inference about latent variables from complex samples," Psychometrika, Springer;The Psychometric Society, vol. 56(2), pages 177-196, June.
    8. Zhou, Xiang & Reiter, Jerome P., 2010. "A Note on Bayesian Inference After Multiple Imputation," The American Statistician, American Statistical Association, vol. 64(2), pages 159-163.
    9. Heinz Schmidli & Sandro Gsteiger & Satrajit Roychoudhury & Anthony O'Hagan & David Spiegelhalter & Beat Neuenschwander, 2014. "Robust meta-analytic-predictive priors in clinical trials with historical control information," Biometrics, The International Biometric Society, vol. 70(4), pages 1023-1032, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Stavros Nikolakopoulos & Ingeborg van der Tweel & Kit C. B. Roes, 2018. "Dynamic borrowing through empirical power priors that control type I error," Biometrics, The International Biometric Society, vol. 74(3), pages 874-880, September.
    2. Moreno Ursino & Nigel Stallard, 2021. "Bayesian Approaches for Confirmatory Trials in Rare Diseases: Opportunities and Challenges," IJERPH, MDPI, vol. 18(3), pages 1-9, January.
    3. Peng Yang & Yuansong Zhao & Lei Nie & Jonathon Vallejo & Ying Yuan, 2023. "SAM: Self‐adapting mixture prior to dynamically borrow information from historical data in clinical trials," Biometrics, The International Biometric Society, vol. 79(4), pages 2857-2868, December.
    4. Heinz Schmidli & Sandro Gsteiger & Satrajit Roychoudhury & Anthony O'Hagan & David Spiegelhalter & Beat Neuenschwander, 2014. "Robust meta-analytic-predictive priors in clinical trials with historical control information," Biometrics, The International Biometric Society, vol. 70(4), pages 1023-1032, December.
    5. Wenlin Yuan & Ming-Hui Chen & John Zhong, 2022. "Flexible Conditional Borrowing Approaches for Leveraging Historical Data in the Bayesian Design of Superiority Trials," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(2), pages 197-215, July.
    6. Egidi, Leonardo, 2022. "Effective sample size for a mixture prior," Statistics & Probability Letters, Elsevier, vol. 183(C).
    7. Jingjing Ye & Gregory Reaman, 2022. "Improving Early Futility Determination by Learning from External Data in Pediatric Cancer Clinical Trials," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(2), pages 337-351, July.
    8. Lanju Zhang & Zailong Wang & Li Wang & Lu Cui & Jeremy Sokolove & Ivan Chan, 2022. "A Simple Approach to Incorporating Historical Control Data in Clinical Trial Design and Analysis," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(2), pages 216-236, July.
    9. Schmidli, Heinz & Neuenschwander, Beat & Friede, Tim, 2017. "Meta-analytic-predictive use of historical variance data for the design and analysis of clinical trials," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 100-110.
    10. Chenghao Chu & Bingming Yi, 2021. "Dynamic historical data borrowing using weighted average," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(5), pages 1259-1280, November.
    11. Alexander Kaizer & John Kittelson, 2020. "Discussion on “Predictively Consistent Prior Effective Sample Sizes” by Beat Neuenschwander, Sebastian Weber, Heinz Schmidli, and Anthony O'Hagan," Biometrics, The International Biometric Society, vol. 76(2), pages 588-590, June.
    12. Danila Azzolina & Giulia Lorenzoni & Silvia Bressan & Liviana Da Dalt & Ileana Baldi & Dario Gregori, 2021. "Handling Poor Accrual in Pediatric Trials: A Simulation Study Using a Bayesian Approach," IJERPH, MDPI, vol. 18(4), pages 1-16, February.
    13. Chen, Nan & Carlin, Bradley P. & Hobbs, Brian P., 2018. "Web-based statistical tools for the analysis and design of clinical trials that incorporate historical controls," Computational Statistics & Data Analysis, Elsevier, vol. 127(C), pages 50-68.
    14. Liyun Jiang & Lei Nie & Ying Yuan, 2023. "Elastic priors to dynamically borrow information from historical data in clinical trials," Biometrics, The International Biometric Society, vol. 79(1), pages 49-60, March.
    15. Ian Wadsworth & Lisa V. Hampson & Thomas Jaki & Graeme J. Sills & Anthony G. Marson & Richard Appleton, 2020. "A quantitative framework to inform extrapolation decisions in children," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(2), pages 515-534, February.
    16. Andrea Arfè & Brian Alexander & Lorenzo Trippa, 2021. "Optimality of testing procedures for survival data in the nonproportional hazards setting," Biometrics, The International Biometric Society, vol. 77(2), pages 587-598, June.
    17. Beat Neuenschwander & Sebastian Weber & Heinz Schmidli & Anthony O'Hagan, 2020. "Predictively consistent prior effective sample sizes," Biometrics, The International Biometric Society, vol. 76(2), pages 578-587, June.
    18. Md. Tuhin Sheikh & Ming-Hui Chen & Jonathan A. Gelfond & Joseph G. Ibrahim, 2022. "A Power Prior Approach for Leveraging External Longitudinal and Competing Risks Survival Data Within the Joint Modeling Framework," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(2), pages 318-336, July.
    19. Arnaud Monseur & Bradley P. Carlin & Bruno Boulanger & Andreea Seferian & Laurent Servais & Chris Freitag & Leen Thielemans, 2022. "Leveraging Natural History Data in One- and Two-Arm Hierarchical Bayesian Studies of Rare Disease Progression," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 14(2), pages 237-258, July.
    20. François Gardes, 2021. "On the value of time and human life," Documents de travail du Centre d'Economie de la Sorbonne 21023, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:88:y:2023:i:1:d:10.1007_s11336-022-09869-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.