A systematic review and evaluation of Zika virus forecasting and prediction research during a public health emergency of international concern

Introduction: Epidemic forecasting and prediction tools have the potential to provide actionable information in the midst of emerging epidemics. While numerous predictive studies were published during the 2016–2017 Zika Virus (ZIKV) pandemic, it remains unknown how timely, reproducible, and actionable the information produced by these studies was. Methods: To improve the functional use of mathematical modeling in support of future infectious disease outbreaks, we conducted a systematic review of all ZIKV prediction studies published during the recent ZIKV pandemic using the PRISMA guidelines. Using MEDLINE, EMBASE, and grey literature review, we identified studies that forecasted, predicted, or simulated ecological or epidemiological phenomena related to the Zika pandemic that were published as of March 01, 2017. Eligible studies underwent evaluation of objectives, data sources, methods, timeliness, reproducibility, accessibility, and clarity by independent reviewers. Results: 2034 studies were identified, of which n = 73 met the eligibility criteria. Spatial spread, R0 (basic reproductive number), and epidemic dynamics were most commonly predicted, with few studies predicting Guillain-Barré Syndrome burden (4%), sexual transmission risk (4%), and intervention impact (4%). Most studies specifically examined populations in the Americas (52%), with few African-specific studies (4%). Case count (67%), vector (41%), and demographic data (37%) were the most common data sources. Real-time internet data and pathogen genomic information were used in 7% and 0% of studies, respectively, and social science and behavioral data were typically absent in modeling efforts. Deterministic models were favored over stochastic approaches. Forty percent of studies made model data entirely available, 29% provided all relevant model code, 43% presented uncertainty in all predictions, and 54% provided sufficient methodological detail to allow complete reproducibility. Fifty-one percent of predictions were published after the epidemic peak in the Americas. While the use of preprints improved the accessibility of ZIKV predictions by a median of 119 days sooner than journal publication dates, they were used in only 30% of studies. Conclusions: Many ZIKV predictions were published during the 2016–2017 pandemic. The accessibility, reproducibility, timeliness, and incorporation of uncertainty in these published predictions varied and indicates there is substantial room for improvement. To enhance the utility of analytical tools for outbreak response it is essential to improve the sharing of model data, code, and preprints for future outbreaks, epidemics, and pandemics. Author summary: Researchers published many studies which sought to predict and forecast important features of Zika virus (ZIKV) infections and their spread during the 2016–2017 ZIKV pandemic. We conducted a comprehensive review of such ZIKV prediction studies and evaluated their aims, the data sources they used, which methods were used, how timely they were published, and whether they provided sufficient information to be used or reproduced by others. Of the 73 studies evaluated, we found the accessibility, reproducibility, timeliness, and incorporation of uncertainty in these published predictions varied; indicating there is substantial room for improvement. We identified that the release of study findings before formal journal publication (‘pre-prints’) increased the timeliness of Zika prediction studies, but note they were infrequently used during this public health emergency. Addressing these areas can improve our understanding of Zika and other outbreaks and ensure forecasts can inform preparedness and response to future outbreaks, epidemics, and pandemics.