Nonmechanistic forecasts of seasonal influenza with iterative one-week-ahead distributions

Accurate and reliable forecasts of seasonal epidemics of infectious disease can assist in the design of countermeasures and increase public awareness and preparedness. This article describes two main contributions we made recently toward this goal: a novel approach to probabilistic modeling of surveillance time series based on “delta densities”, and an optimization scheme for combining output from multiple forecasting methods into an adaptively weighted ensemble. Delta densities describe the probability distribution of the change between one observation and the next, conditioned on available data; chaining together nonparametric estimates of these distributions yields a model for an entire trajectory. Corresponding distributional forecasts cover more observed events than alternatives that treat the whole season as a unit, and improve upon multiple evaluation metrics when extracting key targets of interest to public health officials. Adaptively weighted ensembles integrate the results of multiple forecasting methods, such as delta density, using weights that can change from situation to situation. We treat selection of optimal weightings across forecasting methods as a separate estimation task, and describe an estimation procedure based on optimizing cross-validation performance. We consider some details of the data generation process, including data revisions and holiday effects, both in the construction of these forecasting methods and when performing retrospective evaluation. The delta density method and an adaptively weighted ensemble of other forecasting methods each improve significantly on the next best ensemble component when applied separately, and achieve even better cross-validated performance when used in conjunction. We submitted real-time forecasts based on these contributions as part of CDC’s 2015/2016 FluSight Collaborative Comparison. Among the fourteen submissions that season, this system was ranked by CDC as the most accurate.Author summary: Seasonal influenza is associated with 250 000 to 500 000 deaths worldwide each year (WHO estimates). In the United States and other temperate regions, seasonal influenza epidemics occur annually, but their timing and intensity varies significantly; accurate and reliable forecasts that quantify their uncertainty can assist policymakers when planning countermeasures such as vaccination campaigns, and increase awareness and preparedness of hospitals and the general public. Starting with the 2013/2014 flu season, CDC has solicited, collected, evaluated, and compared weekly forecasts from external research groups. We developed a new method for forecasting flu surveillance data, which stitches together models of changes that happen each week, and a way of combining its output with other forecasts. The resulting forecasting system produced the most accurate forecasts in CDC’s 2015/2016 FluSight comparison of fourteen forecasting systems. We describe our new forecasting methods, analyze their performance in the 2015/2016 comparison and on data from previous seasons, and describe idiosyncrasies of epidemiological data that should be considered when constructing and evaluating forecasting systems.