Comparing Predictive Accuracy of COVID-19 Prediction Models: A Case Study

Decision-making in the face of COVID-19 outbreaks is highly dependent on the accuracy of forecasting the development of the epidemic. The success of the fight against coronavirus depends largely on the soundness of the decisions made, which often entail serious economic consequences. Having received the forecast of the epidemic curve, the decision-making center can work out an independent decision or use the experience of other regions. In the first case, it is necessary to correctly assess the accuracy of the obtained forecasts, and in the second, it is required to find out how the observed conditions are similar to the prototype, in particular, whether the epidemic data have the same distribution. Traditional models for forecasting epidemics, based on differential equations and a number of factors characterized by high uncertainty, often give inaccurate predictions, forcing experts to analyze possible optimistic and pessimistic scenarios, rather than the most plausible course of events. These models are being replaced by other numerical alternatives and machine learning models that predict time series based on training samples. The training accuracy and generalization error of such algorithms usually boil down to cross-validation. In this case, it also becomes necessary to compare the distribution of errors on each of the training samples. Therefore, the comparison of predictive models can be reduced to checking the homogeneity of the samples of their errors. This chapter describes a simplified variant of the Klyushin-Petunin test for testing the homogeneity of two samples and the results of its application to compare two methods for predicting the epidemic curve of COVID-19 (cube spline and hybrid Euler method) using the cases of Germany, India, Italy, South Africa, and South Korea. We demonstrate the effectiveness and practicality of the predictive models proposed by the evaluation method.