Making useful conflict predictions

One of the major issues in predicting state failure is the relatively rare occurrence of event onset. This class skew problem can cause difficulties in both estimating a model and selecting a decision boundary. Since the publication of King & Zeng’s studies in 2001, scholars have utilized case-control methods to address this issue. This article builds on the landmark research of the Political Instability Task Force comparing the case-control approach to several other methods from the machine learning field and some original to this study. Case-control methods have several practical disadvantages and show no measurable advantages in prediction. The article also introduces cost-sensitive methods for determining a decision boundary. This explication raises questions about the Task Force’s formulation of a decision boundary and suggests methods for making useful predictions for policy. I find that the decision boundary chosen by the PITF implicitly assumes that the cost of intervention is about 7.7% of the cost of non-intervention when state failure will take place. These findings demonstrate that there is still much work to be done in predicting state failure, especially in limiting the number of false positives. More generally, it suggests caution in using accuracy as a measure of success when significant class imbalance exists in the data.