Reducing uncertainty in Delphi surveys: A case study on immigration to the EU

Background: Following the rapid increase of asylum seekers arriving in the European Union in 2015/16, policymakers have invested heavily in improving their foresight and forecasting capabilities. A common method to elicit expert predictions are Delphi surveys. This approach has attracted concern in the literature, given the high uncertainty in experts’ predictions. However, there exists limited guidance on specific design choices for future-related Delphi surveys. Objective: We test whether or not small adjustments to the Delphi survey can increase certainty (i.e., reduce variation) in expert predictions on immigration to the EU in 2030. Methods: Based on a two-round Delphi survey with 178 migration experts, we compare variation and subjective confidence in expert predictions and assess whether additional context information (type of migration flow, sociopolitical context) promotes convergence among experts (i.e., less variation) and confidence in their own estimates. Results: We find that additional context information does not reduce variation and does not increase confidence in expert predictions on migration. Conclusions: The results reaffirm recent concerns regarding the limited scope for reducing uncertainty by manipulating the survey setup. Persistent uncertainty may be a result of the complexity of migration processes and limited agreement among migration experts regarding key drivers. Contribution: We caution policymakers and academics on the use of Delphi surveys for eliciting expert predictions on immigration, even when conducted based on a large pool of experts and using specific scenarios. The potential of alternative approaches such as prediction markets should be further explored.