Measuring the accuracy of gridded human population density surfaces: A case study in Bioko Island, Equatorial Guinea

Background: Geospatial datasets of population are becoming more common in models used for health policy. Publicly-available maps of human population make a consistent picture from inconsistent census data, and the techniques they use to impute data makes each population map unique. Each mapping model explains its methods, but it can be difficult to know which map is appropriate for which policy work. High quality census datasets, where available, are a unique opportunity to characterize maps by comparing them with truth. Methods: We use census data from a bed-net mass-distribution campaign on Bioko Island, Equatorial Guinea, conducted by the Bioko Island Malaria Elimination Program as a gold standard to evaluate LandScan (LS), WorldPop Constrained (WP-C) and WorldPop Unconstrained (WP-U), Gridded Population of the World (GPW), and the High-Resolution Settlement Layer (HRSL). Each layer is compared to the gold-standard using statistical measures to evaluate distribution, error, and bias. We investigated how map choice affects burden estimates from a malaria prevalence model. Results: Specific population layers were able to match the gold-standard distribution at different population densities. LandScan was able to most accurately capture highly urban distribution, HRSL and WP-C matched best at all other lower population densities. GPW and WP-U performed poorly everywhere. Correctly capturing empty pixels is key, and smaller pixel sizes (100 m vs 1 km) improve this. Normalizing areas based on known district populations increased performance. The use of differing population layers in a malaria model showed a disparity in results around transition points between endemicity levels. Discussion: The metrics in this paper, some of them novel in this context, characterize how these population maps differ from the gold standard census and from each other. We show that the metrics help understand the performance of a population map within a malaria model. The closest match to the census data would combine LandScan within urban areas and the HRSL for rural areas. Researchers should prefer particular maps if health calculations have a strong dependency on knowing where people are not, or if it is important to categorize variation in density within a city.