A Tale of Two Surveys : Comparing the Outcomes of an In-Person and Web-Based Survey of MentalHealth in the West Bank and Gaza

Internet surveys may never replace in-person surveys as a gold standard, but they remainimportant tools for rapid, remote, and low-cost data collection. The West Bank and Gaza Poverty and Equity teamhad a unique opportunity to compare a Facebook survey with an in-person survey covering conflict exposure andpotentially associated socioeconomic and mental health outcomes over a similar time period. It is reasonable toexpect that the estimates from internet surveys and in-person surveys would differ. In this case, the Facebooksurvey estimates more severe outcomes (e.g., higher exposure to conflict and worse mental health) than its in-personcounterpart for most topics and populations. Multiple mechanisms may have contributed to this difference inestimates, including overrepresentation in the Facebook sample of respondents who were interested in the surveytopics, reduced sensitivity bias in the context of a self-administered online questionnaire, and reporting moresevere outcomes than personally experienced to encourage resource flows to perceived needs. Estimated outcomes tendto be more similar for people in Gaza, possibly because of greater homogeneity in socioeconomic experiences andexposure to violent conflict and broader interest in a survey on the effects of the May 2021 violence. The mainresults are robust to different ways of controlling for observable characteristics; neither alternative weights norsample restrictions erase the systematic differences between the surveys.