IDEAS home Printed from https://ideas.repec.org/p/ags/iaae21/315175.html
   My bibliography  Save this paper

Representativeness of Individual-Level Data in COVID-19 Phone Surveys: Findings from Sub-Saharan Africa

Author

Listed:
  • Brubaker, Joshua
  • Kilic, Talip
  • Kilic, Talip

Abstract

The COVID-19 pandemic has created urgent demand for timely data, leading to a surge in mobile phone surveys for tracking the impacts of and responses to the pandemic. Using data from national phone surveys implemented in Ethiopia, Malawi, Nigeria and Uganda during the pandemic and the pre-COVID-19 national face-to-face surveys that served as the sampling frames for the phone surveys, this paper documents selection the biases in individual-level analyses based on phone survey data. In most cases, individual-level data are available only for phone survey respondents, who we find are more likely to be household heads or their spouses and non-farm enterprise owners, and on average, are older and better educated vis-a-vis the general adult population. These differences are the result of uneven access to mobile phones in the population and the way that phone survey respondents are selected. To improve the representativeness of individual-level analysis using phone survey data, we recalibrate the phone survey sampling weights based on propensity score adjustments that are derived from a model of an individual’s likelihood of being interviewed as a function of individual- and household-level attributes. We find that reweighting improves the representativeness of the estimates for phone survey respondents, moving them closer to those of the general adult population. This holds for both women and men and for a range of demographic, education, and labor market outcomes. However, reweighting increases the variance of the estimates and, in most cases, fails to overcome selection biases. This indicates limitations to deriving representative individual-level estimates from phone survey data. Obtaining reliable data on men and women through future phone surveys will require random selection of adult interviewees within sampled households.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Brubaker, Joshua & Kilic, Talip & Kilic, Talip, 2021. "Representativeness of Individual-Level Data in COVID-19 Phone Surveys: Findings from Sub-Saharan Africa," 2021 Conference, August 17-31, 2021, Virtual 315175, International Association of Agricultural Economists.
    • Handle: RePEc:ags:iaae21:315175
    DOI: 10.22004/ag.econ.315175
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/315175/files/0-0_Paper_18998_handout_541_0.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.22004/ag.econ.315175?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Khamis, Melanie & Prinz, Daniel & Newhouse, David Locke & Palacios-Lopez, Amparo & Pape, Utz Johann & Weber, Michael, 2021. "Early Labor Market Impacts of COVID-19 in Developing Countries," Jobs Group Papers, Notes, and Guides 32769257, The World Bank.
    2. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    3. Kanyanda,Shelton Sofiel Elisa & Markhof,Yannick Valentin & Wollburg,Philip Randolph & Zezza,Alberto, 2021. "Acceptance of COVID-19 Vaccines in Sub-Saharan Africa : Evidence from Six National Phone Surveys," Policy Research Working Paper Series 9739, The World Bank.
    4. Kilic,Talip & Van den Broeck,Goedele & Koolwal,Gayatri B. & Moylan,Heather G., 2020. "Are You Being Asked ? Impacts of Respondent Selection on Measuring Employment," Policy Research Working Paper Series 9152, The World Bank.
    5. Kilic, Talip & Moylan, Heather & Koolwal, Gayatri, 2021. "Getting the (Gender-Disaggregated) lay of the land: Impact of survey respondent selection on measuring land ownership and rights," World Development, Elsevier, vol. 146(C).
    6. Kelly L’Engle & Eunice Sefa & Edward Akolgo Adimazoya & Emmanuel Yartey & Rachel Lenzi & Cindy Tarpo & Nii Lante Heward-Mills & Katherine Lew & Yvonne Ampeh, 2018. "Survey research with a random digit dial national mobile phone sample in Ghana: Methods and sample quality," PLOS ONE, Public Library of Science, vol. 13(1), pages 1-11, January.
    7. Quisumbing, Agnes R. & Kumar, Neha & Meinzen-Dick, Ruth Suseela & Ringler, Claudia, 2020. "Why gender matters in COVID-19 responses: Now and in the future," IFPRI book chapters, in: COVID-19 and global food security, chapter 20, pages 88-90, International Food Policy Research Institute (IFPRI).
    8. Josephson,Anna Leigh & Kilic,Talip & Michler,Jeffrey David, 2020. "Socioeconomic Impacts of COVID-19 in Four African Countries," Policy Research Working Paper Series 9466, The World Bank.
    9. Ben Leo, Robert Morello, Jonathan Mellon, Tiago Peixoto, and Stephen Davenport, 2015. "Do Mobile Phone Surveys Work in Poor Countries? - Working Paper 398," Working Papers 398, Center for Global Development.
    10. Furbush,Ann & Josephson,Anna Leigh & Kilic,Talip & Michler,Jeffrey David, 2021. "The Evolving Socioeconomic Impacts of COVID-19 in Four African Countries," Policy Research Working Paper Series 9556, The World Bank.
    11. Anna Josephson & Talip Kilic & Jeffrey D. Michler, 2021. "Socioeconomic impacts of COVID-19 in low-income countries," Nature Human Behaviour, Nature, vol. 5(5), pages 557-565, May.
    12. Andrew Dabalen & Alvin Etang & Johannes Hoogeveen & Elvis Mushi & Youdi Schipper & Johannes von Engelhardt, 2016. "Mobile Phone Panel Surveys in Developing Countries," World Bank Publications - Books, The World Bank Group, number 24595.
    13. Ambel,Alemayehu A. & Mcgee,Kevin Robert & Tsegay,Asmelash Haile, 2021. "Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries," Policy Research Working Paper Series 9676, The World Bank.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kugler, Maurice & Viollaz, Mariana & Duque, Daniel & Gaddis, Isis & Newhouse, David & Palacios-Lopez, Amparo & Weber, Michael, 2023. "How did the COVID-19 crisis affect different types of workers in the developing world?," World Development, Elsevier, vol. 170(C).
    2. Baumüller, Heike & Kornher, Lukas, 2024. "Inside the crowd: Assessing the suitability of SMSbased surveys to monitor the food security situation in Uganda," IAAE 2024 Conference, August 2-7, 2024, New Delhi, India 344389, International Association of Agricultural Economists (IAAE).
    3. Tabakis,Chrysostomos & Ten,Gi Khan & Merfeld,Joshua David & Newhouse,David Locke & Pape,Utz Johann & Weber,Michael, 2022. "The Welfare Implications of COVID-19 for Fragile and Conflict-Affected Areas," Policy Research Working Paper Series 10081, The World Bank.
    4. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    5. Kanyanda,Shelton Sofiel Elisa & Markhof,Yannick Valentin & Wollburg,Philip Randolph & Zezza,Alberto, 2021. "Acceptance of COVID-19 Vaccines in Sub-Saharan Africa : Evidence from Six National Phone Surveys," Policy Research Working Paper Series 9739, The World Bank.
    6. Martin Paul Jr. Tabe-Ojong & Emmanuel Nshakira-Rukundo & Bisrat Haile Gebrekidan, 2023. "COVID-19 and food insecurity in Africa: A review of the emerging empirical evidence," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 50(3), pages 853-878.
    7. Calogero Carletto, 2021. "Better data, higher impact: improving agricultural data systems for societal change [Correlated non-classical measurement errors, ‘second best’ policy inference, and the inverse size-productivity r," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 48(4), pages 719-740.
    8. Margherita Squarcina & Eva-Maria Egger, 2022. "Effects of the COVID-19 crisis on household food consumption and child nutrition in Mozambique," WIDER Working Paper Series wp-2022-169, World Institute for Development Economic Research (UNU-WIDER).
    9. Semakula, Henry Musoke & Liang, Song & Mukwaya, Paul Isolo & Mugagga, Frank, 2023. "Application of a Bayesian network modelling approach to predict the cascading effects of COVID-19 restrictions on the planting activities of smallholder farmers in Uganda," Agricultural Systems, Elsevier, vol. 211(C).
    10. Zezza,Alberto & Mcgee,Kevin Robert & Wollburg,Philip Randolph & Assefa,Thomas Woldu & Gourlay,Sydney, 2022. "From Necessity to Opportunity : Lessons for Integrating Phone and In-Person Data Collectionfor Agricultural Statistics in a Post-Pandemic World," Policy Research Working Paper Series 10168, The World Bank.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    2. Carletto,Calogero & Dillon,Andrew S. & Zezza,Alberto, 2021. "Agricultural Data Collection to Minimize Measurement Error and Maximize Coverage," Policy Research Working Paper Series 9745, The World Bank.
    3. Sofia Amaral & Lelys Dinarte-Diaz & Patricio Dominguez & Steffanny Romero & Santiago M. Perez-Vincent, 2022. "Talk or Text? Evaluating Response Rates by Remote Survey Method during Covid-19," CESifo Working Paper Series 9517, CESifo.
    4. Anderson, Ellen & Lybbert, Travis J. & Shenoy, Ashish & Singh, Rupika & Stein, Daniel, 2024. "Does survey mode matter? Comparing in-person and phone agricultural surveys in India," Journal of Development Economics, Elsevier, vol. 166(C).
    5. Elisa M. Maffioli, 2020. "Collecting Data During an Epidemic: A Novel Mobile Phone Research Method," Journal of International Development, John Wiley & Sons, Ltd., vol. 32(8), pages 1231-1255, November.
    6. Calogero Carletto, 2021. "Better data, higher impact: improving agricultural data systems for societal change [Correlated non-classical measurement errors, ‘second best’ policy inference, and the inverse size-productivity r," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 48(4), pages 719-740.
    7. Zezza,Alberto & Mcgee,Kevin Robert & Wollburg,Philip Randolph & Assefa,Thomas Woldu & Gourlay,Sydney, 2022. "From Necessity to Opportunity : Lessons for Integrating Phone and In-Person Data Collectionfor Agricultural Statistics in a Post-Pandemic World," Policy Research Working Paper Series 10168, The World Bank.
    8. Kugler, Maurice & Viollaz, Mariana & Duque, Daniel & Gaddis, Isis & Newhouse, David & Palacios-Lopez, Amparo & Weber, Michael, 2023. "How did the COVID-19 crisis affect different types of workers in the developing world?," World Development, Elsevier, vol. 170(C).
    9. Abdullah Kaviani Rad & Redmond R. Shamshiri & Hassan Azarm & Siva K. Balasundram & Muhammad Sultan, 2021. "Effects of the COVID-19 Pandemic on Food Security and Agriculture in Iran: A Survey," Sustainability, MDPI, vol. 13(18), pages 1-20, September.
    10. Joanna Upton & Elizabeth Tennant & Kathryn J. Fiorella & Christopher B. Barrett, 2023. "COVID-19, Household Resilience, and Rural Food Systems: Evidence from Southern and Eastern Africa," Palgrave Studies in Agricultural Economics and Food Policy, in: Christophe Béné & Stephen Devereux (ed.), Resilience and Food Security in a Food Systems Context, chapter 0, pages 281-320, Palgrave Macmillan.
    11. Tilman Brück & Mekdim D. Regassa, 2023. "Usefulness and misrepresentation of phone surveys on COVID-19 and food security in Africa," Food Security: The Science, Sociology and Economics of Food Production and Access to Food, Springer;The International Society for Plant Pathology, vol. 15(2), pages 423-453, April.
    12. Abate, Gashaw T. & de Brauw, Alan & Hirvonen, Kalle & Wolle, Abdulazize, 2023. "Measuring consumption over the phone: Evidence from a survey experiment in urban Ethiopia," Journal of Development Economics, Elsevier, vol. 161(C).
    13. Ahmed Mushfiq Mobarak & Edward Miguel, 2022. "The Economics of the COVID-19 Pandemic in Poor Countries," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 253-285, August.
    14. Ambel, Alemayehu & McGee, Kevin & Tsegay, Asmelash, 2021. "Reducing Bias in Phone Survey Samples: Effectiveness of Reweighting Techniques Using Face-to-Face Surveys As Frames," 2021 Conference, August 17-31, 2021, Virtual 315301, International Association of Agricultural Economists.
    15. Kalle Hirvonen & Alan de Brauw & Gashaw T. Abate, 2021. "Food Consumption and Food Security during the COVID‐19 Pandemic in Addis Ababa," American Journal of Agricultural Economics, John Wiley & Sons, vol. 103(3), pages 772-789, May.
    16. Elena Serfilippi & Daniele Giovannucci & David Ameyaw & Ankur Bansal & Thomas Asafua Nketsia Wobill & Roberta Blankson & Rashi Mishra, 2022. "Benefits and Challenges of Making Data More Agile: A Review of Recent Key Approaches in Agriculture," Sustainability, MDPI, vol. 14(24), pages 1-18, December.
    17. World Bank Group, 2016. "Republic of Guinea," World Bank Publications - Reports 24724, The World Bank Group.
    18. Aravind Moorthy & Thomas Coen & Katie Naeve & Jeremy Brecher-Haimson & Sarah Hughes, "undated". "Evaluation of the Irrigation and Water Resource Management Project in Senegal: Design Report," Mathematica Policy Research Reports c9f29d5746b14250b145a2030, Mathematica Policy Research.
    19. Paul,Boban Varghese & Finn,Arden Jeremy & Chaudhary,Sarang & Mayer Gukovas,Renata & Sundaram,Ramya, 2021. "COVID-19, Poverty, and Social Safety Net Response in Zambia," Policy Research Working Paper Series 9571, The World Bank.
    20. Esther Gehrke & Friederike Lenel & Claudia Schupp, 2023. "COVID-19 Crisis, Economic Hardships, and Schooling Outcomes," Education Finance and Policy, MIT Press, vol. 18(3), pages 522-546, Summer.

    More about this item

    Keywords

    Health Economics and Policy;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:iaae21:315175. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/iaaeeea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.