IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_9517.html
   My bibliography  Save this paper

Talk or Text? Evaluating Response Rates by Remote Survey Method during Covid-19

Author

Listed:
  • Sofia Amaral
  • Lelys Dinarte-Diaz
  • Patricio Dominguez
  • Steffanny Romero
  • Santiago M. Perez-Vincent

Abstract

Researchers and policy makers face significant challenges in selecting a method to conduct remote surveys, especially when collecting sensitive information or during turbulent life stages of hard-to-reach groups. In the context of the COVID-19 lockdown, we randomly selected about 600 adults in El Salvador to survey using two different tools: telephone interviews or a self-completion survey via WhatsApp. We find that phone-based surveys increase the rate of survey completion by 42 percentage points. We document even larger effects for women and older adults. Although direct costs of phone-based surveys are substantially larger—doubling implementation cost—our estimates imply that when adjusted for the probability of completion, the costs of conducting phone-based surveys can be 25 percent lower.

Suggested Citation

  • Sofia Amaral & Lelys Dinarte-Diaz & Patricio Dominguez & Steffanny Romero & Santiago M. Perez-Vincent, 2022. "Talk or Text? Evaluating Response Rates by Remote Survey Method during Covid-19," CESifo Working Paper Series 9517, CESifo.
    • Handle: RePEc:ces:ceswps:_9517
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp9517.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Peng Ding & Avi Feller & Luke Miratrix, 2016. "Randomization inference for treatment effect variation," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(3), pages 655-671, June.
    2. Abigail R Greenleaf & Aliou Gadiaga & Georges Guiella & Shani Turke & Noelle Battle & Saifuddin Ahmed & Caroline Moreau, 2020. "Comparability of modern contraceptive use estimates between a face-to-face survey and a cellphone survey among women in Burkina Faso," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-15, May.
    3. Philip Brenner & John DeLamater, 2014. "Social Desirability Bias in Self-reports of Physical Activity: Is an Exercise Identity the Culprit?," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 117(2), pages 489-504, June.
    4. Sergei Guriev & Nikita Melnikov & Ekaterina Zhuravskaya, 2021. "3G Internet and Confidence in Government," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 136(4), pages 2533-2613.
    5. Beegle, Kathleen & De Weerdt, Joachim & Friedman, Jed & Gibson, John, 2012. "Methods of household consumption measurement through surveys: Experimental results from Tanzania," Journal of Development Economics, Elsevier, vol. 98(1), pages 3-18.
    6. John Gibson & Kathleen Beegle & Joachim De Weerdt & Jed Friedman, 2015. "What does Variation in Survey Design Reveal about the Nature of Measurement Errors in Household Consumption?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(3), pages 466-474, June.
    7. Scott J. Savage & Donald M. Waldman, 2008. "Learning and fatigue during choice experiments: a comparison of online and mail survey modes," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 351-371.
    8. Özler, Berk & Çelik, Çiğdem & Cunningham, Scott & Cuevas, P. Facundo & Parisotto, Luca, 2021. "Children on the move: Progressive redistribution of humanitarian cash transfers among refugees," Journal of Development Economics, Elsevier, vol. 153(C).
    9. George W Pariyo & Abigail R Greenleaf & Dustin G Gibson & Joseph Ali & Hannah Selig & Alain B Labrique & Gulam Muhammed Al Kibria & Iqbal Ansary Khan & Honorati Masanja & Meerjady Sabrina Flora & Saif, 2019. "Does mobile phone survey method matter? Reliability of computer-assisted telephone interviews and interactive voice response non-communicable diseases risk factor surveys in low and middle income coun," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-25, April.
    10. Kelly L’Engle & Eunice Sefa & Edward Akolgo Adimazoya & Emmanuel Yartey & Rachel Lenzi & Cindy Tarpo & Nii Lante Heward-Mills & Katherine Lew & Yvonne Ampeh, 2018. "Survey research with a random digit dial national mobile phone sample in Ghana: Methods and sample quality," PLOS ONE, Public Library of Science, vol. 13(1), pages 1-11, January.
    11. Monteiro Amaral,Sofia Fernando & Dinarte Diaz,Lelys Ileana & Dominguez,Patricio & Perez-Vincent,Santiago M., 2021. "Helping Families Help Themselves ? Heterogeneous Effects of a Digital Parenting Program," Policy Research Working Paper Series 9850, The World Bank.
    12. Demombynes, Gabriel & Gubbins, Paul & Romeo, Alessandro, 2013. "Challenges and opportunities of mobile phone-based data collection : evidence from South Sudan," Policy Research Working Paper Series 6321, The World Bank.
    13. Joachim De Weerdt & John Gibson & Kathleen Beegle, 2020. "What Can We Learn from Experimenting with Survey Methods?," Annual Review of Resource Economics, Annual Reviews, vol. 12(1), pages 431-447, October.
    14. Cullen,Claire Alexis, 2020. "Method Matters : Underreporting of Intimate Partner Violence in Nigeria and Rwanda," Policy Research Working Paper Series 9274, The World Bank.
    15. Brian Dillon, 2012. "Using mobile phones to collect panel data in developing countries," Journal of International Development, John Wiley & Sons, Ltd., vol. 24(4), pages 518-527, May.
    16. Lindhjem, Henrik & Navrud, Ståle, 2011. "Are Internet surveys an alternative to face-to-face interviews in contingent valuation?," Ecological Economics, Elsevier, vol. 70(9), pages 1628-1637, July.
    17. Ben Leo, Robert Morello, Jonathan Mellon, Tiago Peixoto, and Stephen Davenport, 2015. "Do Mobile Phone Surveys Work in Poor Countries? - Working Paper 398," Working Papers 398, Center for Global Development.
    18. Michele Cocco & Arjuna Tuzzi, 2013. "New data collection modes for surveys: a comparative analysis of the influence of survey mode on question-wording effects," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(6), pages 3135-3152, October.
    19. Matthew W. Ridley & Gautam Rao & Frank Schilbach & Vikram H. Patel, 2020. "Poverty, Depression, and Anxiety: Causal Evidence and Mechanisms," NBER Working Papers 27157, National Bureau of Economic Research, Inc.
    20. Joachim De Weerdt & Kathleen Beegle & Jed Friedman & John Gibson, 2016. "The Challenge of Measuring Hunger through Survey," Economic Development and Cultural Change, University of Chicago Press, vol. 64(4), pages 727-758.
    21. Andrew Dabalen & Alvin Etang & Johannes Hoogeveen & Elvis Mushi & Youdi Schipper & Johannes von Engelhardt, 2016. "Mobile Phone Panel Surveys in Developing Countries," World Bank Publications - Books, The World Bank Group, number 24595, December.
    22. Robert Garlick & Kate Orkin & Simon Quinn, 2020. "Call Me Maybe: Experimental Evidence on Frequency and Medium Effects in Microenterprise Surveys," The World Bank Economic Review, World Bank, vol. 34(2), pages 418-443.
    23. Friedman, Jed & Beegle, Kathleen & De Weerdt, Joachim & Gibson, John, 2017. "Decomposing response error in food consumption measurement: Implications for survey design from a randomized survey experiment in Tanzania," Food Policy, Elsevier, vol. 72(C), pages 94-111.
    24. Charles Q. Lau & Eric Johnson & Ashley Amaya & Patricia LeBaron & Herschel Sanders, 2018. "High Stakes, Low Resources: What Mode(s) Should Youth Employment Training Programs Use to Track Alumni? Evidence From South Africa," Journal of International Development, John Wiley & Sons, Ltd., vol. 30(7), pages 1166-1185, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Amaral, Sofia & Dinarte-Diaz, Lelys & Dominguez, Patricio & Perez-Vincent, Santiago M., 2024. "Helping families help themselves: The (Un)intended impacts of a digital parenting program," Journal of Development Economics, Elsevier, vol. 166(C).
    2. Beam, Emily A., 2023. "Social media as a recruitment and data collection tool: Experimental evidence on the relative effectiveness of web surveys and chatbots," Journal of Development Economics, Elsevier, vol. 162(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abate, Gashaw T. & de Brauw, Alan & Hirvonen, Kalle & Wolle, Abdulazize, 2023. "Measuring consumption over the phone: Evidence from a survey experiment in urban Ethiopia," Journal of Development Economics, Elsevier, vol. 161(C).
    2. Joachim De Weerdt & John Gibson & Kathleen Beegle, 2020. "What Can We Learn from Experimenting with Survey Methods?," Annual Review of Resource Economics, Annual Reviews, vol. 12(1), pages 431-447, October.
    3. Elisa M. Maffioli, 2020. "Collecting Data During an Epidemic: A Novel Mobile Phone Research Method," Journal of International Development, John Wiley & Sons, Ltd., vol. 32(8), pages 1231-1255, November.
    4. Ameye, Hannah & De Weerdt, Joachim & Gibson, John, 2021. "Measuring macro- and micronutrient consumption in multi-purpose surveys: Evidence from a survey experiment in Tanzania," Food Policy, Elsevier, vol. 102(C).
    5. Zezza,Alberto & Mcgee,Kevin Robert & Wollburg,Philip Randolph & Assefa,Thomas Woldu & Gourlay,Sydney, 2022. "From Necessity to Opportunity : Lessons for Integrating Phone and In-Person Data Collectionfor Agricultural Statistics in a Post-Pandemic World," Policy Research Working Paper Series 10168, The World Bank.
    6. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    7. Gashaw T Abate & Alan de Brauw & John Gibson & Kalle Hirvonen & Abdulazize Wolle, 2022. "Telescoping Error in Recalled Food Consumption: Evidence from a Survey Experiment in Ethiopia [Video-Based Behavioral Change Communication to Change Consumption Patterns: Experimental Evidence from," The World Bank Economic Review, World Bank, vol. 36(4), pages 889-908.
    8. Abay, Kibrom A. & Berhane, Guush & Hoddinott, John F. & Tafere, Kibrom, 2021. "Assessing response fatigue in phone surveys: Experimental evidence on dietary diversity in Ethiopia," IFPRI discussion papers 2017, International Food Policy Research Institute (IFPRI).
    9. Carletto,Calogero & Dillon,Andrew S. & Zezza,Alberto, 2021. "Agricultural Data Collection to Minimize Measurement Error and Maximize Coverage," Policy Research Working Paper Series 9745, The World Bank.
    10. Hannah Ameye & Joachim De Weerdt & John Gibson, 2020. "Measuring Macro- and Micronutrient Intake in Multi-Purpose Surveys: Evidence from a Survey Experiment in Tanzania," LICOS Discussion Papers 42120, LICOS - Centre for Institutions and Economic Performance, KU Leuven.
    11. Kilic, Talip & Moylan, Heather & Ilukor, John & Mtengula, Clement & Pangapanga-Phiri, Innocent, 2021. "Root for the tubers: Extended-harvest crop production and productivity measurement in surveys," Food Policy, Elsevier, vol. 102(C).
    12. Beam, Emily A., 2022. "Social Media as a Recruitment and Data Collection Tool: Experimental Evidence on the Relative Effectiveness of Web Surveys and Chatbots," IZA Discussion Papers 15597, Institute of Labor Economics (IZA).
    13. Beam, Emily A., 2023. "Social media as a recruitment and data collection tool: Experimental evidence on the relative effectiveness of web surveys and chatbots," Journal of Development Economics, Elsevier, vol. 162(C).
    14. Arthi, Vellore & Beegle, Kathleen & De Weerdt, Joachim & Palacios-López, Amparo, 2018. "Not your average job: Measuring farm labor in Tanzania," Journal of Development Economics, Elsevier, vol. 130(C), pages 160-172.
    15. Johanna Choumert-Nkolo & Pascale Phelinas, 2018. "New paradigms for household surveys in low and middle income countries [Nouveaux paradigmes d'élaboration des enquêtes ménages dans les pays du Sud]," CERDI Working papers halshs-01888609, HAL.
    16. Fiala, Nathan & Masselus, Lise, 2022. "Whom to ask? Testing respondent effects in household surveys," Ruhr Economic Papers 935, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    17. Godlonton, Susan & Hernandez, Manuel A. & Paz, Cynthia, 2021. "Can survey design reduce anchoring bias in recall data? Evidence from Malawi," IFPRI discussion papers 2055, International Food Policy Research Institute (IFPRI).
    18. Joshua Brubaker & Talip Kilic & Philip Wollburg, 2021. "Representativeness of individual-level data in COVID-19 phone surveys: Findings from Sub-Saharan Africa," PLOS ONE, Public Library of Science, vol. 16(11), pages 1-27, November.
    19. Zezza, Alberto & Carletto, Gero & Fiedler, John L & Gennari, Pietro & Jolliffe, Dean M, 2017. "Food Counts. Measuring Food Consumption And Expenditures In Household Consumption And Expenditure Surveys (HCES)," 2017 International Congress, August 28-September 1, 2017, Parma, Italy 260886, European Association of Agricultural Economists.
    20. Friedman, Jed & Beegle, Kathleen & De Weerdt, Joachim & Gibson, John, 2017. "Decomposing response error in food consumption measurement: Implications for survey design from a randomized survey experiment in Tanzania," Food Policy, Elsevier, vol. 72(C), pages 94-111.

    More about this item

    Keywords

    phone surveys; WhatsApp surveys; response rate; survey experiments;
    All these keywords.

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_9517. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.