IDEAS home Printed from https://ideas.repec.org/a/eee/deveco/v162y2023ics030438782300024x.html
   My bibliography  Save this article

Social media as a recruitment and data collection tool: Experimental evidence on the relative effectiveness of web surveys and chatbots

Author

Listed:
  • Beam, Emily A.

Abstract

Online technologies enable lower-cost, rapid data collection, but concerns about sample composition biases and mode-specific measurement error impede their use. I conduct a randomized experiment in the Philippines to test the effectiveness of web-form and chatbot surveys of K–12 teachers recruited through social media and compare their effectiveness with phone surveys of teachers recruited from a pre-existing frame. Chatbot surveys yield higher response rates and higher-quality data than web-form surveys in terms of missed question and item differentiation. The results suggest that chatbot responses match CATI responses on multiple dimensions of quality. Relative to CATI, online methods also yield higher rates or information disclosure on potentially sensitive topics, revealing substantially higher levels of distress among teachers. I show that social media-based recruitment can be an attractive alternative for targeted sampling and that online surveys can be implemented effectively at a fraction of the cost of phone surveys.

Suggested Citation

  • Beam, Emily A., 2023. "Social media as a recruitment and data collection tool: Experimental evidence on the relative effectiveness of web surveys and chatbots," Journal of Development Economics, Elsevier, vol. 162(C).
    • Handle: RePEc:eee:deveco:v:162:y:2023:i:c:s030438782300024x
    DOI: 10.1016/j.jdeveco.2023.103069
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S030438782300024X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jdeveco.2023.103069?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    2. Ivar Krumpal, 2013. "Determinants of social desirability bias in sensitive surveys: a literature review," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(4), pages 2025-2047, June.
    3. Marcel Fafchamps & Bart Minten, 2012. "Impact of SMS-Based Agricultural Information on Indian Farmers," The World Bank Economic Review, World Bank, vol. 26(3), pages 383-414.
    4. Emily Oster, 2019. "Unobservable Selection and Coefficient Stability: Theory and Evidence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(2), pages 187-204, April.
    5. Arthi, Vellore & Beegle, Kathleen & De Weerdt, Joachim & Palacios-López, Amparo, 2018. "Not your average job: Measuring farm labor in Tanzania," Journal of Development Economics, Elsevier, vol. 130(C), pages 160-172.
    6. Robert Garlick & Kate Orkin & Simon Quinn, 2020. "Call Me Maybe: Experimental Evidence on Frequency and Medium Effects in Microenterprise Surveys," The World Bank Economic Review, World Bank, vol. 34(2), pages 418-443.
    7. David Sungho Park & Shilpa Aggarwal & Dahyeon Jeong & Naresh Kumar & Jonathan Robinson & Alan Spearot, 2021. "Private but Misunderstood? Evidence on Measuring Intimate Partner Violence via Self-Interviewing in Rural Liberia and Malawi," NBER Working Papers 29584, National Bureau of Economic Research, Inc.
    8. Monteiro Amaral,Sofia Fernando & Dinarte Diaz,Lelys Ileana & Dominguez,Patricio & Perez-Vincent,Santiago M. & Romero,Steffanny, 2022. "Talk or Text ? Evaluating Response Rates by Remote Survey Method during COVID-19," Policy Research Working Paper Series 9999, The World Bank.
    9. Brian Dillon, 2012. "Using mobile phones to collect panel data in developing countries," Journal of International Development, John Wiley & Sons, Ltd., vol. 24(4), pages 518-527, May.
    10. Leah Rosenzweig & Bence Bago & Adam J. Berinsky & David Rand, 2021. "Happiness and Surprise are associated with worse truth discernment of COVID-19 headlines among social media users in Nigeria," Post-Print hal-03509537, HAL.
    11. Assefa, Thomas W. & Kadam, Aditi & Magnan, Nicholas & McCullough, Ellen & McGavock, Tamara, 2022. "Who is asking and how? The effects of enumerator gender and survey method in measuring intimate partner violence," 2022 Annual Meeting, July 31-August 2, Anaheim, California 322543, Agricultural and Applied Economics Association.
    12. Charles Q. Lau & Eric Johnson & Ashley Amaya & Patricia LeBaron & Herschel Sanders, 2018. "High Stakes, Low Resources: What Mode(s) Should Youth Employment Training Programs Use to Track Alumni? Evidence From South Africa," Journal of International Development, John Wiley & Sons, Ltd., vol. 30(7), pages 1166-1185, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Beam, Emily A., 2022. "Social Media as a Recruitment and Data Collection Tool: Experimental Evidence on the Relative Effectiveness of Web Surveys and Chatbots," IZA Discussion Papers 15597, Institute of Labor Economics (IZA).
    2. Bensch, Gunther & Kluve, Jochen & Stöterau, Jonathan, 2021. "The market-based dissemination of energy-access technologies as a business model for rural entrepreneurs: Evidence from Kenya," Resource and Energy Economics, Elsevier, vol. 66(C).
    3. Monteiro Amaral,Sofia Fernando & Dinarte Diaz,Lelys Ileana & Dominguez,Patricio & Perez-Vincent,Santiago M. & Romero,Steffanny, 2022. "Talk or Text ? Evaluating Response Rates by Remote Survey Method during COVID-19," Policy Research Working Paper Series 9999, The World Bank.
    4. Pritchett, Lant, 2023. "Rely (only) on the rigorous evidence” is bad advice," LSE Research Online Documents on Economics 119818, London School of Economics and Political Science, LSE Library.
    5. Mikel Bedayo & Gabriel Jiménez & José-Luis Peydró & Raquel Vegas, 2020. "Screening and Loan Origination Time: Lending Standards, Loan Defaults and Bank Failures," Working Papers 1215, Barcelona School of Economics.
    6. Roxana Elena Manea, 2021. "School Feeding Programmes, Education and Food Security in Rural Malawi," CIES Research Paper series 63-2020, Centre for International Environmental Studies, The Graduate Institute.
    7. Awaworyi Churchill, Sefa & Smyth, Russell & Trinh, Trong-Anh, 2022. "Energy poverty, temperature and climate change," Energy Economics, Elsevier, vol. 114(C).
    8. Upasak Das & Rupayan Pal & Udayan Rathore & Bibhas Saha, 2023. "Rein in pandemic by pricing vaccine: Does social trust matter?," Indira Gandhi Institute of Development Research, Mumbai Working Papers 2023-008, Indira Gandhi Institute of Development Research, Mumbai, India.
    9. Stefan Bauernschuster & Michael Grimm & Cathy M. Hajo, 2023. "The Impact of Margaret Sanger’s Birth Control Clinics on Early 20th Century U.S. Fertility and Mortality," CESifo Working Paper Series 10421, CESifo.
    10. Maja Adena & Julian Harke, 2022. "COVID-19 and pro-sociality: How do donors respond to local pandemic severity, increased salience, and media coverage?," Experimental Economics, Springer;Economic Science Association, vol. 25(3), pages 824-844, June.
    11. Bossavie, Laurent & Cho, Yoonyoung & Heath, Rachel, 2023. "The effects of international scrutiny on manufacturing workers: Evidence from the Rana Plaza collapse in Bangladesh," Journal of Development Economics, Elsevier, vol. 163(C).
    12. Stephen G. Dimmock & Jiekun Huang & Scott J. Weisbenner, 2022. "Give Me Your Tired, Your Poor, Your High-Skilled Labor: H-1B Lottery Outcomes and Entrepreneurial Success," Management Science, INFORMS, vol. 68(9), pages 6950-6970, September.
    13. Cemal Eren Arbath & Quamral H. Ashraf & Oded Galor & Marc Klemp, 2018. "Diversity and Conflict," Working Papers 2018-6, Brown University, Department of Economics.
    14. Takahashi, Ryo, 2021. "How to stimulate environmentally friendly consumption: Evidence from a nationwide social experiment in Japan to promote eco-friendly coffee," Ecological Economics, Elsevier, vol. 186(C).
    15. Vittorio Bassi & Raffaela Muoio & Tommaso Porzio & Ritwika Sen & Esau Tugume, 2022. "Achieving Scale Collectively," Econometrica, Econometric Society, vol. 90(6), pages 2937-2978, November.
    16. Gibbons, Stephen & Overman, Henry & Sarvimäki, Matti, 2021. "The local economic impacts of regeneration projects: Evidence from UK's single regeneration budget," Journal of Urban Economics, Elsevier, vol. 122(C).
    17. Tarantino, Emanuele & Pavanini, Nicola & Mayordomo, Sergio, 2020. "The Impact of Alternative Forms of Bank Consolidation on Credit Supply and Financial Stability," CEPR Discussion Papers 15069, C.E.P.R. Discussion Papers.
    18. Shusen Qi & Steven Ongena & Hua Cheng, 2022. "Working with women, do men get all the credit?," Small Business Economics, Springer, vol. 59(4), pages 1427-1447, December.
    19. Burlinson, Andrew & Giulietti, Monica & Law, Cherry & Liu, Hui-Hsuan, 2021. "Fuel poverty and financial distress," Energy Economics, Elsevier, vol. 102(C).
    20. Sebastian Doerr & Thomas Drechsel & Donggyu Lee, 2021. "Income inequality, financial intermediation, and small firms," BIS Working Papers 944, Bank for International Settlements.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:deveco:v:162:y:2023:i:c:s030438782300024x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/devec .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.