IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/10359.html

How Well Do Internet-Based Surveys Track Labor Market Indicators in Middle-IncomeCountries ?

Author

Listed:
  • Soundararajan,Vidhya
  • Soubeiga,Sidiki
  • Newhouse,David Locke
  • Palacios-Lopez,Amparo
  • Pape,Utz Johann
  • Weber,Michael

Abstract

Online surveys are convenient, cost-effective, speedy, and increasingly popular instrumentsfor data collection. This study investigates whether online surveys that used Random Domain Intercept Technology torecruit respondents were accurately measured labor market outcomes in six middle-income countries in the aftermath ofthe COVID-19 pandemic. Compared with the national average, online surveys oversampled males, youth, those with higherlevels of education, and those in smaller households. Reweighting using propensity score estimates fails toequalize the means of variables excluded from the model. When comparing the employment-to-population ratio from theinternet surveys to the most recent relevant nationally representative surveys, the average deviation is 30 percent.Reweighting using propensity scores in that case worsened the bias. Internet survey estimates of informal andself-employment rates also tend to be inconsistent with benchmark data, although the latter are available for fewercountries. Overall, the results suggest that despite the advantages and convenience of recruiting internet surveyparticipants through Random Domain Intercept Technology, the resulting sample is not representative and even afterpropensity score reweighting, it can yield estimates that are at odds with nationally representative surveys.

Suggested Citation

  • Soundararajan,Vidhya & Soubeiga,Sidiki & Newhouse,David Locke & Palacios-Lopez,Amparo & Pape,Utz Johann & Weber,Michael, 2023. "How Well Do Internet-Based Surveys Track Labor Market Indicators in Middle-IncomeCountries ?," Policy Research Working Paper Series 10359, The World Bank.
    • Handle: RePEc:wbk:wbrwps:10359
    as

    Download full text from publisher

    File URL: http://documents.worldbank.org/curated/en/099254503132376827/pdf/IDU06596cf9a09dbd0473a09c6e0c4dc3e9383bd.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Arie Kapteyn & James P. Smith & Arthur van Soest, 2007. "Vignettes and Self-Reports of Work Disability in the United States and the Netherlands," American Economic Review, American Economic Association, vol. 97(1), pages 461-473, March.
    2. Magdalena Smyk & Joanna Tyrowicz & Lucas van der Velde, 2021. "A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator," Sociological Methods & Research, , vol. 50(1), pages 429-464, February.
    3. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    4. Alexis Diamond & Jasjeet S. Sekhon, 2013. "Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 932-945, July.
    5. Sanam Roder-DeWan & Anna D Gage & Lisa R Hirschhorn & Nana A Y Twum-Danso & Jerker Liljestrand & Kwanele Asante-Shongwe & Viviana Rodríguez & Talhiya Yahya & Margaret E Kruk, 2019. "Expectations of healthcare quality: A cross-sectional study of internet users in 12 low- and middle-income countries," PLOS Medicine, Public Library of Science, vol. 16(8), pages 1-17, August.
    6. Matthias Schonlau & Arthur van Soest & Arie Kapteyn & Mick Couper, 2009. "Selection Bias in Web Surveys and the Use of Propensity Scores," Sociological Methods & Research, , vol. 37(3), pages 291-318, February.
    7. Braunsberger, Karin & Wybenga, Hans & Gates, Roger, 2007. "A comparison of reliability between telephone and web-based surveys," Journal of Business Research, Elsevier, vol. 60(7), pages 758-764, July.
    8. Fan Li & Kari Lock Morgan & Alan M. Zaslavsky, 2018. "Balancing Covariates via Propensity Score Weighting," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 390-400, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Magdalena Smyk & Joanna Tyrowicz & Lucas van der Velde, 2021. "A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator," Sociological Methods & Research, , vol. 50(1), pages 429-464, February.
    2. Yimin Dai & Ying Yan, 2024. "Mahalanobis balancing: A multivariate perspective on approximate covariate balancing," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 51(4), pages 1450-1471, December.
    3. Cousineau, Martin & Verter, Vedat & Murphy, Susan A. & Pineau, Joelle, 2023. "Estimating causal effects with optimization-based methods: A review and empirical comparison," European Journal of Operational Research, Elsevier, vol. 304(2), pages 367-380.
    4. Martin Cousineau & Vedat Verter & Susan A. Murphy & Joelle Pineau, 2022. "Estimating causal effects with optimization-based methods: A review and empirical comparison," Papers 2203.00097, arXiv.org.
    5. Guido Imbens & Yiqing Xu, 2024. "Comparing Experimental and Nonexperimental Methods: What Lessons Have We Learned Four Decades After LaLonde (1986)?," Papers 2406.00827, arXiv.org, revised May 2025.
    6. Schwartz, Joseph A. & Allen, Samantha L., 2024. "The accumulated impact of direct and indirect workplace violence exposure on mental health and physiological activity among correctional officers," Journal of Criminal Justice, Elsevier, vol. 93(C).
    7. Yuheng Lin & Dooruj Rambaccussing & Yu Zhu, 2024. "The impact of international students in the UK on the cultural goods trade," French Stata Users' Group Meetings 2024 29, Stata Users Group.
    8. Friedrich, Sarah & Pauly, Markus, 2018. "MATS: Inference for potentially singular and heteroscedastic MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 166-179.
    9. Hairu Wang & Yukun Liu & Haiying Zhou, 2025. "Score test for unconfoundedness under a logistic treatment assignment model," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 77(4), pages 517-533, August.
    10. Jung, Haeil & Kim, Jun Hyung & Hong, Gihyeon, 2023. "Impacts of the COVID-19 crisis on single-person households in South Korea," Journal of Asian Economics, Elsevier, vol. 84(C).
    11. Hossain, Marup & Songsermsawas, Tisorn, 2025. "Adapting to Thrive: Training and Access to Finance to Reduce Climate Vulnerability Among Smallholder Farmers in Nepal," 2025 AAEA & WAEA Joint Annual Meeting, July 27-29, 2025, Denver, CO 361170, Agricultural and Applied Economics Association.
    12. Cain Polidano & Justin van de Ven & Sarah Voitchovsky, 2017. "The Power of Self-Interest: Effects of Education and Training Entitlements in Later-Life," Melbourne Institute Working Paper Series wp2017n12, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    13. Calomiris, Charles W. & Larrain, Mauricio & Schmukler, Sergio L., 2021. "Capital inflows, equity issuance activity, and corporate investment," Journal of Financial Intermediation, Elsevier, vol. 46(C).
    14. Jorge Cuartas, 2017. "Neighborhood Crime Undermines Parenting: Violence in the Vicinity of Households as a Predictor of Aggressive Discipline," Documentos de trabajo 17646, Escuela de Gobierno - Universidad de los Andes.
    15. Bratti, Massimiliano & Mendola, Mariapia, 2014. "Parental health and child schooling," Journal of Health Economics, Elsevier, vol. 35(C), pages 94-108.
    16. José Valentim Machado Vicente & Jaqueline Terra Moura Marins, 2019. "A Volatility Smile-Based Uncertainty Index," Working Papers Series 502, Central Bank of Brazil, Research Department.
    17. Hagemann, Andreas, 2019. "Placebo inference on treatment effects when the number of clusters is small," Journal of Econometrics, Elsevier, vol. 213(1), pages 190-209.
    18. William H. Greene & Mark N. Harris & Rachel J. Knott & Nigel Rice, 2021. "Specification and testing of hierarchical ordered response models with anchoring vignettes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 31-64, January.
    19. Mario J. Crucini & Gregor W. Smith, 2016. "Distance and Time Effects in Swedish Commodity Prices, 1732–1914," NBER Working Papers 22175, National Bureau of Economic Research, Inc.
    20. Federico Belotti & Edoardo Di Porto & Gianluca Santoni, 2021. "The effect of local taxes on firm performance: Evidence from geo‐referenced data," Journal of Regional Science, Wiley Blackwell, vol. 61(2), pages 492-510, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:10359. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.