IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/10359.html
   My bibliography  Save this paper

How Well Do Internet-Based Surveys Track Labor Market Indicators in Middle-IncomeCountries ?

Author

Listed:
  • Soundararajan,Vidhya
  • Soubeiga,Sidiki
  • Newhouse,David Locke
  • Palacios-Lopez,Amparo
  • Pape,Utz Johann
  • Weber,Michael

Abstract

Online surveys are convenient, cost-effective, speedy, and increasingly popular instrumentsfor data collection. This study investigates whether online surveys that used Random Domain Intercept Technology torecruit respondents were accurately measured labor market outcomes in six middle-income countries in the aftermath ofthe COVID-19 pandemic. Compared with the national average, online surveys oversampled males, youth, those with higherlevels of education, and those in smaller households. Reweighting using propensity score estimates fails toequalize the means of variables excluded from the model. When comparing the employment-to-population ratio from theinternet surveys to the most recent relevant nationally representative surveys, the average deviation is 30 percent.Reweighting using propensity scores in that case worsened the bias. Internet survey estimates of informal andself-employment rates also tend to be inconsistent with benchmark data, although the latter are available for fewercountries. Overall, the results suggest that despite the advantages and convenience of recruiting internet surveyparticipants through Random Domain Intercept Technology, the resulting sample is not representative and even afterpropensity score reweighting, it can yield estimates that are at odds with nationally representative surveys.

Suggested Citation

  • Soundararajan,Vidhya & Soubeiga,Sidiki & Newhouse,David Locke & Palacios-Lopez,Amparo & Pape,Utz Johann & Weber,Michael, 2023. "How Well Do Internet-Based Surveys Track Labor Market Indicators in Middle-IncomeCountries ?," Policy Research Working Paper Series 10359, The World Bank.
    • Handle: RePEc:wbk:wbrwps:10359
    as

    Download full text from publisher

    File URL: http://documents.worldbank.org/curated/en/099254503132376827/pdf/IDU06596cf9a09dbd0473a09c6e0c4dc3e9383bd.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Arie Kapteyn & James P. Smith & Arthur van Soest, 2007. "Vignettes and Self-Reports of Work Disability in the United States and the Netherlands," American Economic Review, American Economic Association, vol. 97(1), pages 461-473, March.
    2. A. Colin Cameron & Douglas L. Miller, 2015. "A Practitioner’s Guide to Cluster-Robust Inference," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 317-372.
    3. Sanam Roder-DeWan & Anna D Gage & Lisa R Hirschhorn & Nana A Y Twum-Danso & Jerker Liljestrand & Kwanele Asante-Shongwe & Viviana Rodríguez & Talhiya Yahya & Margaret E Kruk, 2019. "Expectations of healthcare quality: A cross-sectional study of internet users in 12 low- and middle-income countries," PLOS Medicine, Public Library of Science, vol. 16(8), pages 1-17, August.
    4. Matthias Schonlau & Arthur van Soest & Arie Kapteyn & Mick Couper, 2009. "Selection Bias in Web Surveys and the Use of Propensity Scores," Sociological Methods & Research, , vol. 37(3), pages 291-318, February.
    5. Braunsberger, Karin & Wybenga, Hans & Gates, Roger, 2007. "A comparison of reliability between telephone and web-based surveys," Journal of Business Research, Elsevier, vol. 60(7), pages 758-764, July.
    6. Magdalena Smyk & Joanna Tyrowicz & Lucas van der Velde, 2021. "A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator," Sociological Methods & Research, , vol. 50(1), pages 429-464, February.
    7. Alexis Diamond & Jasjeet S. Sekhon, 2013. "Genetic Matching for Estimating Causal Effects: A General Multivariate Matching Method for Achieving Balance in Observational Studies," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 932-945, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Magdalena Smyk & Joanna Tyrowicz & Lucas van der Velde, 2021. "A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator," Sociological Methods & Research, , vol. 50(1), pages 429-464, February.
    2. Schwartz, Joseph A. & Allen, Samantha L., 2024. "The accumulated impact of direct and indirect workplace violence exposure on mental health and physiological activity among correctional officers," Journal of Criminal Justice, Elsevier, vol. 93(C).
    3. Clément de Chaisemartin & Jaime Ramirez-Cuellar, 2024. "At What Level Should One Cluster Standard Errors in Paired and Small-Strata Experiments?," American Economic Journal: Applied Economics, American Economic Association, vol. 16(1), pages 193-212, January.
    4. Yuheng Lin & Dooruj Rambaccussing & Yu Zhu, 2024. "The impact of international students in the UK on the cultural goods trade," French Stata Users' Group Meetings 2024 29, Stata Users Group.
    5. Friedrich, Sarah & Pauly, Markus, 2018. "MATS: Inference for potentially singular and heteroscedastic MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 166-179.
    6. Borisova, Ekaterina & Gründler, Klaus & Hackenberger, Armin & Harter, Anina & Potrafke, Niklas & Schoors, Koen, 2023. "Crisis experience and the deep roots of COVID-19 vaccination preferences," European Economic Review, Elsevier, vol. 160(C).
    7. Choi, Eleanor Jawon & Choi, Jaewoo & Son, Hyelim, 2020. "The long-term effects of labor market entry in a recession: Evidence from the Asian financial crisis," Labour Economics, Elsevier, vol. 67(C).
    8. Berthélemy Michel & Bonev Petyo & Dussaux Damien & Söderberg Magnus, 2019. "Methods for strengthening a weak instrument in the case of a persistent treatment," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 23(1), pages 1-30, February.
    9. Carrieri, Vincenzo & Madio, Leonardo & Principe, Francesco, 2019. "Light cannabis and organized crime: Evidence from (unintended) liberalization in Italy," European Economic Review, Elsevier, vol. 113(C), pages 63-76.
    10. Jung, Haeil & Kim, Jun Hyung & Hong, Gihyeon, 2023. "Impacts of the COVID-19 crisis on single-person households in South Korea," Journal of Asian Economics, Elsevier, vol. 84(C).
    11. Cain Polidano & Justin van de Ven & Sarah Voitchovsky, 2017. "The Power of Self-Interest: Effects of Education and Training Entitlements in Later-Life," Melbourne Institute Working Paper Series wp2017n12, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    12. Calomiris, Charles W. & Larrain, Mauricio & Schmukler, Sergio L., 2021. "Capital inflows, equity issuance activity, and corporate investment," Journal of Financial Intermediation, Elsevier, vol. 46(C).
    13. Jorge Cuartas, 2017. "Neighborhood Crime Undermines Parenting: Violence in the Vicinity of Households as a Predictor of Aggressive Discipline," Documentos de trabajo 17646, Escuela de Gobierno - Universidad de los Andes.
    14. Bratti, Massimiliano & Mendola, Mariapia, 2014. "Parental health and child schooling," Journal of Health Economics, Elsevier, vol. 35(C), pages 94-108.
    15. José Valentim Machado Vicente & Jaqueline Terra Moura Marins, 2019. "A Volatility Smile-Based Uncertainty Index," Working Papers Series 502, Central Bank of Brazil, Research Department.
    16. Hagemann, Andreas, 2019. "Placebo inference on treatment effects when the number of clusters is small," Journal of Econometrics, Elsevier, vol. 213(1), pages 190-209.
    17. William H. Greene & Mark N. Harris & Rachel J. Knott & Nigel Rice, 2021. "Specification and testing of hierarchical ordered response models with anchoring vignettes," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 31-64, January.
    18. Ciccarelli, Carlo & De Fraja, Gianni & Vuri, Daniela, 2021. "Effects of passive smoking on prenatal and infant development: Lessons from the past," Economics & Human Biology, Elsevier, vol. 42(C).
    19. Mario J. Crucini & Gregor W. Smith, 2016. "Distance and Time Effects in Swedish Commodity Prices, 1732–1914," NBER Working Papers 22175, National Bureau of Economic Research, Inc.
    20. Federico Belotti & Edoardo Di Porto & Gianluca Santoni, 2021. "The effect of local taxes on firm performance: Evidence from geo‐referenced data," Journal of Regional Science, Wiley Blackwell, vol. 61(2), pages 492-510, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:10359. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.