IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v181y2018i4p1033-1056.html
   My bibliography  Save this article

Can incentives improve survey data quality in developing countries?: results from a field experiment in India

Author

Listed:
  • Guy Stecklov
  • Alexander Weinreb
  • Calogero Carletto

Abstract

We report results of an experiment designed to assess whether the payment of contingent incentives to respondents in Karnataka, India, impacts the quality of survey data. Of 2276 households sampled at the city block level, 934 were randomly assigned to receive a small one‐time payment at the time of the survey, whereas the remaining households did not receive this incentive. We analyse the effects of incentives across a range of questions that are common in survey research in less developed countries. Our study suggests that incentives reduced unit non‐response. Conditionally on participation, we also find little impact of incentives on a broad range of sociodemographic, behavioural and attitudinal questions. In contrast, we consistently find that households that received incentives reported substantially lower consumption and income levels and fewer assets. Given random assignment and very high response rates, the most plausible interpretation of this finding is that incentivizing respondents in this setting may increase their motivation to present themselves as more needy, whether to justify the current payment or to increase the chance of receiving resources in the future. Therefore, despite early indications that contingent incentives may raise response rates, the net effect on data quality must be carefully considered.

Suggested Citation

  • Guy Stecklov & Alexander Weinreb & Calogero Carletto, 2018. "Can incentives improve survey data quality in developing countries?: results from a field experiment in India," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1033-1056, October.
  • Handle: RePEc:bla:jorssa:v:181:y:2018:i:4:p:1033-1056
    DOI: 10.1111/rssa.12333
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12333
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssa.12333?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gary S. Becker, 1974. "Crime and Punishment: An Economic Approach," NBER Chapters, in: Essays in the Economics of Crime and Punishment, pages 1-54, National Bureau of Economic Research, Inc.
    2. Frey, Bruno S & Oberholzer-Gee, Felix, 1997. "The Cost of Price Incentives: An Empirical Analysis of Motivation Crowding-Out," American Economic Review, American Economic Association, vol. 87(4), pages 746-755, September.
    3. Simona Bignami-Van Assche & Georges Reniers & Alexander A. Weinreb, 2003. "An Assessment of the KDICP and MDICP Data Quality," Demographic Research Special Collections, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 1(2), pages 31-76.
    4. Barbara Mensch & Paul Hewett & Annabel Erulkar, 2003. "The reporting of sensitive behavior by adolescents: A methodological experiment in Kenya," Demography, Springer;Population Association of America (PAA), vol. 40(2), pages 247-268, May.
    5. Annelies G. Blom, 2012. "Explaining cross‐country differences in survey contact rates: application of decomposition methods," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(1), pages 217-242, January.
    6. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    7. Lara Fleischer & Conal Smith & Carine Viac, 2016. "A Review of General Social Surveys," OECD Statistics Working Papers 2016/9, OECD Publishing.
    8. Kellie J. Archer & Stanley Lemeshow, 2006. "Goodness-of-fit test for a logistic regression model fitted using survey sample data," Stata Journal, StataCorp LP, vol. 6(1), pages 97-105, March.
    9. Rosenbaum, Stephen Mark & Billinger, Stephan & Stieglitz, Nils, 2014. "Let’s be honest: A review of experimental evidence of honesty and truth-telling," Journal of Economic Psychology, Elsevier, vol. 45(C), pages 181-196.
    10. Deaton, Angus, 1992. "Understanding Consumption," OUP Catalogue, Oxford University Press, number 9780198288244.
    11. Pforr, Klaus & Blohm, Michael & Blom, Annelies G. & Erdel, Barbara & Felderer, Barbara & Fräßdorf, Mathis & Hajek, Kristin & Helmschrott, Susanne & Kleinert, Corinna & Koch, Achim & Krieger, Ulrich & , 2015. "Are Incentive Effects on Response Rates and Nonresponse Bias in Large-scale, Face-to-face Surveys Generalizable to Germany? Evidence from Ten Experiments," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 79(3), pages 740-768.
    12. Simona Bignami-Van Assche, 2003. "Are we measuring what we want to measure?," Demographic Research Special Collections, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 1(3), pages 77-108.
    13. Angus Deaton & Valerie Kozel, 2005. "Data and Dogma: The Great Indian Poverty Debate," The World Bank Research Observer, World Bank, vol. 20(2), pages 177-199.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shahriar, Abu Zafar M. & Shepherd, Dean A., 2019. "Violence against women and new venture initiation with microcredit: Self-efficacy, fear of failure, and disaster experiences," Journal of Business Venturing, Elsevier, vol. 34(6).
    2. Stefano Cabras & J. James Reade & J.D. Tena, 2018. "Social Pressure or Rational Reactions to Incentives? A Historical Analysis of Reasons for Referee Bias in the Spanish Football," Working Papers 20189, University of Liverpool, Department of Economics.
    3. Joachim De Weerdt & John Gibson & Kathleen Beegle, 2020. "What Can We Learn from Experimenting with Survey Methods?," Annual Review of Resource Economics, Annual Reviews, vol. 12(1), pages 431-447, October.
    4. Abhijit Banerjee & Paul Niehaus & Tavneet Suri, 2019. "Universal Basic Income in the Developing World," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 959-983, August.
    5. Shahidul Islam, 2021. "Impact Of Lottery Incentive On Response Rate And Data Quality: Evidence From Organic Food Consumption Survey Of Conventional Shoppers," Cultural Communication and Socialization Journal (CCSJ), Zibeline International Publishing, vol. 2(2), pages 68-74, November.
    6. Jiandong Chen & Ming Gao & Shulei Cheng & Yiyin Xu & Malin Song & Yu Liu & Wenxuan Hou & Shuhong Wang, 2022. "Evaluation and drivers of global low-carbon economies based on satellite data," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-12, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michelle Poulin, 2010. "Reporting on first sexual experience," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 22(11), pages 237-288.
    2. Gary Bolton & Eugen Dimant & Ulrich Schmidt, 2018. "When a Nudge Backfires. Using Observation with Social and Economic Incentives to Promote Pro-Social Behavior," PPE Working Papers 0017, Philosophy, Politics and Economics, University of Pennsylvania.
    3. Heinicke, Franziska & Rosenkranz, Stephanie & Weitzel, Utz, 2019. "The effect of pledges on the distribution of lying behavior: An online experiment," Journal of Economic Psychology, Elsevier, vol. 73(C), pages 136-151.
    4. Yahagi, Ken, 2021. "Law enforcement with motivated agents," International Review of Law and Economics, Elsevier, vol. 66(C).
    5. Günther G. Schulze & Björn Frank, 2003. "Deterrence versus intrinsic motivation: Experimental evidence on the determinants of corruptibility," Economics of Governance, Springer, vol. 4(2), pages 143-160, August.
    6. Giovanni Mastrobuoni & Paolo Pinotti, 2011. "Legal status of immigrants and criminal behavior: evidence from a natural experiment," Temi di discussione (Economic working papers) 813, Bank of Italy, Economic Research and International Relations Area.
    7. Paolo Pinotti, 2012. "The economic costs of organized crime: evidence from southern Italy," Temi di discussione (Economic working papers) 868, Bank of Italy, Economic Research and International Relations Area.
    8. Alm, James & Bruner, David M. & McKee, Michael, 2016. "Honesty or dishonesty of taxpayer communications in an enforcement regime," Journal of Economic Psychology, Elsevier, vol. 56(C), pages 85-96.
    9. Friehe, Tim & Utikal, Verena, 2018. "Intentions under cover – Hiding intentions is considered unfair," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 73(C), pages 11-21.
    10. Ojah, Kalu & Muhanji, Stella & Kodongo, Odongo, 2020. "Insider trading laws and price informativeness in emerging stock markets: The South African case," Emerging Markets Review, Elsevier, vol. 43(C).
    11. Garbarino, Ellen & Slonim, Robert & Villeval, Marie Claire, 2019. "Loss aversion and lying behavior," Journal of Economic Behavior & Organization, Elsevier, vol. 158(C), pages 379-393.
    12. Aksoy, Billur & Palma, Marco A., 2019. "The effects of scarcity on cheating and in-group favoritism," Journal of Economic Behavior & Organization, Elsevier, vol. 165(C), pages 100-117.
    13. Gold, Natalie, 2020. "How should we reconcile self-regarding and pro-social motivations? A renaissance of “Das Adam Smith Problem”," LSE Research Online Documents on Economics 109218, London School of Economics and Political Science, LSE Library.
    14. Giovanni Mastrobuoni & Paolo Pinotti, 2015. "Legal Status and the Criminal Activity of Immigrants," American Economic Journal: Applied Economics, American Economic Association, vol. 7(2), pages 175-206, April.
    15. Tobol, Yossef & Siniver, Erez & Yaniv, Gideon, 2020. "Do tightwads cheat more? Evidence from three field experiments," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 148-158.
    16. Daniel Mejía & Pascual Restrepo & Sandra V. Rozo, 2017. "On the Effects of Enforcement on Illegal Markets: Evidence from a Quasi-Experiment in Colombia," The World Bank Economic Review, World Bank, vol. 31(2), pages 570-594.
    17. Seeun Jung & Kenneth Houngbedji, 2014. "Shirking, Monitoring, and Risk Aversion," PSE Working Papers halshs-00965532, HAL.
    18. Mastrobuoni, Giovanni & Pinotti, Paolo, 2011. "Migration Restrictions and Criminal Behavior: Evidence from a Natural Experiment," Economy and Society 115723, Fondazione Eni Enrico Mattei (FEEM).
    19. Bech, Mickael, 2005. "The economics of non-attendance and the expected effect of charging a fine on non-attendees," Health Policy, Elsevier, vol. 74(2), pages 181-191, October.
    20. Jakob Alfitian & Dirk Sliwka & Timo Vogelsang, 2021. "When Bonuses Backfire: Evidence from the Workplace," Natural Field Experiments 00725, The Field Experiments Website.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:181:y:2018:i:4:p:1033-1056. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.