IDEAS home Printed from https://ideas.repec.org/a/zbw/espost/146094.html
   My bibliography  Save this article

Are Incentive Effects on Response Rates and Nonresponse Bias in Large-scale, Face-to-face Surveys Generalizable to Germany? Evidence from Ten Experiments

Author

Listed:
  • Pforr, Klaus
  • Blohm, Michael
  • Blom, Annelies G.
  • Erdel, Barbara
  • Felderer, Barbara
  • Fräßdorf, Mathis
  • Hajek, Kristin
  • Helmschrott, Susanne
  • Kleinert, Corinna
  • Koch, Achim
  • Krieger, Ulrich
  • Kroh, Martin
  • Martin, Silke
  • Saßenroth, Denise
  • Schmiedeberg, Claudia
  • Trüdinger, Eva-Maria
  • Rammstedt, Beatrice

Abstract

In survey research, a consensus has grown regarding the effectiveness of incentives encouraging survey participation across different survey modes and target populations. Most of this research has been based on surveys from the United States, whereas few studies have provided evidence that these results can be generalized to other contexts. This paper is the first to present comprehensive information concerning the effects of incentives on response rates and nonresponse bias across large-scale surveys in Germany. The context could be viewed as a critical test for incentive effects because Germany’s population is among the most survey-critical in the world, with very low response rates. Our results suggest positive incentive effects on response rates and patterns of effects that are similar to those in previous research: The effect increased with the monetary value of the incentive; cash incentives affected response propensity more strongly than lottery tickets do; and prepaid incentives could be more cost effective than conditional incentives. We found mixed results for the effects of incentives on nonresponse bias. Regarding large-scale panel surveys, we could not unequivocally confirm that incentives increased response rates in later panel waves.

Suggested Citation

  • Pforr, Klaus & Blohm, Michael & Blom, Annelies G. & Erdel, Barbara & Felderer, Barbara & Fräßdorf, Mathis & Hajek, Kristin & Helmschrott, Susanne & Kleinert, Corinna & Koch, Achim & Krieger, Ulrich & , 2015. "Are Incentive Effects on Response Rates and Nonresponse Bias in Large-scale, Face-to-face Surveys Generalizable to Germany? Evidence from Ten Experiments," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 79(3), pages 740-768.
    • Handle: RePEc:zbw:espost:146094
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/146094/1/Pforr_2015_Are-Incentive-Effects.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David Richter & Jürgen Schupp, 2012. "SOEP Innovation Sample (SOEP-IS): Description, Structure and Documentation," SOEPpapers on Multidisciplinary Panel Data Research 463, DIW Berlin, The German Socio-Economic Panel (SOEP).
    2. Robert F. Schoeni & Frank Stafford & Katherine A. Mcgonagle & Patricia Andreski, 2013. "Response Rates in National Panel Surveys," The ANNALS of the American Academy of Political and Social Science, , vol. 645(1), pages 60-87, January.
    3. Tomas Philipson, 1997. "Data Markets and the Production of Surveys," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(1), pages 47-72.
    4. Lynn, Peter & Laurie, Heather, 2008. "The use of respondent incentives on longitudinal surveys," ISER Working Paper Series 2008-42, Institute for Social and Economic Research.
    5. Mark Trappman & Stefanie Gundert & Claudia Wenzig & Daniel Gebhardt, 2010. "PASS – A Household Panel Survey for Research on Unemployment and Poverty," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 130(4), pages 609-622.
    6. Mathis Schröder & Denise Saßenroth & John Körtner & Martin Kroh & Jürgen Schupp, 2013. "Experimental Evidence of the Effect of Monetary Incentives on Cross-Sectional and Longitudinal Response: Experiences from the Socio-Economic Panel (SOEP)," SOEPpapers on Multidisciplinary Panel Data Research 603, DIW Berlin, The German Socio-Economic Panel (SOEP).
    7. Nicoletti, Cheti & Buck, Nick, 2004. "Explaining interviewee contact and co-operation in the British and German Household Panels," ISER Working Paper Series 2004-06, Institute for Social and Economic Research.
    8. Gert G. Wagner & Joachim R. Frick & Jürgen Schupp, 2007. "The German Socio-Economic Panel Study (SOEP) – Scope, Evolution and Enhancements," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 127(1), pages 139-169.
    9. Laurie, Heather, 2007. "The effect of increasing financial incentives in a panel survey: an experiment on the British Household Panel Survey, Wave 14," ISER Working Paper Series 2007-05, Institute for Social and Economic Research.
    10. Lynn, Peter & Jäckle, Annette & G. Blom, Annelies, 2008. "Understanding cross-national differences in unit non-response: the role of contact data," ISER Working Paper Series 2008-01, Institute for Social and Economic Research.
    11. Gelman, Andrew & Stevens, Matt & Chan, Valerie, 2003. "Regression Modeling and Meta-analysis for Decision Making: A Cost-Benefit Analysis of Incentives in Telephone Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(2), pages 213-225, April.
    12. Corinna Kleinert & Britta Matthes & Manfred Antoni & Katrin Drasch & Michael Ruland & Annette Trahms, 2011. "ALWA – New Life Course Data for Germany," Schmollers Jahrbuch : Journal of Applied Social Science Studies / Zeitschrift für Wirtschafts- und Sozialwissenschaften, Duncker & Humblot, Berlin, vol. 131(4), pages 625-634.
    13. Jürgen Schupp, 2012. "Die verborgenen Kosten monetärer Anreize - lohnt sich Motivierung durch Incentivierung?: Kommentar," DIW Wochenbericht, DIW Berlin, German Institute for Economic Research, vol. 79(6), pages 20-20.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Martin Huber & Anna Solovyeva, 2020. "Direct and Indirect Effects under Sample Selection and Outcome Attrition," Econometrics, MDPI, vol. 8(4), pages 1-25, December.
    2. Hans Fricke & Markus Frölich & Martin Huber & Michael Lechner, 2020. "Endogeneity and non‐response bias in treatment evaluation – nonparametric identification of causal effects by instruments," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 481-504, August.
    3. Sprietsma, Maresa, 2016. "Which incentives to increase survey response of secondary school pupils?," ZEW Discussion Papers 16-071, ZEW - Leibniz Centre for European Economic Research.
    4. Altmann Kristina & Bernard René & Le Blanc Julia & Gabor-Toth Enikö & Hebbat Malik & Kothmayr Lisa & Schmidt Tobias & Tzamourani Panagiota & Werner Daniel & Zhu Junyi, 2020. "The Panel on Household Finances (PHF) – Microdata on household wealth in Germany," German Economic Review, De Gruyter, vol. 21(3), pages 373-400, September.
    5. Jan Eckhard, 2018. "Indicators of Social Isolation: A Comparison Based on Survey Data from Germany," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 139(3), pages 963-988, October.
    6. Guy Stecklov & Alexander Weinreb & Calogero Carletto, 2018. "Can incentives improve survey data quality in developing countries?: results from a field experiment in India," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1033-1056, October.
    7. Tobias Gummer, 2019. "Assessing Trends and Decomposing Change in Nonresponse Bias: The Case of Bias in Cohort Distributions," Sociological Methods & Research, , vol. 48(1), pages 92-115, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:iab:iabfda:201307(en is not listed on IDEAS
    2. Nicole Watson & Mark Wooden, 2011. "Re-engaging with Survey Non-respondents: The BHPS, SOEP and HILDA Survey Experience," Melbourne Institute Working Paper Series wp2011n02, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    3. Jürgen Schupp, 2014. "40 Jahre Sozialberichterstattung und Lebensqualitätsforschung in Deutschland: Rückblick und Perspektiven," SOEPpapers on Multidisciplinary Panel Data Research 680, DIW Berlin, The German Socio-Economic Panel (SOEP).
    4. Ina Schöllgen & Denis Gerstorf & Jutta Heckhausen, 2014. "Control Strivings in the Socio-Economic Panel (SOEP)," SOEPpapers on Multidisciplinary Panel Data Research 727, DIW Berlin, The German Socio-Economic Panel (SOEP).
    5. Eckhardt Bode & Stephan Brunow & Ingrid Ott & Alina Sorgner, 2019. "Worker Personality: Another Skill Bias beyond Education in the Digital Age," German Economic Review, Verein für Socialpolitik, vol. 20(4), pages 254-294, November.
    6. Ivana Anusic & Richard E. Lucas & M. Brent Donnellan, 2017. "The Validity of the Day Reconstruction Method in the German Socio-economic Panel Study," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 130(1), pages 213-232, January.
    7. Marius Leckelt & Mitja D. Back & Joshua D. Foster & Ross Hutteman & Garrett Jaeger & Jessica McCain & Jean M. Twenge & W. Keith Campbell, 2016. "Entering Adulthood in a Recession Tempers Later Narcissism: But Only in Men," SOEPpapers on Multidisciplinary Panel Data Research 886, DIW Berlin, The German Socio-Economic Panel (SOEP).
    8. André Hajek & Hans-Helmut König, 2019. "Not Getting What You Want? The Impact of Income Comparisons on Subjective Well-Being—Findings of a Population-Based Longitudinal Study in Germany," IJERPH, MDPI, vol. 16(15), pages 1-8, July.
    9. Frick, Joachim R. & Grabka, Markus M. & Groh-Samberg, Olaf, 2012. "Dealing With Incomplete Household Panel Data in Inequality Research," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 41(1), pages 89-123.
    10. Giesselmann, Marco & Windzio, Michael, 2014. "Paneldaten in der Soziologie: Fixed Effects Paradigma und empirische Praxis in Panelregression und Ereignisanalyse," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 66(1), pages 95-113.
    11. Tamara M. Pfeiler & Boris Egloff, 2017. "Examining the “Veggie” Personality: Results from a Representative German Sample," SOEPpapers on Multidisciplinary Panel Data Research 941, DIW Berlin, The German Socio-Economic Panel (SOEP).
    12. Annamaria Bianchi & Silvia Biffignandi, 2019. "Social Indicators to Explain Response in Longitudinal Studies," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(3), pages 931-957, February.
    13. Kleinert, Corinna & Ruland, Michael & Trahms, Annette, 2013. "Bias in einem komplexen Surveydesign : Ausfallprozesse und Selektivität in der IAB-Befragung ALWA," FDZ Methodenreport 201302_de, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    14. Sascha O. Becker & Dolores Messer & Stefan C. Wolter, 2013. "A Gift is Not Always a Gift: Heterogeneity and Long-term Effects in a Gift Exchange Experiment," Economica, London School of Economics and Political Science, vol. 80(318), pages 345-371, April.
    15. G. Blom, Annelies, 2009. "Explaining cross-country differences in contact rates," ISER Working Paper Series 2009-08, Institute for Social and Economic Research.
    16. McGonagle Katherine A. & Schoeni Robert F. & Couper Mick P., 2013. "The Effects of a Between-Wave Incentive Experiment on Contact Update and Production Outcomes in a Panel Study," Journal of Official Statistics, Sciendo, vol. 29(2), pages 261-276, September.
    17. repec:iab:iabfda:201104(en is not listed on IDEAS
    18. André Hajek & Hans-Helmut König, 2020. "Prevalence and Correlates of Individuals Screening Positive for Depression and Anxiety on the PHQ-4 in the German General Population: Findings from the Nationally Representative German Socio-Economic ," IJERPH, MDPI, vol. 17(21), pages 1-11, October.
    19. Felderer, Barbara & Müller, Gerrit & Kreuter, Frauke & Winter, Joachim, 2018. "The Effect of Differential Incentives on Attrition Bias: Evidence from the PASS Wave 3 Incentive Experiment," Munich Reprints in Economics 62837, University of Munich, Department of Economics.
    20. repec:iab:iabfme:201302(de is not listed on IDEAS
    21. Sadig, Husam, 2014. "Unknown eligibility whilst weighting for non-response: the puzzle of who has died and who is still alive?," ISER Working Paper Series 2014-35, Institute for Social and Economic Research.
    22. Sascha Becker & Dolores Messer & Stefan C. Wolter & Sascha O. Becker, 2011. "A Gift is not Always a Gift: Gift Exchange in a Voucher Experiment," CESifo Working Paper Series 3488, CESifo.
    23. Heineck, Guido & Süssmuth, Bernd, 2013. "A different look at Lenin’s legacy: Social capital and risk taking in the Two Germanies," Journal of Comparative Economics, Elsevier, vol. 41(3), pages 789-803.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:espost:146094. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.