IDEAS home Printed from https://ideas.repec.org/p/ets/wpaper/3.html
   My bibliography  Save this paper

Sensitive Questions in Online Surveys: Experimental Results for the Randomized Response Technique (RRT) and the Unmatched Count Technique (UCT)

Author

Listed:
  • Elisabeth Coutts
  • Ben Jann

Abstract

Gaining valid answers to so-called sensitive questions is an age-old problem in survey research. Various techniques have been developed to guarantee anonymity and minimize the respondent's feelings of jeopardy. Two such techniques are the randomized response technique (RRT) and the unmatched count technique (UCT). In this study we evaluate the effectiveness of different implementations of the RRT (using a forced-response design) in a computer-assisted setting and also compare the use of the RRT to that of the UCT. The techniques are evaluated according to various quality criteria, such as the prevalence estimates they provide, the ease of their use, and respondent trust in the techniques. Our results indicate that the RRTs are problematic with respect to several domains, such as the limited trust they inspire and non-response, and that the RRT estimates are unreliable due to a strong false "no" bias, especially for the more sensitive questions. The UCT, however, performed well compared to the RRTs on all the evaluated measures. The UCT estimates also had more face validity than the RRT estimates. We conclude that the UCT is a promising alternative to RRT in self-administered surveys and that future research should be directed towards evaluating and improving the technique.

Suggested Citation

  • Elisabeth Coutts & Ben Jann, 2008. "Sensitive Questions in Online Surveys: Experimental Results for the Randomized Response Technique (RRT) and the Unmatched Count Technique (UCT)," ETH Zurich Sociology Working Papers 3, ETH Zurich, Chair of Sociology.
  • Handle: RePEc:ets:wpaper:3
    as

    Download full text from publisher

    File URL: http://repec.ethz.ch/ets/papers/coutts_jann_online_rrt.pdf
    File Function: First version, 2008
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Andrei Postoaca, 2006. "The Anonymous Elect," Springer Books, Springer, number 978-3-540-29030-8, February.
    2. Gerty J. L. M. Lensvelt‐Mulders & Peter G. M. Van Der Heijden & Olav Laudy & Ger Van Gils, 2006. "A validation of a computer‐assisted randomized response survey to estimate the prevalence of fraud in social security," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(2), pages 305-318, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peter Preisendörfer, 2008. "Heikle Fragen in mündlichen Interviews: Ergebnisse einer Methodenstudie im studentischen Milieu (Sensitive Questions in Face-to-Face Interviews: Findings of a Methodological Study with University Stud," ETH Zurich Sociology Working Papers 6, ETH Zurich, Chair of Sociology.
    2. Thorben C. Kundt & Florian Misch & Birger Nerré, 2017. "Re-assessing the merits of measuring tax evasion through business surveys: an application of the crosswise model," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 24(1), pages 112-133, February.
    3. Jouni Kuha & Jonathan Jackson, 2014. "The item count method for sensitive survey questions: modelling criminal behaviour," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 63(2), pages 321-341, February.
    4. Friesenbichler, Klaus S. & Selenko, Eva & Clarke, George R.G., 2015. "How much of a nuisance is greasing the palms? A study on job dedication and attitudes towards corruption reports under answer bias control," MPRA Paper 67331, University Library of Munich, Germany.
    5. Andreas Quatember, 2019. "A discussion of the two different aspects of privacy protection in indirect questioning designs," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(1), pages 269-282, January.
    6. Roe-Sepowitz, Dominique & Bontrager, Stephanie & Pickett, Justin T. & Kosloski, Anna E., 2019. "Estimating the sex buying behavior of adult males in the United States: List experiment and direct question estimates," Journal of Criminal Justice, Elsevier, vol. 63(C), pages 41-48.
    7. Korndörfer, Martin & Krumpal, Ivar & Schmukle, Stefan C., 2014. "Measuring and explaining tax evasion: Improving self-reports using the crosswise model," Journal of Economic Psychology, Elsevier, vol. 45(C), pages 18-32.
    8. Malesky, Edmund J. & Nguyen, Cuong Viet & Tran, Anh, 2014. "The Impact of Recentralization on Public Services: A Difference-in-Differences Analysis of the Abolition of Elected Councils in Vietnam," American Political Science Review, Cambridge University Press, vol. 108(1), pages 144-168, February.
    9. Katherine B. Coffman & Lucas C. Coffman & Keith M. Marzilli Ericson, 2017. "The Size of the LGBT Population and the Magnitude of Antigay Sentiment Are Substantially Underestimated," Management Science, INFORMS, vol. 63(10), pages 3168-3186, October.
    10. Gueorguiev, Dimitar & Malesky, Edmund, 2012. "Foreign investment and bribery: A firm-level analysis of corruption in Vietnam," Journal of Asian Economics, Elsevier, vol. 23(2), pages 111-129.
    11. Marc Höglinger & Ben Jann, 2018. "More is not always better: An experimental individual-level validation of the randomized response technique and the crosswise model," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-22, August.
    12. John, Leslie K. & Loewenstein, George & Acquisti, Alessandro & Vosgerau, Joachim, 2018. "When and why randomized response techniques (fail to) elicit the truth," Organizational Behavior and Human Decision Processes, Elsevier, vol. 148(C), pages 101-123.
    13. La Forgia, Gerard & Raha, Shomikho & Shaik, Shabbeer & Maheshwari, Sunil Kumar & Ali, Rabia, 2014. "Parallel systems and human resource management in India's public health services : a view from the front lines," Policy Research Working Paper Series 6953, The World Bank.
    14. Artem Shcherbina & Rostyslav Maiboroda, 2011. "Finite mixtures model approach to sensitive questions in surveys," Statistics in Transition new series, Główny Urząd Statystyczny (Polska), vol. 12(2), pages 331-344, October.
    15. Coutts Elisabethen & Jann Ben & Krumpal Ivar & Näher Anatol-Fiete, 2011. "Plagiarism in Student Papers: Prevalence Estimates Using Special Techniques for Sensitive Questions," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 231(5-6), pages 749-760, October.
    16. Katherine I. Tierney, 2019. "Abortion Underreporting in Add Health: Findings and Implications," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 38(3), pages 417-428, June.
    17. Ivar Krumpal, 2013. "Determinants of social desirability bias in sensitive surveys: a literature review," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(4), pages 2025-2047, June.
    18. Antonio Arcos & María del Rueda & Sarjinder Singh, 2015. "A generalized approach to randomised response for quantitative variables," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 1239-1256, May.
    19. Liu, Yin & Tian, Guo-Liang, 2013. "A variant of the parallel model for sample surveys with sensitive characteristics," Computational Statistics & Data Analysis, Elsevier, vol. 67(C), pages 115-135.
    20. Monika Frenger & Eike Emrich & Werner Pitsch, 2019. "Corruption in Olympic Sports: Prevalence Estimations of Match Fixing Among German Squad Athletes," SAGE Open, , vol. 9(3), pages 21582440198, July.
    21. Marc Höglinger & Ben Jann & Andreas Diekmann, 2014. "Online Survey on "Exams and Written Papers". Documentation," University of Bern Social Sciences Working Papers 8, University of Bern, Department of Social Sciences, revised 06 Oct 2014.
    22. Anatol-Fiete Näher & Ivar Krumpal, 2012. "Asking sensitive questions: the impact of forgiving wording and question context on social desirability bias," Quality & Quantity: International Journal of Methodology, Springer, vol. 46(5), pages 1601-1616, August.
    23. Klaus Friesenbichler & George Clarke & Michael Wong, 2014. "Price competition and market transparency: evidence from a random response technique," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 41(1), pages 5-21, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. van den Hout, Ardo & van der Heijden, Peter G.M. & Gilchrist, Robert, 2007. "The logistic regression model with response variables subject to randomized response," Computational Statistics & Data Analysis, Elsevier, vol. 51(12), pages 6060-6069, August.
    2. Maarten J. L. F. Cruyff & Ardo van den Hout & Peter G. M. van der Heijden & Ulf Böckenholt, 2007. "Log-Linear Randomized-Response Models Taking Self-Protective Response Behavior Into Account," Sociological Methods & Research, , vol. 36(2), pages 266-282, November.
    3. Heiko Groenitz, 2015. "Using prior information in privacy-protecting survey designs for categorical sensitive variables," Statistical Papers, Springer, vol. 56(1), pages 167-189, February.
    4. Mühlböck Monika & Steiber Nadia & Kittel Bernhard, 2017. "Less Supervision, More Satisficing? Comparing Completely Self-Administered Web-Surveys and Interviews Under Controlled Conditions," Statistics, Politics and Policy, De Gruyter, vol. 8(1), pages 13-28, October.
    5. Ulf Böckenholt & Peter van der Heijden, 2007. "Item Randomized-Response Models for Measuring Noncompliance: Risk-Return Perceptions, Social Influences, and Self-Protective Responses," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 245-262, June.
    6. María del Mar García Rueda & Pier Francesco Perri & Beatriz Rodríguez Cobo, 2018. "Advances in estimation by the item sum technique using auxiliary information in complex surveys," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 102(3), pages 455-478, July.
    7. Sabrina Giordano & Pier Perri, 2012. "Efficiency comparison of unrelated question models based on same privacy protection degree," Statistical Papers, Springer, vol. 53(4), pages 987-999, November.
    8. Andreas Quatember, 2019. "A discussion of the two different aspects of privacy protection in indirect questioning designs," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(1), pages 269-282, January.
    9. Leif Appelgren, 2019. "Optimal auditing of social benefit fraud: a case study," Empirical Economics, Springer, vol. 56(1), pages 203-231, January.
    10. Carel F. W. Peeters & Gerty J. L. M. Lensvelt-Mulders & Karin Lasthuizen, 2010. "A Note on a Simple and Practical Randomized Response Framework for Eliciting Sensitive Dichotomous and Quantitative Information," Sociological Methods & Research, , vol. 39(2), pages 283-296, November.
    11. Pier Francesco Perri & Elvira Pelle & Manuela Stranges, 2016. "Estimating Induced Abortion and Foreign Irregular Presence Using the Randomized Response Crossed Model," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 129(2), pages 601-618, November.
    12. Groenitz, Heiko, 2016. "A covariate nonrandomized response model for multicategorical sensitive variables," Computational Statistics & Data Analysis, Elsevier, vol. 103(C), pages 124-138.

    More about this item

    Keywords

    sensitive questions; online survey; randomized response technique; unmatched count technique; item count technique; methodological experiment;
    All these keywords.

    JEL classification:

    • C42 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Survey Methods
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ets:wpaper:3. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.socio.ethz.ch/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Heidi Bruderer The email address of this maintainer does not seem to be valid anymore. Please ask Heidi Bruderer to update the entry or send us the correct address (email available below). General contact details of provider: http://www.socio.ethz.ch/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.