IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0235403.html
   My bibliography  Save this article

Can detailed instructions and comprehension checks increase the validity of crosswise model estimates?

Author

Listed:
  • Julia Meisters
  • Adrian Hoffmann
  • Jochen Musch

Abstract

The crosswise model is an indirect questioning technique designed to control for socially desirable responding. Although the technique has delivered promising results in terms of improved validity in survey studies of sensitive issues, recent studies have indicated that the crosswise model may sometimes produce false positives. Hence, we investigated whether an insufficient understanding of the crosswise model instructions might be responsible for these false positives and whether ensuring a deeper understanding of the model and surveying more highly educated respondents reduces the problem of false positives. To this end, we experimentally manipulated the amount of information respondents received in the crosswise model instructions. We compared a crosswise model condition with only brief instructions and a crosswise model condition with detailed instructions and additional comprehension checks. Additionally, we compared the validity of crosswise model estimates between a higher- and a lower-educated subgroup of respondents. Our results indicate that false positives among highly educated respondents can be reduced when detailed instructions and comprehension checks are employed. Since false positives can also occur in direct questioning, they do not appear to be a specific flaw of the crosswise model, but rather a more general problem of self-reports on sensitive topics. False negatives were found to occur for all questioning techniques, but were less prevalent in the crosswise model than in the direct questioning condition. We highlight the importance of comprehension checks when applying indirect questioning and emphasize the necessity of developing instructions suitable for lower-educated respondents.

Suggested Citation

  • Julia Meisters & Adrian Hoffmann & Jochen Musch, 2020. "Can detailed instructions and comprehension checks increase the validity of crosswise model estimates?," PLOS ONE, Public Library of Science, vol. 15(6), pages 1-19, June.
  • Handle: RePEc:plo:pone00:0235403
    DOI: 10.1371/journal.pone.0235403
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0235403
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0235403&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0235403?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Thorben C. Kundt & Florian Misch & Birger Nerré, 2017. "Re-assessing the merits of measuring tax evasion through business surveys: an application of the crosswise model," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 24(1), pages 112-133, February.
    2. Johannes Landsheer & Peter Van Der Heijden & Ger Van Gils, 1999. "Trust and Understanding, Two Psychological Aspects of Randomized Response," Quality & Quantity: International Journal of Methodology, Springer, vol. 33(1), pages 1-12, February.
    3. Ivar Krumpal, 2013. "Determinants of social desirability bias in sensitive surveys: a literature review," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(4), pages 2025-2047, June.
    4. Korndörfer, Martin & Krumpal, Ivar & Schmukle, Stefan C., 2014. "Measuring and explaining tax evasion: Improving self-reports using the crosswise model," Journal of Economic Psychology, Elsevier, vol. 45(C), pages 18-32.
    5. Xiangen Hu & William Batchelder, 1994. "The statistical analysis of general processing tree models with the EM algorithm," Psychometrika, Springer;The Psychometric Society, vol. 59(1), pages 21-47, March.
    6. Jun-Wu Yu & Guo-Liang Tian & Man-Lai Tang, 2008. "Two new models for survey sampling with sensitive characteristic: design and analysis," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 67(3), pages 251-263, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Burgstaller, Lilith & Feld, Lars P. & Pfeil, Katharina, 2022. "Working in the shadow: Survey techniques for measuring and explaining undeclared work," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 661-671.
    2. Walzenbach, Sandra & Hinz, Thomas, 2022. "Puzzling Answers to Crosswise Questions - Examining Overall Prevalence Rates, Primacy Effects and Learning Effects," EconStor Preprints 249353, ZBW - Leibniz Information Centre for Economics.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Burgstaller, Lilith & Feld, Lars P. & Pfeil, Katharina, 2022. "Working in the shadow: Survey techniques for measuring and explaining undeclared work," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 661-671.
    2. Ivar Krumpal & Thomas Voss, 2020. "Sensitive Questions and Trust: Explaining Respondents’ Behavior in Randomized Response Surveys," SAGE Open, , vol. 10(3), pages 21582440209, July.
    3. Kirchner Antje, 2015. "Validating Sensitive Questions: A Comparison of Survey and Register Data," Journal of Official Statistics, Sciendo, vol. 31(1), pages 31-59, March.
    4. Adrian Hoffmann & Julia Meisters & Jochen Musch, 2021. "Nothing but the truth? Effects of faking on the validity of the crosswise model," PLOS ONE, Public Library of Science, vol. 16(10), pages 1-20, October.
    5. Marc Höglinger & Ben Jann, 2018. "More is not always better: An experimental individual-level validation of the randomized response technique and the crosswise model," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-22, August.
    6. Liu, Yin & Tian, Guo-Liang, 2013. "A variant of the parallel model for sample surveys with sensitive characteristics," Computational Statistics & Data Analysis, Elsevier, vol. 67(C), pages 115-135.
    7. Ó Ceallaigh, Diarmaid & Timmons, Shane & Robertson, Deirdre & Lunn, Pete, 2023. "Measures of problem gambling, gambling behaviours and perceptions of gambling in Ireland," Research Series, Economic and Social Research Institute (ESRI), number RS169.
    8. Walzenbach, Sandra & Hinz, Thomas, 2022. "Puzzling Answers to Crosswise Questions - Examining Overall Prevalence Rates, Primacy Effects and Learning Effects," EconStor Preprints 249353, ZBW - Leibniz Information Centre for Economics.
    9. Korndörfer, Martin & Krumpal, Ivar & Schmukle, Stefan C., 2014. "Measuring and explaining tax evasion: Improving self-reports using the crosswise model," Journal of Economic Psychology, Elsevier, vol. 45(C), pages 18-32.
    10. Ulrich Thy Jensen, 2020. "Is self-reported social distancing susceptible to social desirability bias? Using the crosswise model to elicit sensitive behaviors," Journal of Behavioral Public Administration, Center for Experimental and Behavioral Public Administration, vol. 3(2).
    11. Julia Meisters & Adrian Hoffmann & Jochen Musch, 2020. "Controlling social desirability bias: An experimental investigation of the extended crosswise model," PLOS ONE, Public Library of Science, vol. 15(12), pages 1-13, December.
    12. Carlos Barros, 2012. "Sustainable Tourism in Inhambane-Mozambique," CEsA Working Papers 105, CEsA - Centre for African and Development Studies.
    13. Michael T Gastner & Károly Takács & Máté Gulyás & Zsuzsanna Szvetelszky & Beáta Oborny, 2019. "The impact of hypocrisy on opinion formation: A dynamic model," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-21, June.
    14. Andreas Lagerås & Mathias Lindholm, 2020. "How to ask sensitive multiple‐choice questions," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(2), pages 397-424, June.
    15. Coutts Elisabethen & Jann Ben & Krumpal Ivar & Näher Anatol-Fiete, 2011. "Plagiarism in Student Papers: Prevalence Estimates Using Special Techniques for Sensitive Questions," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 231(5-6), pages 749-760, October.
    16. Babette Bühler & Katja Möhring & Andreas P. Weiland, 2022. "Assessing dissimilarity of employment history information from survey and administrative data using sequence analysis techniques," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(6), pages 4747-4774, December.
    17. Sjöstedt, Martin & Sundström, Aksel & Jagers, Sverker C. & Ntuli, Herbert, 2022. "Governance through community policing: What makes citizens report poaching of wildlife to state officials?," World Development, Elsevier, vol. 160(C).
    18. Tricia Koroknay†Palicz & Joao Montalvao, 2020. "Sex, Lies, and Surveys: The Role of Interviewer Characteristics," Economics Bulletin, AccessEcon, vol. 40(4), pages 3313-3324.
    19. Burke, Mary A. & Carman, Katherine G., 2017. "You can be too thin (but not too tall): Social desirability bias in self-reports of weight and height," Economics & Human Biology, Elsevier, vol. 27(PA), pages 198-222.
    20. Shinichi Kitano, 2021. "Estimation of Determinants of Farmland Abandonment and Its Data Problems," Land, MDPI, vol. 10(6), pages 1-17, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0235403. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.