IDEAS home Printed from https://ideas.repec.org/p/hal/spmain/hal-03119861.html
   My bibliography  Save this paper

Sex, Lies and Measurement: Consistency tests for Indirect Response Survey Methods

Author

Listed:
  • Erica Chuang
  • Pascaline Dupas

    (Stanford University)

  • Élise Huillery

    (LEDa - Laboratoire d'Economie de Dauphine - IRD - Institut de Recherche pour le Développement - Université Paris Dauphine-PSL - PSL - Université Paris Sciences et Lettres - CNRS - Centre National de la Recherche Scientifique)

  • Juliette Seban

    (Sciences Po - Sciences Po)

Abstract

Social scientists seeking to analyze socially sanctioned behaviors or attitudes increasingly rely on indirect response survey methods, meant to veil the answers of individual respondents. We propose simple internal consistency tests for two such methods, the list experiment and the randomized response technique (its Warner and Crosswise variants). We implement these tests in two studies on sexual and reproductive health behavior in Cameroon and Côte d'Ivoire. Non-compliance with instructions among surveyed individuals appears high and not easily char-acterizable. The tests we propose can be easily and cheaply embedded in measurement tools, allowing researchers to at least know whether their data is reliable before using it.

Suggested Citation

  • Erica Chuang & Pascaline Dupas & Élise Huillery & Juliette Seban, 2021. "Sex, Lies and Measurement: Consistency tests for Indirect Response Survey Methods," SciencePo Working papers Main hal-03119861, HAL.
    • Handle: RePEc:hal:spmain:hal-03119861
    DOI: 10.1016/j.jdeveco.2020.102582
    as

    Download full text from publisher

    To our knowledge, this item is not available for download. To find whether it is available, there are three options:
    1. Check below whether another version of this item is available online.
    2. Check on the provider's web page whether it is in fact available.
    3. Perform a search for a similarly titled item that would be available.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Höglinger, Marc & Diekmann, Andreas, 2017. "Uncovering a Blind Spot in Sensitive Question Research: False Positives Undermine the Crosswise-Model RRT," Political Analysis, Cambridge University Press, vol. 25(1), pages 131-137, January.
    2. Dupas, Pascaline & Huillery, Elise & Seban, Juliette, 2018. "Risk information, risk salience, and adolescent sexual behavior: Experimental evidence from Cameroon," Journal of Economic Behavior & Organization, Elsevier, vol. 145(C), pages 151-175.
    3. Eady, Gregory, 2017. "The Statistical Analysis of Misreporting on Sensitive Survey Questions," Political Analysis, Cambridge University Press, vol. 25(2), pages 241-259, April.
    4. Graeme Blair & Kosuke Imai & Yang-Yang Zhou, 2015. "Design and Analysis of the Randomized Response Technique," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(511), pages 1304-1319, September.
    5. Pascaline Dupas & Élise Huillery & Juliette Seban, 2018. "Risk Information, Risk Salience, and Teenagers Sexual Behavior: Experimental Evidence from Cameroon," Post-Print hal-01645677, HAL.
    6. David McKenzie & Melissa Siegel, 2013. "Eliciting Illegal migration rates through list randomization," RF Berlin - CReAM Discussion Paper Series 1310, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    7. Marc Höglinger & Ben Jann, 2018. "More is not always better: An experimental individual-level validation of the randomized response technique and the crosswise model," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-22, August.
    8. Blair, Graeme & Imai, Kosuke, 2012. "Statistical Analysis of List Experiments," Political Analysis, Cambridge University Press, vol. 20(1), pages 47-77, January.
    9. Imai, Kosuke, 2011. "Multivariate Regression Analysis for the Item Count Technique," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 407-416.
    10. Ulf Böckenholt & Sema Barlas & Peter G. M. van der Heijden, 2009. "Do randomized‐response designs eliminate response biases? An empirical study of non‐compliance behavior," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 24(3), pages 377-392, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ham, Andrés & Guarín, Ángela & Ruiz, Juanita, 2024. "How accurately are household surveys measuring the LGBT population in Colombia? Evidence from a list experiment," Labour Economics, Elsevier, vol. 87(C).
    2. Gutiérrez Fernández, Emilio & Rubli, Adrian, 2023. "Challenges for Measuring the LGBT+ Population and Homophobia in Mexico," IDB Publications (Working Papers) 12707, Inter-American Development Bank.
    3. David Sungho Park & Shilpa Aggarwal & Dahyeon Jeong & Naresh Kumar & Jonathan Robinson & Alan Spearot, 2021. "Private but Misunderstood? Evidence on Measuring Intimate Partner Violence via Self-Interviewing in Rural Liberia and Malawi," NBER Working Papers 29584, National Bureau of Economic Research, Inc.
    4. Carroll, Eamonn & Timmons, Shane & McGinnity, Frances, 2023. "Experimental tests of public support for disability policy," Research Series, Economic and Social Research Institute (ESRI), number RS159.
    5. Gutierrez, Emilio & Rubli, Adrian, 2024. "LGBT+ persons and homophobia prevalence across job sectors: Survey evidence from Mexico," Labour Economics, Elsevier, vol. 87(C).
    6. Ham, Andrés & Guarin, Angela & Ruiz, Juanita, 2023. "How Accurately are Household Surveys Measuring the Size and Inequalities for the LGBT Population in Bogota, Colombia? Evidence from a List Experiment," IDB Publications (Working Papers) 12677, Inter-American Development Bank.
    7. Marco Gregori & Martijn G. Jong & Rik Pieters, 2024. "The Crosswise Model for Surveys on Sensitive Topics: A General Framework for Item Selection and Statistical Analysis," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 1007-1033, September.
    8. Bertelli, Olivia & Calvo, Thomas & Lavallée, Emmanuelle & Mercier, Marion & Mesplé-Somps, Sandrine, 2024. "What one thinks, what one says and what one does: male justifications and practices of gender-based violence in Mali," CEPREMAP Working Papers (Docweb) 2406, CEPREMAP.
    9. Claire Cullen, 2023. "Method Matters: The Underreporting of Intimate Partner Violence," The World Bank Economic Review, World Bank, vol. 37(1), pages 49-73.
    10. Olivia Bertelli & Thomas Calvo & Emmanuelle Lavallée & Marion Mercier & Sandrine Mesplé-Somps, 2023. "Measuring insecurity-related experiences and preferences in a fragile State. A list experiment in Mali," Working Papers DT/2023/01, DIAL (Développement, Institutions et Mondialisation).
    11. Aksoy, Billur & Carpenter, Christopher S. & Sansone, Dario, 2022. "Understanding Labor Market Discrimination against Transgender People: Evidence from a Double List Experiment and a Survey," IZA Discussion Papers 15542, Institute of Labor Economics (IZA).
    12. Osman, Adam & Speer, Jamin D. & Weaver, Andrew, 2023. "Discrimination against Women in Hiring," IZA Discussion Papers 16598, Institute of Labor Economics (IZA).
    13. Hiroyuki Yamada & Yuki Kanayama & Kanako Yoshikawa & Kyaw Wai Aung, 2022. "Place‐based price differentials of prostitution: a case study in Yangon, Myanmar," Asian-Pacific Economic Literature, The Crawford School, The Australian National University, vol. 36(1), pages 17-29, May.
    14. Marine JOUVIN, 2021. "Addressing social desirability bias in child labor measurement : an application to cocoa farms in Côte d’Ivoire," Bordeaux Economics Working Papers 2021-08, Bordeaux School of Economics (BSE).
    15. Assefa, Thomas W. & Kadam, Aditi & Magnan, Nicholas & McCullough, Ellen & McGavock, Tamara, 2022. "Who is asking and how? The effects of enumerator gender and survey method in measuring intimate partner violence," 2022 Annual Meeting, July 31-August 2, Anaheim, California 322543, Agricultural and Applied Economics Association.
    16. Henry Cust & Aurélia Lépine & Carole Treibich & Timothy Powell‐Jackson & Rosalba Radice & Cheikh Tidiane Ndour, 2024. "Trading HIV for sheep: Risky sexual behavior and the response of female sex workers to Tabaski in Senegal," Health Economics, John Wiley & Sons, Ltd., vol. 33(1), pages 153-193, January.
    17. Carolina Castilla & David M. A. Murphy, 2023. "Bidirectional intimate partner violence: Evidence from a list experiment in Kenya," Health Economics, John Wiley & Sons, Ltd., vol. 32(1), pages 175-193, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Burgstaller, Lilith & Feld, Lars P. & Pfeil, Katharina, 2022. "Working in the shadow: Survey techniques for measuring and explaining undeclared work," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 661-671.
    2. S. Rinken & S. Pasadas-del-Amo & M. Rueda & B. Cobo, 2021. "No magic bullet: estimating anti-immigrant sentiment and social desirability bias with the item-count technique," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(6), pages 2139-2159, December.
    3. Marc Höglinger & Ben Jann, 2018. "More is not always better: An experimental individual-level validation of the randomized response technique and the crosswise model," PLOS ONE, Public Library of Science, vol. 13(8), pages 1-22, August.
    4. Carole Treibich & Aurélia Lépine, 2019. "Estimating misreporting in condom use and its determinants among sex workers: Evidence from the list randomisation method," Health Economics, John Wiley & Sons, Ltd., vol. 28(1), pages 144-160, January.
    5. Henry Cust & Aurélia Lépine & Carole Treibich & Timothy Powell‐Jackson & Rosalba Radice & Cheikh Tidiane Ndour, 2024. "Trading HIV for sheep: Risky sexual behavior and the response of female sex workers to Tabaski in Senegal," Health Economics, John Wiley & Sons, Ltd., vol. 33(1), pages 153-193, January.
    6. Elisabetta de Cao & Clemens Lutz, 2015. "Measuring attitudes regarding female genital mutilation through a list experiment," CSAE Working Paper Series 2015-20, Centre for the Study of African Economies, University of Oxford.
    7. De Cao, Elisabetta & Lutz, Clemens, 2014. "Sensitive survey questions," Research Report 14017-EEF, University of Groningen, Research Institute SOM (Systems, Organisations and Management).
    8. Lépine, Aurélia & Treibich, Carole & D’Exelle, Ben, 2020. "Nothing but the truth: Consistency and efficiency of the list experiment method for the measurement of sensitive health behaviours," Social Science & Medicine, Elsevier, vol. 266(C).
    9. repec:dgr:rugsom:14017-eef is not listed on IDEAS
    10. Marco Gregori & Martijn G. Jong & Rik Pieters, 2024. "The Crosswise Model for Surveys on Sensitive Topics: A General Framework for Item Selection and Statistical Analysis," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 1007-1033, September.
    11. repec:hal:wpspec:info:hdl:2441/76npisrda99aop75h6fmi4vduu is not listed on IDEAS
    12. Celhay, Pablo A. & Depetris-Chauvin, Emilio & Riquelme, Cristina, 2024. "When a strike strikes twice: Massive student mobilizations and teenage pregnancy in Chile," Journal of Development Economics, Elsevier, vol. 170(C).
    13. Cagé, Julia & Rueda, Valeria, 2020. "Sex and the mission: the conflicting effects of early Christian missions on HIV in sub-Saharan Africa," Journal of Demographic Economics, Cambridge University Press, vol. 86(3), pages 213-257, September.
    14. Ezequiel Gonzalez-Ocantos & Chad Kiewiet de Jonge & Carlos Meléndez & David Nickerson & Javier Osorio, 2020. "Carrots and sticks: Experimental evidence of vote-buying and voter intimidation in Guatemala," Journal of Peace Research, Peace Research Institute Oslo, vol. 57(1), pages 46-61, January.
    15. Breda, Thomas & Grenet, Julien & Monnet, Marion & Van Effenterre, Clémentine, 2020. "Do Female Role Models Reduce the Gender Gap in Science? Evidence from French High Schools," IZA Discussion Papers 13163, Institute of Labor Economics (IZA).
    16. Pier Francesco Perri & Eleni Manoli & Tasos C. Christofides, 2023. "Assessing the effectiveness of indirect questioning techniques by detecting liars," Statistical Papers, Springer, vol. 64(5), pages 1483-1506, October.
    17. Susan Athey & Katy Ann Bergstrom & Vitor Hadad & Julian C. Jamison & Berk Özler & Luca Parisotto & Julius Dohbit Sama, 2021. "Shared Decision-Making: Can Improved Counseling Increase Willingness to Pay for Modern Contraceptives?," Discussion Papers 2105, University of Exeter, Department of Economics.
    18. Jouni Kuha & Jonathan Jackson, 2014. "The item count method for sensitive survey questions: modelling criminal behaviour," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 63(2), pages 321-341, February.
    19. Leopoldo Fergusson & Carlos Molina & Juan Felipe Riaño, 2019. "Consumers as VAT “Evaders”: Incidence, Social Bias, and Correlates in Colombia," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 21-67, April.
    20. Leopoldo Fergusson & Carlos Molina & Juan Felipe Riaño, 2018. "I Sell My Vote, and So What? Incidence, Social Bias, and Correlates of Clientelism in Colombia," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Fall 2018), pages 181-218, November.
    21. M. Niaz Asadullah & Elisabetta De Cao & Fathema Zhura Khatoon & Zahra Siddique, 2021. "Measuring gender attitudes using list experiments," Journal of Population Economics, Springer;European Society for Population Economics, vol. 34(2), pages 367-400, April.
    22. Julia Cage & Valeria Rueda, 2017. "Sex and the Mission: The Conflicting Effects of Early Christian Investments on the HIV Epidemic in sub-Saharan Africa," Sciences Po publications 12192, Sciences Po.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:spmain:hal-03119861. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Contact - Sciences Po Departement of Economics (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.