IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp18072.html
   My bibliography  Save this paper

Unbiased and Accurate: Measuring Sensitive Outcomes Through Ballot-Bag Surveys

Author

Listed:
  • Crépon, Bruno

    (CREST)

  • Elsayed, Ahmed

    (American University in Cairo)

  • Gazeaud, Jules

    (Jameel Poverty Action Lab (JPAL))

Abstract

Prevailing methods for measuring sensitive outcomes confront researchers with an inherent bias-variance trade- off: direct questioning is prone to a sensitivity bias, while indirect methods such as list experiments are substantially less precise. We introduce the ballot-bag, a novel technique that relaxes this trade-off by mitigating bias in direct questioning while improving precision over indirect methods. In a field experiment in Egypt, where direct questions on irregular migration are biased, ballot-bag estimates closely align with those from a list experiment but exhibit significantly lower variance. Consequently, treatment effects are highly significant via the ballot-bag and not via the list experiment.

Suggested Citation

  • Crépon, Bruno & Elsayed, Ahmed & Gazeaud, Jules, 2025. "Unbiased and Accurate: Measuring Sensitive Outcomes Through Ballot-Bag Surveys," IZA Discussion Papers 18072, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp18072
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp18072.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Billur Aksoy & Christopher S. Carpenter & Dario Sansone, 2025. "Understanding Labor Market Discrimination Against Transgender People: Evidence from a Double List Experiment and a Survey," Management Science, INFORMS, vol. 71(1), pages 659-677, January.
    2. Kerwin, Jason & Rostom, Nada & Sterck, Olivier, 2024. "Striking the Right Balance: Why Standard Balance Tests Over-Reject the Null, and How to Fix It," IZA Discussion Papers 17217, Institute of Labor Economics (IZA).
    3. McKenzie D. & Siegel M., 2013. "Eliciting illegal migration rates through list randomization," MERIT Working Papers 2013-023, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    4. Bertelli, Olivia & Calvo, Thomas & Lavallée, Emmanuelle & Mercier, Marion & Mesplé-Somps, Sandrine, 2025. "What one thinks, what one says and what one does: Male justifications and practices of gender-based violence in Mali," Journal of Development Economics, Elsevier, vol. 176(C).
    5. Jeong, Dahyeon & Aggarwal, Shilpa & Robinson, Jonathan & Kumar, Naresh & Spearot, Alan & Park, David Sungho, 2023. "Exhaustive or exhausting? Evidence on respondent fatigue in long surveys," Journal of Development Economics, Elsevier, vol. 161(C).
    6. Claire Cullen, 2023. "Method Matters: The Underreporting of Intimate Partner Violence," The World Bank Economic Review, World Bank, vol. 37(1), pages 49-73.
    7. Chuang, Erica & Dupas, Pascaline & Huillery, Elise & Seban, Juliette, 2021. "Sex, lies, and measurement: Consistency tests for indirect response survey methods," Journal of Development Economics, Elsevier, vol. 148(C).
    8. Marine Jouvin, 2024. "Addressing Social Desirability Bias When Measuring Child Labor Use: An Application to Cocoa Farms in Côte d’Ivoire," The World Bank Economic Review, World Bank, vol. 38(3), pages 625-646.
    9. Chuang, Erica & Dupas, Pascaline & Huillery, Elise & Seban, Juliette, 2021. "Sex, lies, and measurement: Consistency tests for indirect response survey methods," Journal of Development Economics, Elsevier, vol. 148(C).
    10. Alex Armand & Britta Augsburg & Antonella Bancalari & Maitreesh Ghatak, 2023. "Public service delivery, exclusion and externalities: Theory and experimental evidence from India," IFS Working Papers W23/37, Institute for Fiscal Studies.
    11. Lépine, Aurélia & Treibich, Carole & D’Exelle, Ben, 2020. "Nothing but the truth: Consistency and efficiency of the list experiment method for the measurement of sensitive health behaviours," Social Science & Medicine, Elsevier, vol. 266(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andreas Kotsadam & Mette Løvgren, 2025. "Is it Time to Put a Moratorium on List Experiments for Domestic Violence Elicitation?," CESifo Working Paper Series 12028, CESifo.
    2. Listo, Ariel & Muñoz, Ercio A. & Sansone, Dario, 2025. "Measuring the Sources of Taste-Based Discrimination Using List Experiments," IZA Discussion Papers 17976, Institute of Labor Economics (IZA).
    3. Rebecca G. Njuguna & Henry Cust & The POWER Team & Aurélia Lépine, 2025. "Does the Risk Premium Differ Between Women Engaging in Commercial and Transactional Sex? Evidence From Urban Cameroon," Health Economics, John Wiley & Sons, Ltd., vol. 34(8), pages 1474-1486, August.
    4. Henry Cust & Aurélia Lépine & Carole Treibich & Timothy Powell‐Jackson & Rosalba Radice & Cheikh Tidiane Ndour, 2024. "Trading HIV for sheep: Risky sexual behavior and the response of female sex workers to Tabaski in Senegal," Health Economics, John Wiley & Sons, Ltd., vol. 33(1), pages 153-193, January.
    5. Abate, Gashaw T. & Abay, Kibrom A. & Chamberlin, Jordan & Sebsibie, Samuel, 2024. "Measuring land rental market participation in smallholder agriculture can survey design innovations improve land market participation statistics?," IFPRI discussion papers 2255, International Food Policy Research Institute (IFPRI).
    6. Aurelia Lepine & Yah Ariane Bernadette N’Djore & Carole Treibich & Henry Cust & Laurent Foubert & Megan Passey & Selina Binder, 2024. "Estimating the prevalence of child labour in the cocoa industry via indirect elicitation methods: a mixed-methods study," Journal of Population Economics, Springer;European Society for Population Economics, vol. 37(4), pages 1-29, December.
    7. Olivia Bertelli & Thomas Calvo & Emmanuelle Lavallée & Marion Mercier & Sandrine Mesplé-Somps, 2023. "Measuring insecurity-related experiences and preferences in a fragile State. A list experiment in Mali," Working Papers DT/2023/01, DIAL (Développement, Institutions et Mondialisation).
    8. Carolina Castilla & David M. A. Murphy, 2023. "Bidirectional intimate partner violence: Evidence from a list experiment in Kenya," Health Economics, John Wiley & Sons, Ltd., vol. 32(1), pages 175-193, January.
    9. Billur Aksoy & Christopher S. Carpenter & Dario Sansone, 2025. "Understanding Labor Market Discrimination Against Transgender People: Evidence from a Double List Experiment and a Survey," Management Science, INFORMS, vol. 71(1), pages 659-677, January.
    10. Leonardo Bursztyn & Ingar Haaland & Nicolas Röver & Christopher Roth, 2025. "The Social Desirability Atlas," ECONtribute Discussion Papers Series 365, University of Bonn and University of Cologne, Germany.
    11. Bertelli, Olivia & Calvo, Thomas & Lavallée, Emmanuelle & Mercier, Marion & Mesplé-Somps, Sandrine, 2025. "What one thinks, what one says and what one does: Male justifications and practices of gender-based violence in Mali," Journal of Development Economics, Elsevier, vol. 176(C).
    12. David Sungho Park & Shilpa Aggarwal & Dahyeon Jeong & Naresh Kumar & Jonathan Robinson & Alan Spearot, 2021. "Private but Misunderstood? Evidence on Measuring Intimate Partner Violence via Self-Interviewing in Rural Liberia and Malawi," NBER Working Papers 29584, National Bureau of Economic Research, Inc.
    13. Assefa, Thomas W. & Kadam, Aditi & Magnan, Nicholas & McCullough, Ellen & McGavock, Tamara, 2022. "Who is asking and how? The effects of enumerator gender and survey method in measuring intimate partner violence," 2022 Annual Meeting, July 31-August 2, Anaheim, California 322543, Agricultural and Applied Economics Association.
    14. Gutiérrez Fernández, Emilio & Rubli, Adrian, 2023. "Challenges for Measuring the LGBT+ Population and Homophobia in Mexico," IDB Publications (Working Papers) 12707, Inter-American Development Bank.
    15. Carroll, Eamonn & Timmons, Shane & McGinnity, Frances, 2023. "Experimental tests of public support for disability policy," Research Series, Economic and Social Research Institute (ESRI), number RS159.
    16. Claire Cullen, 2023. "Method Matters: The Underreporting of Intimate Partner Violence," The World Bank Economic Review, World Bank, vol. 37(1), pages 49-73.
    17. Hiroyuki Yamada & Yuki Kanayama & Kanako Yoshikawa & Kyaw Wai Aung, 2023. "Risk attitude, risky behaviour and price determination in the sex market: A case study of Yangon, Myanmar," Pacific Economic Review, Wiley Blackwell, vol. 28(5), pages 665-691, December.
    18. Granja, Cintia & Visentin, Fabiana & Carneiro, Ana Maria, 2023. "Can international mobility shape students' attitudes toward inequality?," MERIT Working Papers 2023-001, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    19. Peterson-Wilhelm, Bailey & Schwab, Benjamin, 2024. "How does recall bias in farm labor impact separability tests?," Food Policy, Elsevier, vol. 128(C).
    20. Beegle, Kathleen & Dillon, Andrew & Karlan, Dean & Udry, Christopher, 2024. "Introduction to the journal of development economics special issue on methods and measurement," Journal of Development Economics, Elsevier, vol. 170(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • J61 - Labor and Demographic Economics - - Mobility, Unemployment, Vacancies, and Immigrant Workers - - - Geographic Labor Mobility; Immigrant Workers
    • O15 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Economic Development: Human Resources; Human Development; Income Distribution; Migration
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp18072. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.