IDEAS home Printed from https://ideas.repec.org/a/bla/revinw/v65y2019i1p144-165.html

Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies

Author

Listed:
  • Talip Kilic
  • Thomas Pave Sohnesen

Abstract

Based on a randomized survey experiment that was implemented in Malawi, the study finds that observationally‐equivalent, as well as same, households answer the same questions differently depending on whether they are interviewed with a short questionnaire or its longer counterpart. Statistically significant differences in reporting emerge across all topics and question types. In proxy‐based poverty measurement, these reporting differences lead to significantly different predicted poverty rates and Gini coefficients. The difference in poverty predictions ranges from 3 to 7 percentage points, depending on the model specification. A prediction model based only on the proxies that are elicited prior to the variation in questionnaire design yields identical poverty predictions irrespective of the short‐versus‐long questionnaire treatment. The results are relevant for estimating trends with questionnaires exhibiting inter‐temporal variation in design, impact evaluations administering questionnaires of different length and complexity to treatment and control samples, and development programs utilizing proxy‐means tests for targeting.

Suggested Citation

  • Talip Kilic & Thomas Pave Sohnesen, 2019. "Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 65(1), pages 144-165, March.
    • Handle: RePEc:bla:revinw:v:65:y:2019:i:1:p:144-165
    DOI: 10.1111/roiw.12343
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/roiw.12343
    Download Restriction: no
    File URL: https://libkey.io/10.1111/roiw.12343?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Thomas Pave Sohnesen & Niels Stender, 2017. "Is Random Forest a Superior Methodology for Predicting Poverty? An Empirical Assessment," Poverty & Public Policy, John Wiley & Sons, vol. 9(1), pages 118-133, March.
    2. Dang, Hai-Anh H & Kilic, Talip & Hlasny, Vladimir & Abanokova, Kseniya & Carletto, Calogero, 2024. "Using Survey-to-Survey Imputation to Fill Poverty Data Gaps at a Low Cost: Evidence from a Randomized Survey Experiment," IZA Discussion Papers 16792, IZA Network @ LISER.
    3. Dang, Hai-Anh H & Kilic, Talip & Abanokova, Kseniya & Carletto, Calogero, 2024. "Imputing Poverty Indicators without Consumption Data: An Exploratory Analysis," IZA Discussion Papers 17136, IZA Network @ LISER.
    4. Amankwah, Akuffo & Genou Johnson, Darcey Jeanne & Adofo, Josephine Ofori & Gul, Maryam & Palacios-Lopez, Amparo, 2025. "Measuring poverty in Tanzania: Comparison of diary and recall approaches to food consumption data collection," World Development Perspectives, Elsevier, vol. 39(C).
    5. Adan Silverio‐Murillo & Jose Roberto Balmori de la Miyar, 2022. "Remittances and domestic violence," Review of Development Economics, Wiley Blackwell, vol. 26(4), pages 2274-2295, November.
    6. Abate, Gashaw T. & de Brauw, Alan & Hirvonen, Kalle & Wolle, Abdulazize, 2023. "Measuring consumption over the phone: Evidence from a survey experiment in urban Ethiopia," Journal of Development Economics, Elsevier, vol. 161(C).
    7. Abay, Kibrom A. & Berhane, Guush & Hoddinott, John F. & Tafere, Kibrom, 2021. "Assessing response fatigue in phone surveys: Experimental evidence on dietary diversity in Ethiopia," IFPRI discussion papers 2017, International Food Policy Research Institute (IFPRI).
    8. Ligon, Ethan & Christiaensen, Luc & Sohnesen, Thomas P, 2020. "Should Consumption Sub-Aggregates be Used to Measure Poverty?," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt9b9929jh, Department of Agricultural & Resource Economics, UC Berkeley.
    9. Brown, Caitlin & Ravallion, Martin & van de Walle, Dominique, 2018. "A poor means test? Econometric targeting in Africa," Journal of Development Economics, Elsevier, vol. 134(C), pages 109-124.
    10. Jeong, Dahyeon & Aggarwal, Shilpa & Robinson, Jonathan & Kumar, Naresh & Spearot, Alan & Park, David Sungho, 2023. "Exhaustive or exhausting? Evidence on respondent fatigue in long surveys," Journal of Development Economics, Elsevier, vol. 161(C).
    11. Joachim De Weerdt & John Gibson & Kathleen Beegle, 2020. "What Can We Learn from Experimenting with Survey Methods?," Annual Review of Resource Economics, Annual Reviews, vol. 12(1), pages 431-447, October.
    12. Dillon, Andrew & Mensah, Edouard, 2024. "Respondent biases in agricultural household surveys," Journal of Development Economics, Elsevier, vol. 166(C).
    13. Hai‐Anh H. Dang & Talip Kilic & Kseniya Abanokova & Calogero Carletto, 2025. "Poverty Imputation in Contexts Without Consumption Data: A Revisit With Further Refinements," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 71(1), February.
    14. Daniel Gerszon Mahler & Elizabeth Foster & Christoph Lakner & Zander Prinsloo & Rostand Tchouakam Mbouendeu & Samuel K. Tetteh-Baah, 2025. "Constructing Comparable Global Poverty Trends," Global Poverty Monitoring Technical Note Series 45, The World Bank.
    15. Masselus, Lise & Fiala, Nathan, 2024. "Whom to ask? Testing respondent effects in household surveys," Journal of Development Economics, Elsevier, vol. 168(C).
    16. Pave Sohnesen,Thomas & Stender,Niels, 2016. "Is random forest a superior methodology for predicting poverty ? an empirical assessment," Policy Research Working Paper Series 7612, The World Bank.
    17. Astrid Mathiassen & Bjørn K. Getz Wold, 2021. "Predicting poverty trends by survey-to-survey imputation: the challenge of comparability," Oxford Economic Papers, Oxford University Press, vol. 73(3), pages 1153-1174.
    18. Fiala, Nathan & Rose, Julian & Aryemo, Filder & Peters, Jörg, 2022. "The (very) long-run impacts of cash grants during a crisis," Ruhr Economic Papers 961, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    19. Deepti Sharma & Hema Swaminathan & Rahul Lahoti, 2024. "Does it matter who you ask for time-use data?," WIDER Working Paper Series wp-2024-1, World Institute for Development Economic Research (UNU-WIDER).
    20. Fiala, Nathan & Masselus, Lise, 2022. "Whom to ask? Testing respondent effects in household surveys," Ruhr Economic Papers 935, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    21. Kalyani Raghunathan & Mai Mahmoud & Jessica Heckert & Gayathri Ramani & Greg Seymour, 2025. "Do Estimates of Women’s Control over Income and Decisionmaking Vary Across Nationally Representative Survey Programs?," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 179(1), pages 95-122, August.
    22. Astrid Mathiassen & Bjørn K. Wold, 2019. "Challenges in predicting poverty trends using survey to survey imputation. Experiences from Malawi," Discussion Papers 900, Statistics Norway, Research Department.
    23. Peterson-Wilhelm, Bailey & Schwab, Benjamin, 2024. "How does recall bias in farm labor impact separability tests?," Food Policy, Elsevier, vol. 128(C).
    24. Luc Christiaensen & Ethan Ligon & Thomas Pave Sohnesen, 2022. "Consumption Subaggregates Should Not Be Used to Measure Poverty," The World Bank Economic Review, World Bank, vol. 36(2), pages 413-432.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:revinw:v:65:y:2019:i:1:p:144-165. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/iariwea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.