IDEAS home Printed from https://ideas.repec.org/a/eee/deveco/v166y2024ics0304387823001554.html
   My bibliography  Save this article

Does survey mode matter? Comparing in-person and phone agricultural surveys in India

Author

Listed:
  • Anderson, Ellen
  • Lybbert, Travis J.
  • Shenoy, Ashish
  • Singh, Rupika
  • Stein, Daniel

Abstract

Ubiquitous mobile phone ownership makes phone surveying an attractive method of low-cost data collection. We explore differences between in-person and phone survey measures of agricultural production collected for an impact evaluation in India. Phone responses have greater mean and variance, a difference that persists even within a subset of respondents that answered the same question over both modes. Treatment effect estimation remains stable across survey mode, but estimates are less precise when using phone data. These patterns are informative for cost and sample size considerations in study design and for aggregating evidence across study sites or time periods.

Suggested Citation

  • Anderson, Ellen & Lybbert, Travis J. & Shenoy, Ashish & Singh, Rupika & Stein, Daniel, 2024. "Does survey mode matter? Comparing in-person and phone agricultural surveys in India," Journal of Development Economics, Elsevier, vol. 166(C).
    • Handle: RePEc:eee:deveco:v:166:y:2024:i:c:s0304387823001554
    DOI: 10.1016/j.jdeveco.2023.103199
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304387823001554
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jdeveco.2023.103199?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Kilic, Talip & Moylan, Heather & Ilukor, John & Mtengula, Clement & Pangapanga-Phiri, Innocent, 2021. "Root for the tubers: Extended-harvest crop production and productivity measurement in surveys," Food Policy, Elsevier, vol. 102(C).
    2. Abigail R Greenleaf & Aliou Gadiaga & Georges Guiella & Shani Turke & Noelle Battle & Saifuddin Ahmed & Caroline Moreau, 2020. "Comparability of modern contraceptive use estimates between a face-to-face survey and a cellphone survey among women in Burkina Faso," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-15, May.
    3. Angus Deaton, 2005. "ERRATUM: Measuring Poverty in a Growing World (or Measuring Growth in a Poor World)," The Review of Economics and Statistics, MIT Press, vol. 87(2), pages 395-395, May.
    4. Ambel, Alemayehu & McGee, Kevin & Tsegay, Asmelash, 2021. "Reducing Bias in Phone Survey Samples: Effectiveness of Reweighting Techniques Using Face-to-Face Surveys As Frames," 2021 Conference, August 17-31, 2021, Virtual 315301, International Association of Agricultural Economists.
    5. Gourlay, Sydney & Kilic, Talip & Lobell, David B., 2019. "A new spin on an old debate: Errors in farmer-reported production and their implications for inverse scale - Productivity relationship in Uganda," Journal of Development Economics, Elsevier, vol. 141(C).
    6. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    7. David B Lobell & George Azzari & Marshall Burke & Sydney Gourlay & Zhenong Jin & Talip Kilic & Siobhan Murray, 2020. "Eyes in the Sky, Boots on the Ground: Assessing Satellite‐ and Ground‐Based Approaches to Crop Yield Measurement and Analysis," American Journal of Agricultural Economics, John Wiley & Sons, vol. 102(1), pages 202-219, January.
    8. Kosmowski, Frederic & Chamberlin, Jordan & Ayalew, Hailemariam & Sida, Tesfaye & Abay, Kibrom & Craufurd, Peter, 2021. "How accurate are yield estimates from crop cuts? Evidence from smallholder maize farms in Ethiopia," Food Policy, Elsevier, vol. 102(C).
    9. Lant Pritchett & Justin Sandefur, 2015. "Learning from Experiments When Context Matters," American Economic Review, American Economic Association, vol. 105(5), pages 471-475, May.
    10. Anna Josephson & Talip Kilic & Jeffrey D. Michler, 2021. "Socioeconomic impacts of COVID-19 in low-income countries," Nature Human Behaviour, Nature, vol. 5(5), pages 557-565, May.
    11. Robert Garlick & Kate Orkin & Simon Quinn, 2020. "Call Me Maybe: Experimental Evidence on Frequency and Medium Effects in Microenterprise Surveys," The World Bank Economic Review, World Bank, vol. 34(2), pages 418-443.
    12. Zezza,Alberto & Mcgee,Kevin Robert & Wollburg,Philip Randolph & Assefa,Thomas Woldu & Gourlay,Sydney, 2022. "From Necessity to Opportunity : Lessons for Integrating Phone and In-Person Data Collectionfor Agricultural Statistics in a Post-Pandemic World," Policy Research Working Paper Series 10168, The World Bank.
    13. Ambel,Alemayehu A. & Mcgee,Kevin Robert & Tsegay,Asmelash Haile, 2021. "Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries," Policy Research Working Paper Series 9676, The World Bank.
    14. Rachael Meager, 2019. "Understanding the Average Impact of Microcredit Expansions: A Bayesian Hierarchical Analysis of Seven Randomized Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 11(1), pages 57-91, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Markhof, Yannick & Wollburg, Philip & Zezza, Alberto, 2025. "Beyond the records: Data quality and COVID-19 vaccination progress in low- and middle-income countries," Journal of Development Economics, Elsevier, vol. 174(C).
    2. Peterson-Wilhelm, Bailey & Schwab, Benjamin, 2024. "How does recall bias in farm labor impact separability tests?," Food Policy, Elsevier, vol. 128(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    2. Abate, Gashaw T. & de Brauw, Alan & Hirvonen, Kalle & Wolle, Abdulazize, 2023. "Measuring consumption over the phone: Evidence from a survey experiment in urban Ethiopia," Journal of Development Economics, Elsevier, vol. 161(C).
    3. Carletto,Calogero & Dillon,Andrew S. & Zezza,Alberto, 2021. "Agricultural Data Collection to Minimize Measurement Error and Maximize Coverage," Policy Research Working Paper Series 9745, The World Bank.
    4. Joshua Brubaker & Talip Kilic & Philip Wollburg, 2021. "Representativeness of individual-level data in COVID-19 phone surveys: Findings from Sub-Saharan Africa," PLOS ONE, Public Library of Science, vol. 16(11), pages 1-27, November.
    5. Anna Josephson & Jeffrey D. Michler & Talip Kilic & Siobhan Murray, 2024. "The Mismeasure of Weather: Using Remotely Sensed Earth Observation Data in Economic Context," Papers 2409.07506, arXiv.org.
    6. Jovanovic, Nina & Ricker-Gilbert, Jacob, 2025. "Estimating the direct and indirect effects of improved seed adoption on yields: Evidence from DNA-fingerprinting, crop cuts, and self-reporting in Ethiopia," Journal of Development Economics, Elsevier, vol. 174(C).
    7. Furbush, Ann M. & Josephson, Anna & Kilic, Talip & Michler, Jeffrey D., 2025. "Coping or hoping? Livelihood diversification and food insecurity in the COVID-19 pandemic," Food Policy, Elsevier, vol. 131(C).
    8. Elena Serfilippi & Daniele Giovannucci & David Ameyaw & Ankur Bansal & Thomas Asafua Nketsia Wobill & Roberta Blankson & Rashi Mishra, 2022. "Benefits and Challenges of Making Data More Agile: A Review of Recent Key Approaches in Agriculture," Sustainability, MDPI, vol. 14(24), pages 1-18, December.
    9. Zezza,Alberto & Mcgee,Kevin Robert & Wollburg,Philip Randolph & Assefa,Thomas Woldu & Gourlay,Sydney, 2022. "From Necessity to Opportunity : Lessons for Integrating Phone and In-Person Data Collectionfor Agricultural Statistics in a Post-Pandemic World," Policy Research Working Paper Series 10168, The World Bank.
    10. repec:osf:osfxxx:nwp8k_v1 is not listed on IDEAS
    11. Jonathan Fu & Annette Krauss, 2024. "Preparing fertile ground: how does the quality of business environments affect MSE growth?," Small Business Economics, Springer, vol. 63(1), pages 51-103, June.
    12. Kugler, Maurice & Viollaz, Mariana & Duque, Daniel & Gaddis, Isis & Newhouse, David & Palacios-Lopez, Amparo & Weber, Michael, 2023. "How did the COVID-19 crisis affect different types of workers in the developing world?," World Development, Elsevier, vol. 170(C).
    13. Peterson-Wilhelm, Bailey & Schwab, Benjamin, 2024. "How does recall bias in farm labor impact separability tests?," Food Policy, Elsevier, vol. 128(C).
    14. Kibrom A. Abay & Tesfamicheal Wossen & Jordan Chamberlin, 2023. "Mismeasurement and efficiency estimates: Evidence from smallholder survey data in Africa," Journal of Agricultural Economics, Wiley Blackwell, vol. 74(2), pages 413-434, June.
    15. Jeffrey D. Michler & Dewan Abdullah Al Rafi & Jonathan Giezendanner & Anna Josephson & Valerien O. Pede & Elizabeth Tellman, 2024. "Impact Evaluations in Data Poor Settings: The Case of Stress-Tolerant Rice Varieties in Bangladesh," Papers 2409.02201, arXiv.org, revised Jul 2025.
    16. Kibrom A Abay & Nishant Yonzan & Sikandra Kurdi & Kibrom Tafere, 2023. "Revisiting Poverty Trends and the Role of Social Protection Systems in Africa during the COVID-19 Pandemic," Journal of African Economies, Centre for the Study of African Economies, vol. 32(Supplemen), pages 44-68.
    17. Helfand, Steven M. & Taylor, Matthew P.H., 2021. "The inverse relationship between farm size and productivity: Refocusing the debate," Food Policy, Elsevier, vol. 99(C).
    18. Barker, Nathan & Davis, C. Austin & López-Peña, Paula & Mitchell, Harrison & Mobarak, Ahmed Mushfiq & Naguib, Karim & Reimão, Maira Emy & Shenoy, Ashish & Vernot, Corey, 2023. "Migration and resilience during a global crisis," European Economic Review, Elsevier, vol. 158(C).
    19. Masselus, Lise & Petrik, Christina & Ankel-Peters, Jörg, 2024. "Lost in the design space? Construct validity in the microfinance literature," Ruhr Economic Papers 1097, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    20. Dang, Hai-Anh H & Kilic, Talip & Hlasny, Vladimir & Abanokova, Kseniya & Carletto, Calogero, 2024. "Using Survey-to-Survey Imputation to Fill Poverty Data Gaps at a Low Cost: Evidence from a Randomized Survey Experiment," IZA Discussion Papers 16792, Institute of Labor Economics (IZA).
    21. Wollburg, Philip & Tiberti, Marco & Zezza, Alberto, 2021. "Recall length and measurement error in agricultural surveys," Food Policy, Elsevier, vol. 100(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • O13 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Agriculture; Natural Resources; Environment; Other Primary Products
    • Q12 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Agriculture - - - Micro Analysis of Farm Firms, Farm Households, and Farm Input Markets

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:deveco:v:166:y:2024:i:c:s0304387823001554. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/devec .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.