IDEAS home Printed from https://ideas.repec.org/a/eee/deveco/v98y2012i1p19-33.html
   My bibliography  Save this article

Improving consumption measurement and other survey data through CAPI: Evidence from a randomized experiment

Author

Listed:
  • Caeyers, Bet
  • Chalmers, Neil
  • De Weerdt, Joachim

Abstract

This paper reports on a randomized survey experiment among 1840 households, designed to compare pen-and-paper interviewing (PAPI) to computer-assisted personal interviewing (CAPI). We find that PAPI data contain a large number of errors, which can be avoided in CAPI. Error counts are not randomly distributed across the sample, but are correlated with household characteristics, potentially introducing sample bias if dubious observations need to be dropped. We demonstrate a tendency for the spread of total measured consumption to be higher on paper compared to CAPI, translating into significantly higher measured inequality. Investigating further the nature of PAPI's measurement error for consumption, we fail to reject the hypothesis that it is classical: it attenuates the coefficient on consumption when used as explanatory variable and we find no evidence of bias when consumption is used as dependent variable. Finally, CAPI and PAPI are compared in terms of interview length, costs and respondents' perceptions.

Suggested Citation

  • Caeyers, Bet & Chalmers, Neil & De Weerdt, Joachim, 2012. "Improving consumption measurement and other survey data through CAPI: Evidence from a randomized experiment," Journal of Development Economics, Elsevier, vol. 98(1), pages 19-33.
  • Handle: RePEc:eee:deveco:v:98:y:2012:i:1:p:19-33
    DOI: 10.1016/j.jdeveco.2011.12.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304387811001167
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jdeveco.2011.12.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Andrew Chesher & Christian Schluter, 2002. "Welfare Measurement and Measurement Error," Review of Economic Studies, Oxford University Press, vol. 69(2), pages 357-378.
    2. Bound, John & Krueger, Alan B, 1991. "The Extent of Measurement Error in Longitudinal Earnings Data: Do Two Wrongs Make a Right?," Journal of Labor Economics, University of Chicago Press, vol. 9(1), pages 1-24, January.
    3. Bart Capéau & Stefan Dercon, 2006. "Prices, Unit Values and Local Measurement Units in Rural Surveys: an Econometric Approach with an Application to Poverty Measurement in Ethiopia," Journal of African Economies, Centre for the Study of African Economies (CSAE), vol. 15(2), pages 181-211, June.
    4. Fafchamps, Marcel & McKenzie, David & Quinn, Simon & Woodruff, Christopher, 2012. "Using PDA consistency checks to increase the precision of profits and sales measurement in panels," Journal of Development Economics, Elsevier, vol. 98(1), pages 51-57.
    5. Edith de Leeuw & William Nicholls II, 1996. "Technological Innovations in Data Collection: Acceptance, Data Quality and Costs," Sociological Research Online, , vol. 1(4), pages 23-37, December.
    6. Fafchamps, Marcel & McKenzie, David & Quinn, Simon & Woodruff, Christopher, 2012. "Using PDA consistency checks to increase the precision of profits and sales measurement in panels," Journal of Development Economics, Elsevier, vol. 98(1), pages 51-57.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christensen, Cheryl, 2018. "Progress and Challenges in Global Food Security," Amber Waves:The Economics of Food, Farming, Natural Resources, and Rural America, United States Department of Agriculture, Economic Research Service, vol. 0(01), February.
    2. Ambler, Kate & Herskowitz, Sylvan & Maredia, Mywish K., 2021. "Are we done yet? Response fatigue and rural livelihoods," Journal of Development Economics, Elsevier, vol. 153(C).
    3. Joachim De Weerdt & John Gibson & Kathleen Beegle, 2020. "What Can We Learn from Experimenting with Survey Methods?," Annual Review of Resource Economics, Annual Reviews, vol. 12(1), pages 431-447, October.
    4. Joachim De Weerdt & Kathleen Beegle & Jed Friedman & John Gibson, 2016. "The Challenge of Measuring Hunger through Survey," Economic Development and Cultural Change, University of Chicago Press, vol. 64(4), pages 727-758.
    5. De Weerdt, Joachim & Beegle, Kathleen & Friedman,, Jed & Gibson, John, 2014. "The challenge of measuring hunger," Policy Research Working Paper Series 6736, The World Bank.
    6. Demombynes, Gabriel & Gubbins, Paul & Romeo, Alessandro, 2013. "Challenges and opportunities of mobile phone-based data collection : evidence from South Sudan," Policy Research Working Paper Series 6321, The World Bank.
    7. Joachim De Weerdt & Kalle Hirvonen, 2016. "Risk Sharing and Internal Migration," Economic Development and Cultural Change, University of Chicago Press, vol. 65(1), pages 63-86.
    8. De Weerdt,Joachim & Christiaensen,Luc & Kanbur,Ravi, 2021. "When Distance Drives Destination, Towns Can Stimulate Development," Policy Research Working Paper Series 9622, The World Bank.
    9. Mark Brooks & Rattiya S. Lippe & Hermann Waibel, 2020. "Comprehensive data quality studies as a component of poverty assessments," TVSEP Working Papers wp-019, Leibniz Universitaet Hannover, Institute of Development and Agricultural Economics, Project TVSEP.
    10. Utz Pape & Philip Wollburg, 2019. "Estimation of Poverty in Somalia Using Innovative Methodologies," HiCN Working Papers 306, Households in Conflict Network.
    11. Joachim De Weerdt & Garance Genicot & Alice Mesnard, 2019. "Asymmetry of Information within Family Networks," Journal of Human Resources, University of Wisconsin Press, vol. 54(1), pages 225-254.
    12. Crawfurd, Lee & Evans, David K. & Hares, Susannah & Sandefur, Justin, 2023. "Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone," Journal of Development Economics, Elsevier, vol. 164(C).
    13. Root, Christopher & Maredia, Mywish K., 2017. "Testing the Local Enumerator Approach for Farm Level Data Collection: The Case of Natural Resource Management Technology Adoption in India," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 258098, Agricultural and Applied Economics Association.
    14. Utz Pape & Luca Parisotto, 2019. "Estimating Poverty in a Fragile Context – The High Frequency Survey in South Sudan," HiCN Working Papers 305, Households in Conflict Network.
    15. Böhme, Marcus & Stöhr, Tobias, 2012. "Guidelines for the use of household interview duration analysis in CAPI survey management," Kiel Working Papers 1779, Kiel Institute for the World Economy (IfW Kiel).
    16. Reitmann, Ann-Kristin & Goedhuys, Micheline & Grimm, Michael & Nillesen, Eleonora E.M., 2020. "Gender attitudes in the Arab region – The role of framing and priming effects," Journal of Economic Psychology, Elsevier, vol. 80(C).
    17. Xavier Cirera & Diego A. Comin & Marcio Cruz & Kyung Min Lee, 2020. "Anatomy of Technology in the Firm," NBER Working Papers 28080, National Bureau of Economic Research, Inc.
    18. Abay, Kibrom A. & Berhane, Guush & Hoddinott, John F. & Tafere, Kibrom, 2021. "Assessing response fatigue in phone surveys: Experimental evidence on dietary diversity in Ethiopia," IFPRI discussion papers 2017, International Food Policy Research Institute (IFPRI).
    19. Li, Wenchao & Song, Changcheng & Xu, Shu & Yi, Junjian, 2017. "Household Portfolio Choice, Reference Dependence, and the Marriage Market," IZA Discussion Papers 10528, Institute of Labor Economics (IZA).
    20. Johanna Choumert-Nkolo & Pascale Phelinas, 2018. "New paradigms for household surveys in low and middle income countries [Nouveaux paradigmes d'élaboration des enquêtes ménages dans les pays du Sud]," Working Papers halshs-01888609, HAL.
    21. Abate, Gashaw T. & de Brauw, Alan & Hirvonen, Kalle & Wolle, Abdulazize, 2023. "Measuring consumption over the phone: Evidence from a survey experiment in urban Ethiopia," Journal of Development Economics, Elsevier, vol. 161(C).
    22. Fitzpatrick, Anne, 2023. "Which price is right? A comparison of three standard approaches to measuring prices," Journal of Development Economics, Elsevier, vol. 163(C).
    23. Brooks, Mark & Lippe, Rattiya S. & Waibel, Hermann, 2021. "PAPI is gone, but errors remain: Non-sampling errors in household surveys," 2021 Conference, August 17-31, 2021, Virtual 315277, International Association of Agricultural Economists.
    24. Emmanuel Nshakira-Rukundo & Essa Chanie Mussa & Nathan Nshakira & Nicolas Gerber & Joachim von Braun, 2021. "Impact of community-based health insurance on utilisation of preventive health services in rural Uganda: a propensity score matching approach," International Journal of Health Economics and Management, Springer, vol. 21(2), pages 203-227, June.
    25. Rao, Lakshman Nagraj & Gentile, Elisabetta & Pipon, Dave & Roque, Jude David & Thuy, Vu Thi Thu, 2020. "The impact of computer-assisted personal interviewing on survey duration, quality, and cost: Evidence from the Viet Nam Labor Force Survey," GLO Discussion Paper Series 605, Global Labor Organization (GLO).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nicoletti, Cheti & Peracchi, Franco & Foliano, Francesca, 2011. "Estimating Income Poverty in the Presence of Missing Data and Measurement Error," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 61-72.
    2. García-Suaza, A & Lobo, J & Montoya, S & Ordóñez, J & Oviedo, J. D, 2022. "Impact of the collection mode on labor income data. A study in the times of COVID19," Documentos de Trabajo 20396, Universidad del Rosario.
    3. John Abowd & Martha Stinson, 2011. "Estimating Measurement Error in SIPP Annual Job Earnings: A Comparison of Census Bureau Survey and SSA Administrative Data," Working Papers 11-20, Center for Economic Studies, U.S. Census Bureau.
    4. Peter Gottschalk & Minh Huynh, 2010. "Are Earnings Inequality and Mobility Overstated? The Impact of Nonclassical Measurement Error," The Review of Economics and Statistics, MIT Press, vol. 92(2), pages 302-315, May.
    5. de Nicola, Francesca & Giné, Xavier, 2014. "How accurate are recall data? Evidence from coastal India," Journal of Development Economics, Elsevier, vol. 106(C), pages 52-65.
    6. Ingrid Woolard & Stephan Klasen, 2005. "Determinants of Income Mobility and Household Poverty Dynamics in South Africa," Journal of Development Studies, Taylor & Francis Journals, vol. 41(5), pages 865-897.
    7. Lisa M. Dragoset & Gary S. Fields, 2006. "U.S. Earnings Mobility: Comparing Survey-Based and Administrative-Based Estimates," Working Papers 55, ECINEQ, Society for the Study of Economic Inequality.
    8. Meghir, Costas & Pistaferri, Luigi, 2011. "Earnings, Consumption and Life Cycle Choices," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 9, pages 773-854, Elsevier.
    9. Ana Cinta G. Cabral & Norman Gemmell & Nazila Alinaghi, 2021. "Are survey-based self-employment income underreporting estimates biased? New evidence from matched register and survey data," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 28(2), pages 284-322, April.
    10. Abay, Kibrom A. & Abate, Gashaw T. & Barrett, Christopher B. & Bernard, Tanguy, 2019. "Correlated non-classical measurement errors, ‘Second best’ policy inference, and the inverse size-productivity relationship in agriculture," Journal of Development Economics, Elsevier, vol. 139(C), pages 171-184.
    11. D. O'Neill & Sweetman. O. & Van de gaer D., 2005. "The Consequences of Non-Classical Measurement Error for Distributional Analysis," Economics Department Working Paper Series n1490205, Department of Economics, National University of Ireland - Maynooth.
    12. Yingyao Hu & Geert Ridder, 2012. "Estimation of nonlinear models with mismeasured regressors using marginal information," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(3), pages 347-385, April.
    13. Hu, Yingyao & Schennach, Susanne & Shiu, Ji-Liang, 2022. "Identification of nonparametric monotonic regression models with continuous nonclassical measurement errors," Journal of Econometrics, Elsevier, vol. 226(2), pages 269-294.
    14. Daniel Wilhelm, 2018. "Testing for the presence of measurement error," CeMMAP working papers CWP45/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    15. Francisca Antman & David McKenzie, 2007. "Poverty traps and nonlinear income dynamics with measurement error and individual heterogeneity," Journal of Development Studies, Taylor & Francis Journals, vol. 43(6), pages 1057-1083.
    16. Stüber, Heiko & Grabka, Markus M. & Schnitzlein, Daniel D., 2023. "A tale of two data sets: comparing German administrative and survey data using wage inequality as an example," Journal for Labour Market Research, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany], vol. 57, pages 1-8.
    17. Costas Meghir & Luigi Pistaferri, 2004. "Income Variance Dynamics and Heterogeneity," Econometrica, Econometric Society, vol. 72(1), pages 1-32, January.
    18. John Gibson & Kathleen Beegle & Joachim De Weerdt & Jed Friedman, 2015. "What does Variation in Survey Design Reveal about the Nature of Measurement Errors in Household Consumption?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(3), pages 466-474, June.
    19. Donal O'Neill, 2015. "Correcting for Self-Reporting Bias in BMI: A Multiple Imputation Approach," Economics Department Working Paper Series n263-15.pdf, Department of Economics, National University of Ireland - Maynooth.
    20. Peter Gottschalk, 2005. "Downward Nominal-Wage Flexibility: Real or Measurement Error?," The Review of Economics and Statistics, MIT Press, vol. 87(3), pages 556-568, August.

    More about this item

    Keywords

    CAPI; Household surveys; Consumption measurement; Measurement error;
    All these keywords.

    JEL classification:

    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software
    • D12 - Microeconomics - - Household Behavior - - - Consumer Economics: Empirical Analysis
    • I32 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Measurement and Analysis of Poverty

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:deveco:v:98:y:2012:i:1:p:19-33. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/devec .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.