IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/6961.html
   My bibliography  Save this paper

How survey-to-survey imputation can fail

Author

Listed:
  • Newhouse, D.
  • Shivakumaran, S.
  • Takamatsu, S.
  • Yoshida, N.

Abstract

This paper proposes diagnostics to assess the accuracy of survey-to-survey imputation methods and applies them to examine why imputing from the Household Income and Expenditure Survey into the Labor Force Survey fails to accurately project poverty trends in Sri Lanka between 2006 and 2009. Survey-to-survey imputation methods rely on two key assumptions: (i) that the questions in the two surveys are asked in a consistent way and (ii) that common variables of the two surveys explain a large share of the intertemporal change in household expenditure and poverty. In addition, differences in sampling design can lead validation tests to underestimate the accuracy of survey-to-survey predictions. In Sri Lanka, the causes of failure differ across sectors. In the urban sector, the primary culprit is differences between the two surveys in the design of the questionnaire. In the rural and estate sectors, the set of common variables in the prediction model does not adequately capture changes in poverty. The paper concludes that in Sri Lanka, survey-to-survey imputation between the Household Income and Expenditure Survey and the Labor Force Survey cannot produce accurate poverty estimates unless the Labor Force Survey adds additional questions on assets and is redesigned to use a questionnaire that is compatible with the Household Income and Expenditure Survey. Alternatively, a new welfare-tracking survey that satisfies these conditions could be established.

Suggested Citation

  • Newhouse, D. & Shivakumaran, S. & Takamatsu, S. & Yoshida, N., 2014. "How survey-to-survey imputation can fail," Policy Research Working Paper Series 6961, The World Bank.
    • Handle: RePEc:wbk:wbrwps:6961
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2014/07/01/000158349_20140701155002/Rendered/PDF/WPS6961.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Luc Christiaensen & Peter Lanjouw & Jill Luoto & David Stifel, 2012. "Small area estimation-based prediction methods to track poverty: validation and applications," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 10(2), pages 267-297, June.
    2. Kenneth Harttgen & Stephan Klasen & Sebastian Vollmer, 2013. "An African Growth Miracle? Or: What do Asset Indices Tell Us About Trends in Economic Performance?," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 59, pages 37-61, October.
    3. Alessandro Tarozzi & Angus Deaton, 2009. "Using Census and Survey Data to Estimate Poverty and Inequality for Small Areas," The Review of Economics and Statistics, MIT Press, vol. 91(4), pages 773-792, November.
    4. Yoko Kijima & Lanjouw, Peter, 2003. "Poverty in India during the1990s - a regional perspective," Policy Research Working Paper Series 3141, The World Bank.
    5. Ravallion, Martin, 1996. "How Well Can Method Substitute for Data? Five Experiments in Poverty Analysis," The World Bank Research Observer, World Bank, vol. 11(2), pages 199-221, August.
    6. Beegle, Kathleen & De Weerdt, Joachim & Friedman, Jed & Gibson, John, 2012. "Methods of household consumption measurement through surveys: Experimental results from Tanzania," Journal of Development Economics, Elsevier, vol. 98(1), pages 3-18.
    7. David Stifel & Luc Christiaensen, 2007. "Tracking Poverty Over Time in the Absence of Comparable Consumption Data," The World Bank Economic Review, World Bank, vol. 21(2), pages 317-341, June.
    8. Mohamed Douidich & Abdeljaouad Ezzrari & Roy Van der Weide & Paolo Verme, 2016. "Estimating Quarterly Poverty Rates Using Labor Force Surveys: A Primer," The World Bank Economic Review, World Bank, vol. 30(3), pages 475-500.
    9. Tarozzi, Alessandro, 2011. "Can census data alone signal heterogeneity in the estimation of poverty maps?," Journal of Development Economics, Elsevier, vol. 95(2), pages 170-185, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Theresa Beltramo & Hai-Anh H. Dang & Ibrahima Sarr & Paolo Verme, 2020. "Estimating Poverty among Refugee Populations: A Cross-Survey Imputation Exercise for Chad," Working Papers 536, ECINEQ, Society for the Study of Economic Inequality.
    2. Jose Cuesta & Gabriel Lara Ibarra, 2017. "Comparing Cross-Survey Micro Imputation and Macro Projection Techniques: Poverty in Post Revolution Tunisia," Journal of Income Distribution, Ad libros publications inc., vol. 25(1), pages 1-30, March.
    3. Atamanov, Aziz & Tandon, Sharad & Lopez-Acevedo, Gladys & Vergara Bahena, Mexico Alberto, 2020. "Measuring Monetary Poverty in the Middle East and North Africa (MENA) Region: Data Gaps and Different Options to Address Them," IZA Discussion Papers 13363, Institute of Labor Economics (IZA).
    4. Dang,Hai-Anh H. & Lanjouw,Peter F. & Serajuddin,Umar & Dang,Hai-Anh H. & Lanjouw,Peter F. & Serajuddin,Umar, 2014. "Updating poverty estimates at frequent intervals in the absence of consumption data : methods and illustration with reference to a middle-income country," Policy Research Working Paper Series 7043, The World Bank.
    5. World Bank, 2016. "Tunisia Poverty Assessment 2015," World Bank Publications - Reports 24410, The World Bank Group.
    6. Jose Cuesta & Gabriel Lara Ibarra, 2018. "Comparing Cross-Survey Micro Imputation and Macro Projection Techniques: Poverty in Post Revolution Tunisia," Journal of Income Distribution, Ad libros publications inc., vol. 25(1), pages 1-30, March.
    7. Kathleen Beegle & Luc Christiaensen & Andrew Dabalen & Isis Gaddis, 2016. "Poverty in a Rising Africa," World Bank Publications - Books, The World Bank Group, number 22575, December.
    8. Astrid Mathiassen & Bjørn K. Wold, 2019. "Challenges in predicting poverty trends using survey to survey imputation. Experiences from Malawi," Discussion Papers 900, Statistics Norway, Research Department.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ahmed, Faizuddin & Dorji, Cheku & Takamatsu, Shinya & Yoshida, Nobuo, 2014. "Hybrid survey to improve the reliability of poverty statistics in a cost-effective manner," Policy Research Working Paper Series 6909, The World Bank.
    2. Luc Christiaensen & Peter Lanjouw & Jill Luoto & David Stifel, 2012. "Small area estimation-based prediction methods to track poverty: validation and applications," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 10(2), pages 267-297, June.
    3. Dang,Hai-Anh H. & Lanjouw,Peter F. & Serajuddin,Umar & Dang,Hai-Anh H. & Lanjouw,Peter F. & Serajuddin,Umar, 2014. "Updating poverty estimates at frequent intervals in the absence of consumption data : methods and illustration with reference to a middle-income country," Policy Research Working Paper Series 7043, The World Bank.
    4. Dang, Hai-Anh & Lanjouw, Peter F., 2021. "Data Scarcity and Poverty Measurement," IZA Discussion Papers 14631, Institute of Labor Economics (IZA).
    5. Hai-Anh H. Dang & Peter F. Lanjouw & Umar Serajuddin, 2017. "Updating poverty estimates in the absence of regular and comparable consumption data: methods and illustration with reference to a middle-income country," Oxford Economic Papers, Oxford University Press, vol. 69(4), pages 939-962.
    6. Hai‐Anh Dang & Dean Jolliffe & Calogero Carletto, 2019. "Data Gaps, Data Incomparability, And Data Imputation: A Review Of Poverty Measurement Methods For Data‐Scarce Environments," Journal of Economic Surveys, Wiley Blackwell, vol. 33(3), pages 757-797, July.
    7. Hai-Anh H. Dang & Peter F. Lanjouw, 2018. "Poverty Dynamics in India between 2004 and 2012: Insights from Longitudinal Analysis Using Synthetic Panel Data," Economic Development and Cultural Change, University of Chicago Press, vol. 67(1), pages 131-170.
    8. Abate, Gashaw T. & de Brauw, Alan & Hirvonen, Kalle & Wolle, Abdulazize, 2023. "Measuring consumption over the phone: Evidence from a survey experiment in urban Ethiopia," Journal of Development Economics, Elsevier, vol. 161(C).
    9. Hai‐Anh H. Dang, 2021. "To impute or not to impute, and how? A review of poverty‐estimation methods in the absence of consumption data," Development Policy Review, Overseas Development Institute, vol. 39(6), pages 1008-1030, November.
    10. Dang,Hai-Anh H., 2018. "To impute or not to impute ? a review of alternative poverty estimation methods in the context of unavailable consumption data," Policy Research Working Paper Series 8403, The World Bank.
    11. Dang,Hai-Anh H. & Kilic,Talip & Carletto,Calogero & Abanokova,Kseniya, 2021. "Poverty Imputation in Contexts without Consumption Data : A Revisit with Further Refinements," Policy Research Working Paper Series 9838, The World Bank.
    12. Hai-Anh H. Dang & Peter F. Lanjouw, 2023. "Regression-based imputation for poverty measurement in data-scarce settings," Chapters, in: Jacques Silber (ed.), Research Handbook on Measuring Poverty and Deprivation, chapter 13, pages 141-150, Edward Elgar Publishing.
    13. Talip Kilic & Thomas Pave Sohnesen, 2019. "Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 65(1), pages 144-165, March.
    14. Dang, Hai-Anh & Kilic, Talip & Hlasny, Vladimir & Abanokova, Kseniya & Carletto, Calogero, 2024. "Using Survey-to-Survey Imputation to Fill Poverty Data Gaps at a Low Cost: Evidence from a Randomized Survey Experiment," IZA Discussion Papers 16792, Institute of Labor Economics (IZA).
    15. Lain,Jonathan William & Schoch,Marta & Vishwanath,Tara, 2022. "Estimating a Poverty Trend for Nigeria between 2009 and 2019," Policy Research Working Paper Series 9974, The World Bank.
    16. F. Clementi & A. L. Dabalen & V. Molini & F. Schettino, 2017. "When the Centre Cannot Hold: Patterns of Polarization in Nigeria," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 63(4), pages 608-632, December.
    17. Skoufias, Emmanuel & Diamond, Alexis & Vinha, Katja & Gill, Michael & Dellepiane, Miguel Rebolledo, 2020. "Estimating poverty rates in subnational populations of interest: An assessment of the Simple Poverty Scorecard," World Development, Elsevier, vol. 129(C).
    18. Betti,Gianni & Molini,Vasco & Mori,Lorenzo, 2022. "New Algorithm to Estimate Inequality Measures in Cross-Survey Imputation : An Attemptto Correct the Underestimation of Extreme Values," Policy Research Working Paper Series 10013, The World Bank.
    19. Janina Isabel Steinert & Lucie Dale Cluver & G. J. Melendez-Torres & Sebastian Vollmer, 2018. "One Size Fits All? The Validity of a Composite Poverty Index Across Urban and Rural Households in South Africa," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(1), pages 51-72, February.
    20. Sebastian Levine & Benjamin Roberts, 2013. "Robust Estimates of Changes in Poverty and Inequality in Post-Independence Namibia," South African Journal of Economics, Economic Society of South Africa, vol. 81(2), pages 167-191, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:6961. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.