IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/6961.html
   My bibliography  Save this paper

How survey-to-survey imputation can fail

Author

Listed:
  • Newhouse, D.
  • Shivakumaran, S.
  • Takamatsu, S.
  • Yoshida, N.

Abstract

This paper proposes diagnostics to assess the accuracy of survey-to-survey imputation methods and applies them to examine why imputing from the Household Income and Expenditure Survey into the Labor Force Survey fails to accurately project poverty trends in Sri Lanka between 2006 and 2009. Survey-to-survey imputation methods rely on two key assumptions: (i) that the questions in the two surveys are asked in a consistent way and (ii) that common variables of the two surveys explain a large share of the intertemporal change in household expenditure and poverty. In addition, differences in sampling design can lead validation tests to underestimate the accuracy of survey-to-survey predictions. In Sri Lanka, the causes of failure differ across sectors. In the urban sector, the primary culprit is differences between the two surveys in the design of the questionnaire. In the rural and estate sectors, the set of common variables in the prediction model does not adequately capture changes in poverty. The paper concludes that in Sri Lanka, survey-to-survey imputation between the Household Income and Expenditure Survey and the Labor Force Survey cannot produce accurate poverty estimates unless the Labor Force Survey adds additional questions on assets and is redesigned to use a questionnaire that is compatible with the Household Income and Expenditure Survey. Alternatively, a new welfare-tracking survey that satisfies these conditions could be established.

Suggested Citation

  • Newhouse, D. & Shivakumaran, S. & Takamatsu, S. & Yoshida, N., 2014. "How survey-to-survey imputation can fail," Policy Research Working Paper Series 6961, The World Bank.
    • Handle: RePEc:wbk:wbrwps:6961
    as

    Download full text from publisher

    File URL: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2014/07/01/000158349_20140701155002/Rendered/PDF/WPS6961.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Ravallion, Martin, 1996. "How Well Can Method Substitute for Data? Five Experiments in Poverty Analysis," World Bank Research Observer, World Bank Group, vol. 11(2), pages 199-221, August.
    2. Beegle, Kathleen & De Weerdt, Joachim & Friedman, Jed & Gibson, John, 2012. "Methods of household consumption measurement through surveys: Experimental results from Tanzania," Journal of Development Economics, Elsevier, vol. 98(1), pages 3-18.
    3. Luc Christiaensen & Peter Lanjouw & Jill Luoto & David Stifel, 2012. "Small area estimation-based prediction methods to track poverty: validation and applications," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 10(2), pages 267-297, June.
    4. David Stifel & Luc Christiaensen, 2007. "Tracking Poverty Over Time in the Absence of Comparable Consumption Data," World Bank Economic Review, World Bank Group, vol. 21(2), pages 317-341, June.
    5. Mohamed Douidich & Abdeljaouad Ezzrari & Roy Van der Weide & Paolo Verme, 2016. "Estimating Quarterly Poverty Rates Using Labor Force Surveys: A Primer," World Bank Economic Review, World Bank Group, vol. 30(3), pages 475-500.
    6. Kenneth Harttgen & Stephan Klasen & Sebastian Vollmer, 2013. "An African Growth Miracle? Or: What do Asset Indices Tell Us About Trends in Economic Performance?," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 59, pages 37-61, October.
    7. Alessandro Tarozzi & Angus Deaton, 2009. "Using Census and Survey Data to Estimate Poverty and Inequality for Small Areas," The Review of Economics and Statistics, MIT Press, vol. 91(4), pages 773-792, November.
    8. Yoko Kijima & Lanjouw, Peter, 2003. "Poverty in India during the1990s - a regional perspective," Policy Research Working Paper Series 3141, The World Bank.
    9. Tarozzi, Alessandro, 2011. "Can census data alone signal heterogeneity in the estimation of poverty maps?," Journal of Development Economics, Elsevier, vol. 95(2), pages 170-185, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Atamanov,Aziz & Tandon,Sharad Alan & Lopez-Acevedo,Gladys C. & Vergara Bahena,Mexico Alberto, 2020. "Measuring Monetary Poverty in the Middle East and North Africa (MENA) Region : Data Gaps and Different Options to Address Them," Policy Research Working Paper Series 9259, The World Bank.
    2. World Bank, 2016. "Tunisia Poverty Assessment 2015," World Bank Other Operational Studies 24410, The World Bank.
    3. Jose Cuesta & Gabriel Lara Ibarra, 2017. "Comparing Cross-Survey Micro Imputation and Macro Projection Techniques: Poverty in Post Revolution Tunisia," Journal of Income Distribution, Ad libros publications inc., vol. 25(1), pages 1-30, March.
    4. repec:jid:journl:y:2018:v:25:i:1:p:1-30 is not listed on IDEAS
    5. Kathleen Beegle & Luc Christiaensen & Andrew Dabalen & Isis Gaddis, 2016. "Poverty in a Rising Africa," World Bank Publications, The World Bank, number 22575, June.
    6. Dang,Hai-Anh H. & Lanjouw,Peter F. & Serajuddin,Umar & Dang,Hai-Anh H. & Lanjouw,Peter F. & Serajuddin,Umar, 2014. "Updating poverty estimates at frequent intervals in the absence of consumption data : methods and illustration with reference to a middle-income country," Policy Research Working Paper Series 7043, The World Bank.
    7. Astrid Mathiassen & Bjørn K. Wold, 2019. "Challenges in predicting poverty trends using survey to survey imputation. Experiences from Malawi," Discussion Papers 900, Statistics Norway, Research Department.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ahmed, Faizuddin & Dorji, Cheku & Takamatsu, Shinya & Yoshida, Nobuo, 2014. "Hybrid survey to improve the reliability of poverty statistics in a cost-effective manner," Policy Research Working Paper Series 6909, The World Bank.
    2. Dang,Hai-Anh H. & Lanjouw,Peter F. & Serajuddin,Umar & Dang,Hai-Anh H. & Lanjouw,Peter F. & Serajuddin,Umar, 2014. "Updating poverty estimates at frequent intervals in the absence of consumption data : methods and illustration with reference to a middle-income country," Policy Research Working Paper Series 7043, The World Bank.
    3. Hai-Anh H. Dang & Peter F. Lanjouw & Umar Serajuddin, 2017. "Updating poverty estimates in the absence of regular and comparable consumption data: methods and illustration with reference to a middle-income country," Oxford Economic Papers, Oxford University Press, vol. 69(4), pages 939-962.
    4. Hai‐Anh Dang & Dean Jolliffe & Calogero Carletto, 2019. "Data Gaps, Data Incomparability, And Data Imputation: A Review Of Poverty Measurement Methods For Data‐Scarce Environments," Journal of Economic Surveys, Wiley Blackwell, vol. 33(3), pages 757-797, July.
    5. Luc Christiaensen & Peter Lanjouw & Jill Luoto & David Stifel, 2012. "Small area estimation-based prediction methods to track poverty: validation and applications," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 10(2), pages 267-297, June.
    6. Hai-Anh H. Dang & Peter F. Lanjouw, 2018. "Poverty Dynamics in India between 2004 and 2012: Insights from Longitudinal Analysis Using Synthetic Panel Data," Economic Development and Cultural Change, University of Chicago Press, vol. 67(1), pages 131-170.
    7. Dang,Hai-Anh H., 2018. "To impute or not to impute ? a review of alternative poverty estimation methods in the context of unavailable consumption data," Policy Research Working Paper Series 8403, The World Bank.
    8. Talip Kilic & Thomas Pave Sohnesen, 2019. "Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 65(1), pages 144-165, March.
    9. F. Clementi & A. L. Dabalen & V. Molini & F. Schettino, 2017. "When the Centre Cannot Hold: Patterns of Polarization in Nigeria," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 63(4), pages 608-632, December.
    10. Skoufias, Emmanuel & Diamond, Alexis & Vinha, Katja & Gill, Michael & Dellepiane, Miguel Rebolledo, 2020. "Estimating poverty rates in subnational populations of interest: An assessment of the Simple Poverty Scorecard," World Development, Elsevier, vol. 129(C).
    11. Janina Isabel Steinert & Lucie Dale Cluver & G. J. Melendez-Torres & Sebastian Vollmer, 2018. "One Size Fits All? The Validity of a Composite Poverty Index Across Urban and Rural Households in South Africa," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 136(1), pages 51-72, February.
    12. Sebastian Levine & Benjamin Roberts, 2013. "Robust Estimates of Changes in Poverty and Inequality in Post-Independence Namibia," South African Journal of Economics, Economic Society of South Africa, vol. 81(2), pages 167-191, June.
    13. Ligon, Ethan & Christiaensen, Luc & Sohnesen, Thomas P, 2020. "Should Consumption Sub-Aggregates be Used to Measure Poverty?," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt9b9929jh, Department of Agricultural & Resource Economics, UC Berkeley.
    14. Hassine, Nadia Belhaj, 2014. "Economic inequality in the Arab region," Policy Research Working Paper Series 6911, The World Bank.
    15. Jesse Naidoo, 2009. "Finite-Sample Bias and Inconsistency in the Estimation of Poverty Maps," SALDRU Working Papers 36, Southern Africa Labour and Development Research Unit, University of Cape Town.
    16. World Bank, 2015. "Tanzania Poverty Assessment," World Bank Other Operational Studies 21871, The World Bank.
    17. Hassine, Nadia Belhaj, 2015. "Economic Inequality in the Arab Region," World Development, Elsevier, vol. 66(C), pages 532-556.
    18. Oasis Kodila-Tedika, 2014. "Africa's statistical tragedy: best statistics, best government effectiveness," International Journal of Development Issues, Emerald Group Publishing, vol. 13(2), pages 171-178, July.
    19. Atamanov, Aziz & Tandon, Sharad & Lopez-Acevedo, Gladys & Vergara Bahena, Mexico Alberto, 2020. "Measuring Monetary Poverty in the Middle East and North Africa (MENA) Region: Data Gaps and Different Options to Address Them," IZA Discussion Papers 13363, Institute of Labor Economics (IZA).
    20. World Bank, 2016. "Tunisia Poverty Assessment 2015," World Bank Other Operational Studies 24410, The World Bank.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:6961. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Roula I. Yazigi). General contact details of provider: http://edirc.repec.org/data/dvewbus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.