IDEAS home Printed from https://ideas.repec.org/p/inq/inqwps/ecineq2023-642.html
   My bibliography  Save this paper

Predicting Poverty with Missing Incomes

Author

Listed:
  • Paolo Verme

    (World Bank)

Abstract

Poverty prediction models are used by economists to address missing data issues in a variety of contexts such as poverty profiling, targeting with proxy-means tests, cross-survey imputations such as poverty mapping, or vulnerability analyses. Based on the models used by this literature, this paper conducts an experiment by artificially corrupting data with different patterns and shares of missing incomes. It then compares the capacity of classic econometric and machine learning models to predict poverty under these different scenarios. It finds that the quality of predictions and the choice of the optimal prediction model are dependent on the distribution of observed and unobserved incomes, the poverty line, the choice of objective function and policy preferences, and various other modeling choices. Logistic and random forest models are found to be more robust than other models to variations in these features, but no model invariably outperforms all others. The paper concludes with some reflections on the use of these models for predicting poverty.

Suggested Citation

  • Paolo Verme, 2023. "Predicting Poverty with Missing Incomes," Working Papers 642, ECINEQ, Society for the Study of Economic Inequality.
  • Handle: RePEc:inq:inqwps:ecineq2023-642
    as

    Download full text from publisher

    File URL: http://www.ecineq.org/milano/WP/ECINEQ2023-642.pdf
    File Function: First version, 2023
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. David Coady & Margaret Grosh & John Hoddinott, 2004. "Targeting of Transfers in Developing Countries : Review of Lessons and Experience," World Bank Publications - Books, The World Bank Group, number 14902.
    2. Lillard, Lee & Smith, James P & Welch, Finis, 1986. "What Do We Really Know about Wages? The Importance of Nonreporting and Census Imputation," Journal of Political Economy, University of Chicago Press, vol. 94(3), pages 489-506, June.
    3. Anthony B. Atkinson & Thomas Piketty & Emmanuel Saez, 2011. "Top Incomes in the Long Run of History," Journal of Economic Literature, American Economic Association, vol. 49(1), pages 3-71, March.
    4. Baker, Judy L. & Grosh, Margaret E., 1994. "Poverty reduction through geographic targeting: How well does it work?," World Development, Elsevier, vol. 22(7), pages 983-995, July.
    5. Wodon, Quentin T., 1997. "Targeting the poor using ROC curves," World Development, Elsevier, vol. 25(12), pages 2083-2092, December.
    6. Morduch, Jonathan, 1994. "Poverty and Vulnerability," American Economic Review, American Economic Association, vol. 84(2), pages 221-225, May.
    7. John Gibson, 2019. "Are You Estimating the Right Thing? An Editor Reflects," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 41(3), pages 329-350.
    8. Alessandro Tarozzi & Angus Deaton, 2009. "Using Census and Survey Data to Estimate Poverty and Inequality for Small Areas," The Review of Economics and Statistics, MIT Press, vol. 91(4), pages 773-792, November.
    9. Hai‐Anh Dang & Dean Jolliffe & Calogero Carletto, 2019. "Data Gaps, Data Incomparability, And Data Imputation: A Review Of Poverty Measurement Methods For Data‐Scarce Environments," Journal of Economic Surveys, Wiley Blackwell, vol. 33(3), pages 757-797, July.
    10. Emily Aiken & Suzanne Bellue & Dean Karlan & Chris Udry & Joshua E. Blumenstock, 2022. "Machine learning and phone data can improve targeting of humanitarian aid," Nature, Nature, vol. 603(7903), pages 864-870, March.
    11. Stephen P. Jenkins, 2017. "Pareto Models, Top Incomes and Recent Trends in UK Income Inequality," Economica, London School of Economics and Political Science, vol. 84(334), pages 261-289, April.
    12. Brown, Caitlin & Ravallion, Martin & van de Walle, Dominique, 2018. "A poor means test? Econometric targeting in Africa," Journal of Development Economics, Elsevier, vol. 134(C), pages 109-124.
    13. Korinek, Anton & Mistiaen, Johan A. & Ravallion, Martin, 2007. "An econometric method of correcting for unit nonresponse bias in surveys," Journal of Econometrics, Elsevier, vol. 136(1), pages 213-235, January.
    14. Linden McBride & Austin Nichols, 2018. "Retooling Poverty Targeting Using Out-of-Sample Validation and Machine Learning," The World Bank Economic Review, World Bank, vol. 32(3), pages 531-550.
    15. Glewwe, Paul, 1991. "Investigating the determinants of household welfare in Cote d'Ivoire," Journal of Development Economics, Elsevier, vol. 35(2), pages 307-337, April.
    16. Anton Korinek & Johan Mistiaen & Martin Ravallion, 2006. "Survey nonresponse and the distribution of income," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 4(1), pages 33-55, April.
    17. Cesar Calvo & Stefan Dercon, 2013. "Vulnerability to individual and aggregate poverty," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 41(4), pages 721-740, October.
    18. Aiken, Emily L. & Bedoya, Guadalupe & Blumenstock, Joshua E. & Coville, Aidan, 2023. "Program targeting with machine learning and mobile phone data: Evidence from an anti-poverty intervention in Afghanistan," Journal of Development Economics, Elsevier, vol. 161(C).
    19. Cowell, Frank A & Victoria-Feser, Maria-Pia, 1996. "Robustness Properties of Inequality Measures," Econometrica, Econometric Society, vol. 64(1), pages 77-101, January.
    20. Cowell, Frank A. & Victoria-Feser, Maria-Pia, 1996. "Poverty measurement with contaminated data: A robust approach," European Economic Review, Elsevier, vol. 40(9), pages 1761-1771, December.
    21. Christopher R. Bollinger & Barry T. Hirsch & Charles M. Hokayem & James P. Ziliak, 2019. "Trouble in the Tails? What We Know about Earnings Nonresponse 30 Years after Lillard, Smith, and Welch," Journal of Political Economy, University of Chicago Press, vol. 127(5), pages 2143-2185.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vladimir Hlasny & Paolo Verme, 2022. "The Impact of Top Incomes Biases on the Measurement of Inequality in the United States," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 84(4), pages 749-788, August.
    2. Frank Cowell & Emmanuel Flachaire, 2021. "Inequality Measurement: Methods and Data," Post-Print hal-03589066, HAL.
    3. Vladimir Hlasny & Paolo Verme, 2018. "Top Incomes and the Measurement of Inequality in Egypt," The World Bank Economic Review, World Bank, vol. 32(2), pages 428-455.
    4. Vladimir Hlasny & Paolo Verme, 2018. "Top Incomes and Inequality Measurement: A Comparative Analysis of Correction Methods Using the EU SILC Data," Econometrics, MDPI, vol. 6(2), pages 1-21, June.
    5. Arthur Charpentier & Emmanuel Flachaire, 2022. "Pareto models for top incomes and wealth," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 20(1), pages 1-25, March.
    6. Martin Ravallion, 2022. "Missing Top Income Recipients," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 20(1), pages 205-222, March.
    7. Mathias Silva, 2023. "Parametric models of income distributions integrating misreporting and non-response mechanisms," AMSE Working Papers 2311, Aix-Marseille School of Economics, France.
    8. Lendie Follett & Heath Henderson, 2022. "A hybrid approach to targeting social assistance," Papers 2201.01356, arXiv.org.
    9. Vladimir Hlasny, 2019. "Redistributive Impacts of Fiscal Policies in Mexico: Corrections for Top Income Measurement Problems," LIS Working papers 765, LIS Cross-National Data Center in Luxembourg.
    10. Lidia Ceriani & Paolo Verme, 2022. "Population Changes and the Measurement of Inequality," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 162(2), pages 549-575, July.
    11. Nora Lustig, 2019. "The “Missing Rich” in Household Surveys: Causes and Correction Approaches," Commitment to Equity (CEQ) Working Paper Series 75, Tulane University, Department of Economics.
    12. Nora Lustig, 2018. "Measuring the Distribution of Household Income, Consumption and Wealth: State of Play and Measurement Challenges," Working Papers 1801, Tulane University, Department of Economics.
    13. Brunori, Paolo & Salas Rojo, Pedro & Verne, Paolo, 2022. "Estimating inequality with missing incomes," LSE Research Online Documents on Economics 115932, London School of Economics and Political Science, LSE Library.
    14. Follett, Lendie & Henderson, Heath, 2023. "A hybrid approach to targeting social assistance," Journal of Development Economics, Elsevier, vol. 160(C).
    15. Arthur Charpentier & Emmanuel Flachaire, 2019. "Pareto Models for Top Incomes," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) hal-02145024, HAL.
    16. Bartels, Charlotte & Waldenström, Daniel, 2021. "Inequality and top incomes," GLO Discussion Paper Series 959, Global Labor Organization (GLO).
    17. Rafael Carranza & Marc Morgan & Brian Nolan, 2023. "Top Income Adjustments and Inequality: An Investigation of the EU‐SILC," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 69(3), pages 725-754, September.
    18. Nora Lustig, 2020. "The ``missing rich'' in household surveys: causes and correction approaches," Working Papers 520, ECINEQ, Society for the Study of Economic Inequality.
    19. Stephen P. Jenkins, 2022. "Top-income adjustments and official statistics on income distribution: the case of the UK," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 20(1), pages 151-168, March.
    20. Emily Aiken & Suzanne Bellue & Dean Karlan & Christopher R. Udry & Joshua Blumenstock, 2021. "Machine Learning and Mobile Phone Data Can Improve the Targeting of Humanitarian Assistance," NBER Working Papers 29070, National Bureau of Economic Research, Inc.

    More about this item

    Keywords

    Income modeling; Income Distributions; Poverty Predictions;
    All these keywords.

    JEL classification:

    • D31 - Microeconomics - - Distribution - - - Personal Income and Wealth Distribution
    • D63 - Microeconomics - - Welfare Economics - - - Equity, Justice, Inequality, and Other Normative Criteria and Measurement
    • E64 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - Incomes Policy; Price Policy
    • O15 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Economic Development: Human Resources; Human Development; Income Distribution; Migration

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inq:inqwps:ecineq2023-642. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Maria Ana Lugo (email available below). General contact details of provider: https://edirc.repec.org/data/ecineea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.