IDEAS home Printed from https://ideas.repec.org/p/inq/inqwps/ecineq2023-642.html

Predicting Poverty with Missing Incomes

Author

Listed:
  • Paolo Verme

    (World Bank)

Abstract

Poverty prediction models are used by economists to address missing data issues in a variety of contexts such as poverty profiling, targeting with proxy-means tests, cross-survey imputations such as poverty mapping, or vulnerability analyses. Based on the models used by this literature, this paper conducts an experiment by artificially corrupting data with different patterns and shares of missing incomes. It then compares the capacity of classic econometric and machine learning models to predict poverty under these different scenarios. It finds that the quality of predictions and the choice of the optimal prediction model are dependent on the distribution of observed and unobserved incomes, the poverty line, the choice of objective function and policy preferences, and various other modeling choices. Logistic and random forest models are found to be more robust than other models to variations in these features, but no model invariably outperforms all others. The paper concludes with some reflections on the use of these models for predicting poverty.

Suggested Citation

  • Paolo Verme, 2023. "Predicting Poverty with Missing Incomes," Working Papers 642, ECINEQ, Society for the Study of Economic Inequality.
  • Handle: RePEc:inq:inqwps:ecineq2023-642
    as

    Download full text from publisher

    File URL: http://www.ecineq.org/milano/WP/ECINEQ2023-642.pdf
    File Function: First version, 2023
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hai-Anh H. Dang & Paolo Verme, 2023. "Estimating poverty for refugees in data-scarce contexts: an application of cross-survey imputation," Journal of Population Economics, Springer;European Society for Population Economics, vol. 36(2), pages 653-679, April.
    2. Wodon, Quentin T., 1997. "Targeting the poor using ROC curves," World Development, Elsevier, vol. 25(12), pages 2083-2092, December.
    3. Cowell, Frank A. & Victoria-Feser, Maria-Pia, 1996. "Poverty measurement with contaminated data: A robust approach," European Economic Review, Elsevier, vol. 40(9), pages 1761-1771, December.
    4. John Gibson, 2019. "Are You Estimating the Right Thing? An Editor Reflects," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 41(3), pages 329-350.
    5. Korinek, Anton & Mistiaen, Johan A. & Ravallion, Martin, 2007. "An econometric method of correcting for unit nonresponse bias in surveys," Journal of Econometrics, Elsevier, vol. 136(1), pages 213-235, January.
    6. Hai‐Anh Dang & Dean Jolliffe & Calogero Carletto, 2019. "Data Gaps, Data Incomparability, And Data Imputation: A Review Of Poverty Measurement Methods For Data‐Scarce Environments," Journal of Economic Surveys, Wiley Blackwell, vol. 33(3), pages 757-797, July.
    7. Alessandro Tarozzi & Angus Deaton, 2009. "Using Census and Survey Data to Estimate Poverty and Inequality for Small Areas," The Review of Economics and Statistics, MIT Press, vol. 91(4), pages 773-792, November.
    8. David Coady & Margaret Grosh & John Hoddinott, 2004. "Targeting of Transfers in Developing Countries : Review of Lessons and Experience," World Bank Publications - Books, The World Bank Group, number 14902, April.
    9. Morduch, Jonathan, 1994. "Poverty and Vulnerability," American Economic Review, American Economic Association, vol. 84(2), pages 221-225, May.
    10. Cesar Calvo & Stefan Dercon, 2013. "Vulnerability to individual and aggregate poverty," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 41(4), pages 721-740, October.
    11. Verme, Paolo & Gigliarano, Chiara, 2019. "Optimal targeting under budget constraints in a humanitarian context," World Development, Elsevier, vol. 119(C), pages 224-233.
    12. Christopher R. Bollinger & Barry T. Hirsch & Charles M. Hokayem & James P. Ziliak, 2019. "Trouble in the Tails? What We Know about Earnings Nonresponse 30 Years after Lillard, Smith, and Welch," Journal of Political Economy, University of Chicago Press, vol. 127(5), pages 2143-2185.
    13. Anthony B. Atkinson & Thomas Piketty & Emmanuel Saez, 2011. "Top Incomes in the Long Run of History," Journal of Economic Literature, American Economic Association, vol. 49(1), pages 3-71, March.
    14. Lillard, Lee & Smith, James P & Welch, Finis, 1986. "What Do We Really Know about Wages? The Importance of Nonreporting and Census Imputation," Journal of Political Economy, University of Chicago Press, vol. 94(3), pages 489-506, June.
    15. Glewwe, Paul, 1991. "Investigating the determinants of household welfare in Cote d'Ivoire," Journal of Development Economics, Elsevier, vol. 35(2), pages 307-337, April.
    16. Anton Korinek & Johan Mistiaen & Martin Ravallion, 2006. "Survey nonresponse and the distribution of income," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 4(1), pages 33-55, April.
    17. Emily Aiken & Suzanne Bellue & Dean Karlan & Chris Udry & Joshua E. Blumenstock, 2022. "Machine learning and phone data can improve targeting of humanitarian aid," Nature, Nature, vol. 603(7903), pages 864-870, March.
    18. Brown, Caitlin & Ravallion, Martin & van de Walle, Dominique, 2018. "A poor means test? Econometric targeting in Africa," Journal of Development Economics, Elsevier, vol. 134(C), pages 109-124.
    19. Stephen P. Jenkins, 2017. "Pareto Models, Top Incomes and Recent Trends in UK Income Inequality," Economica, London School of Economics and Political Science, vol. 84(334), pages 261-289, April.
    20. Aiken, Emily L. & Bedoya, Guadalupe & Blumenstock, Joshua E. & Coville, Aidan, 2023. "Program targeting with machine learning and mobile phone data: Evidence from an anti-poverty intervention in Afghanistan," Journal of Development Economics, Elsevier, vol. 161(C).
    21. Linden McBride & Austin Nichols, 2018. "Retooling Poverty Targeting Using Out-of-Sample Validation and Machine Learning," The World Bank Economic Review, World Bank, vol. 32(3), pages 531-550.
    22. Baker, Judy L. & Grosh, Margaret E., 1994. "Poverty reduction through geographic targeting: How well does it work?," World Development, Elsevier, vol. 22(7), pages 983-995, July.
    23. Cowell, Frank A & Victoria-Feser, Maria-Pia, 1996. "Robustness Properties of Inequality Measures," Econometrica, Econometric Society, vol. 64(1), pages 77-101, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paolo Verme, 2025. "Predicting Poverty," The World Bank Economic Review, World Bank, vol. 39(4), pages 1039-1064.
    2. Vladimir Hlasny & Paolo Verme, 2022. "The Impact of Top Incomes Biases on the Measurement of Inequality in the United States," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 84(4), pages 749-788, August.
    3. Katy Bergstrom & William Dodds & Nicholas Lacoste & Juan Rios, 2025. "Estimating the Welfare Cost of Labor Supply Frictions," Working Papers 2503, Tulane University, Department of Economics.
    4. Ferreira, Francisco H. G. & Brunori, Paolo, 2024. "Inherited inequality, meritocracy, and the purpose of economic growth," LSE Research Online Documents on Economics 126263, London School of Economics and Political Science, LSE Library.
    5. Frank Cowell & Emmanuel Flachaire, 2021. "Inequality Measurement: Methods and Data," Post-Print hal-03589066, HAL.
    6. Vladimir Hlasny & Paolo Verme, 2018. "Top Incomes and Inequality Measurement: A Comparative Analysis of Correction Methods Using the EU SILC Data," Econometrics, MDPI, vol. 6(2), pages 1-21, June.
    7. Arthur Charpentier & Emmanuel Flachaire, 2022. "Pareto models for top incomes and wealth," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 20(1), pages 1-25, March.
    8. Nora Lustig & Andrea Vigorito, 2025. "The "Missing Rich" in Household Surveys: Causes and Correction Approaches. Extended Version with Technical Appendixes," Documentos de Trabajo (working papers) 25-03, Instituto de Economía - IECON.
    9. Lustig, Nora & Vigorito, Andrea, 2025. "The “Missing Rich” in Household Surveys: Causes and Correction Approaches," SocArXiv 97ng6_v1, Center for Open Science.
    10. Vladimir Hlasny & Paolo Verme, 2018. "Top Incomes and the Measurement of Inequality in Egypt," The World Bank Economic Review, World Bank, vol. 32(2), pages 428-455.
    11. Mathias Silva, 2023. "Parametric models of income distributions integrating misreporting and non-response mechanisms," AMSE Working Papers 2311, Aix-Marseille School of Economics, France.
    12. Emily Aiken & Anik Ashraf & Joshua Blumenstock & Raymond Guiteras & Ahmed Mushfiq Mobarak, 2025. "Scalable Targeting of Social Protection: When Do Algorithms Out-Perform Surveys and Community Knowledge?," NBER Working Papers 33919, National Bureau of Economic Research, Inc.
    13. Rafael Carranza & Marc Morgan & Brian Nolan, 2023. "Top Income Adjustments and Inequality: An Investigation of the EU‐SILC," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 69(3), pages 725-754, September.
    14. Nora Lustig, 2019. "The “Missing Rich” in Household Surveys: Causes and Correction Approaches," Commitment to Equity (CEQ) Working Paper Series 75, Tulane University, Department of Economics.
    15. Nora Lustig, 2018. "Measuring the Distribution of Household Income, Consumption and Wealth: State of Play and Measurement Challenges," Working Papers 1801, Tulane University, Department of Economics.
    16. Brunori, Paolo & Salas-Rojo, Pedro & Verme, Paolo, 2022. "Estimating Inequality with Missing Incomes," GLO Discussion Paper Series 1138, Global Labor Organization (GLO).
    17. Baez, Javier E. & Kshirsagar, Varun & Skoufias, Emmanuel, 2024. "Drought-sensitive targeting and child growth faltering in Southern Africa," World Development, Elsevier, vol. 182(C).
    18. Lyons, Angela C. & Montoya Castano, Alejandro & Kass-Hanna, Josephine & Zhang, Yifang & Soliman, Aiman, 2025. "A machine learning approach to assessing multidimensional poverty and targeting assistance among forcibly displaced populations," World Development, Elsevier, vol. 192(C).
    19. repec:osf:socarx:j23pn_v1 is not listed on IDEAS
    20. Isabella S. Smythe & Joshua E. Blumenstock, 2022. "Geographic microtargeting of social assistance with high-resolution poverty maps," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 119(32), pages 2120025119-, August.
    21. Follett, Lendie & Henderson, Heath, 2023. "A hybrid approach to targeting social assistance," Journal of Development Economics, Elsevier, vol. 160(C).

    More about this item

    Keywords

    ;
    ;
    ;

    JEL classification:

    • D31 - Microeconomics - - Distribution - - - Personal Income and Wealth Distribution
    • D63 - Microeconomics - - Welfare Economics - - - Equity, Justice, Inequality, and Other Normative Criteria and Measurement
    • E64 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - Incomes Policy; Price Policy
    • O15 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Economic Development: Human Resources; Human Development; Income Distribution; Migration

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inq:inqwps:ecineq2023-642. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Maria Ana Lugo The email address of this maintainer does not seem to be valid anymore. Please ask Maria Ana Lugo to update the entry or send us the correct address (email available below). General contact details of provider: https://edirc.repec.org/data/ecineea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.