IDEAS home Printed from https://ideas.repec.org/p/cgd/wpaper/362.html
   My bibliography  Save this paper

We Just Ran Twenty-Three Million Queries of the World Bank's Website - Working Paper 362

Author

Listed:
  • Sarah Dykstra, Benjamin Dykstra, and Justin Sandefur

Abstract

Much of the data underlying global poverty and inequality estimates is not in the public domain, but can be accessed in small pieces using the World Bank’s PovcalNet online tool. To overcome these limitations and reproduce this database in a format more useful to researchers, we ran approximately 23 million queries of the World Bank’s web site, accessing only information that was already in the public domain. This web scraping exercise produced 10,000 points on the cumulative distribution of income or consumption from each of 942 surveys spanning 127 countries over the period 1977 to 2012. This short note describes our methodology, briefly discusses some of the relevant intellectual property issues, and illustrates the kind of calculations that are facilitated by this data set, including growth incidence curves and poverty rates using alternative PPP indices. The full data can be downloaded at www.cgdev.org/povcalnet.

Suggested Citation

  • Sarah Dykstra, Benjamin Dykstra, and Justin Sandefur, 2014. "We Just Ran Twenty-Three Million Queries of the World Bank's Website - Working Paper 362," Working Papers 362, Center for Global Development.
    • Handle: RePEc:cgd:wpaper:362
    as

    Download full text from publisher

    File URL: http://www.cgdev.org/sites/default/files/dykstra-sandefur-povcal-net-world-bank_1.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Branko Milanovic, 2002. "True World Income Distribution, 1988 and 1993: First Calculation Based on Household Surveys Alone," Economic Journal, Royal Economic Society, vol. 112(476), pages 51-92, January.
    2. Shaohua Chen & Martin Ravallion, 2010. "The Developing World is Poorer than We Thought, But No Less Successful in the Fight Against Poverty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(4), pages 1577-1625.
    3. Angus Deaton & Olivier Dupriez, 2011. "Purchasing Power Parity Exchange Rates for the Global Poor," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 137-166, April.
    4. repec:pri:rpdevs:presidential%20address%2017january%202010%20all is not listed on IDEAS
    5. Datt, Gaurav, 1998. "Computational tools for poverty measurement and analysis," FCND discussion papers 50, International Food Policy Research Institute (IFPRI).
    6. repec:pri:rpdevs:deaton_price_indexes_inequality_and_the_measurement_of_world_poverty_aer. is not listed on IDEAS
    7. Angus Deaton, 2010. "Price Indexes, Inequality, and the Measurement of World Poverty," American Economic Review, American Economic Association, vol. 100(1), pages 5-34, March.
    8. repec:pri:rpdevs:presidential%20address%2017january%202010%20all.pdf is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Moatsos, Michail, 2020. "Global Absolute Poverty: The Evolution of its Measurement," EconStor Preprints 216642, ZBW - Leibniz Information Centre for Economics.
    2. Kanbur, Ravi & Ortiz-Juarez, Eduardo & Sumner, Andy, 2024. "Is the era of declining global income inequality over?," Structural Change and Economic Dynamics, Elsevier, vol. 70(C), pages 45-55.
    3. Benfica, R. & Henderson, H., 2018. "IFAD RESEARCH SERIES 23 - The effect of the sectoral composition of economic growth on rural and urban poverty," IFAD Research Series 280072, International Fund for Agricultural Development (IFAD).
    4. Haroon Bhorat & Ravi Kanbur & Benjamin Stanwix, 2017. "Minimum Wages in Sub-Saharan Africa: A Primer," The World Bank Research Observer, World Bank, vol. 32(1), pages 21-74.
    5. Moatsos, Michail & Lazopoulos, Achillefs, 2021. "Global poverty: A first estimation of its uncertainty," World Development Perspectives, Elsevier, vol. 22(C).
    6. Bolch, Kimberly B. & Ceriani, Lidia & López-Calva, Luis F., 2022. "The arithmetics and politics of domestic resource mobilization for poverty eradication," World Development, Elsevier, vol. 149(C).
    7. Moatsos, Michail, 2018. "Scrapping PovcalNet's Distributions for Poverty Research," MPRA Paper 89872, University Library of Munich, Germany.
    8. Robert Inklaar & D. S. Prasada Rao, 2017. "Cross-Country Income Levels over Time: Did the Developing World Suddenly Become Much Richer?," American Economic Journal: Macroeconomics, American Economic Association, vol. 9(1), pages 265-290, January.
    9. Moatsos Michail, 2016. "Global Absolute Poverty: Behind the Veil of Dollars," Journal of Globalization and Development, De Gruyter, vol. 7(2), pages 1-28, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Camelia Minoiu & Sanjay Reddy, 2014. "Kernel density estimation on grouped data: the case of poverty assessment," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 12(2), pages 163-189, June.
    2. Facundo Alvaredo & Leonardo Gasparini, 2013. "Recent Trends in Inequality and Poverty in Developing Countries," CEDLAS, Working Papers 0151, CEDLAS, Universidad Nacional de La Plata.
    3. Segal, Paul, 2011. "Resource Rents, Redistribution, and Halving Global Poverty: The Resource Dividend," World Development, Elsevier, vol. 39(4), pages 475-489, April.
    4. Masato Okamoto, 2014. "Interpolating the Lorenz Curve: Methods to Preserve Shape and Remain Consistent with the Concentration Curves for Components," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 60(2), pages 349-384, June.
    5. Cockburn, John & Duclos, Jean-Yves & Zabsonré, Agnès, 2014. "Is global social welfare increasing? A critical-level enquiry," Journal of Public Economics, Elsevier, vol. 118(C), pages 151-162.
    6. Lahoti Rahul & Jayadev Arjun & Reddy Sanjay, 2016. "The Global Consumption and Income Project (GCIP): An Overview," Journal of Globalization and Development, De Gruyter, vol. 7(1), pages 61-108, June.
    7. Peter Edward & Andy Sumner, 2015. "New estimates of global poverty and inequality: How much difference do price data," Working Papers 365, ECINEQ, Society for the Study of Economic Inequality.
    8. Pinkovskiy, Maxim L., 2013. "World welfare is rising: Estimation using nonparametric bounds on welfare measures," Journal of Public Economics, Elsevier, vol. 97(C), pages 176-195.
    9. Alan Heston & D.S. Prasada Rao, 2021. "Price Levels, Size, Distribution and Growth of the World Economy: Insights from recent International Comparisons of Prices and Real Product," PIER Working Paper Archive 21-013, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    10. Moatsos Michail, 2016. "Global Absolute Poverty: Behind the Veil of Dollars," Journal of Globalization and Development, De Gruyter, vol. 7(2), pages 1-28, December.
    11. Robert C. Allen, 2017. "Absolute Poverty: When Necessity Displaces Desire," American Economic Review, American Economic Association, vol. 107(12), pages 3690-3721, December.
    12. Clare Leaver & Paul Segal, 2014. "The Global Distribution of Income," Economics Series Working Papers 714, University of Oxford, Department of Economics.
    13. Natalie Chun & Rana Hasan & Muhammad Habibur Rahman & Mehmet Ali Ulubaşoğlu, 2017. "The Role of Middle Class in Economic Development: What Do Cross-Country Data Show?," Review of Development Economics, Wiley Blackwell, vol. 21(2), pages 404-424, May.
    14. Sanjay Reddy & Rahul Lahoti & Arjun Jayadev, 2015. "The global consumption and income project: An introduction and preliminary findings," WIDER Working Paper Series 003, World Institute for Development Economic Research (UNU-WIDER).
    15. Mthuli Ncube & Zuzana Brixiova & Zorobabel Bicaba, 2014. "Can Dreams Come True? Eliminating Extreme Poverty In Africa By 2030," William Davidson Institute Working Papers Series wp1076, William Davidson Institute at the University of Michigan.
    16. Sudhir Anand & Paul Segal, 2008. "What Do We Know about Global Income Inequality?," Journal of Economic Literature, American Economic Association, vol. 46(1), pages 57-94, March.
    17. Branko Milanovic, 2012. "Global inequality recalculated and updated: the effect of new PPP estimates on global inequality and 2005 estimates," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 10(1), pages 1-18, March.
    18. Dhongde, Shatakshee & Minoiu, Camelia, 2013. "Global Poverty Estimates: A Sensitivity Analysis," World Development, Elsevier, vol. 44(C), pages 1-13.
    19. Duangkamon Chotikapanich & William Griffiths & Wasana Karunarathne & D.S. Prasada Rao, 2013. "Calculating Poverty Measures from the Generalised Beta Income Distribution," The Economic Record, The Economic Society of Australia, vol. 89, pages 48-66, June.
    20. Nobel Prize Committee, 2015. "Consumption, Poverty, and Welfare," Nobel Prize in Economics documents 2015-2, Nobel Prize Committee.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • D31 - Microeconomics - - Distribution - - - Personal Income and Wealth Distribution
    • I32 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Measurement and Analysis of Poverty
    • O57 - Economic Development, Innovation, Technological Change, and Growth - - Economywide Country Studies - - - Comparative Studies of Countries

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cgd:wpaper:362. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Publications Manager (email available below). General contact details of provider: https://edirc.repec.org/data/cgdevus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.