IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0244317.html

Applying machine learning and geolocation techniques to social media data (Twitter) to develop a resource for urban planning

Author

Listed:
  • Sveta Milusheva
  • Robert Marty
  • Guadalupe Bedoya
  • Sarah Williams
  • Elizabeth Resor
  • Arianna Legovini

Abstract

With all the recent attention focused on big data, it is easy to overlook that basic vital statistics remain difficult to obtain in most of the world. What makes this frustrating is that private companies hold potentially useful data, but it is not accessible by the people who can use it to track poverty, reduce disease, or build urban infrastructure. This project set out to test whether we can transform an openly available dataset (Twitter) into a resource for urban planning and development. We test our hypothesis by creating road traffic crash location data, which is scarce in most resource-poor environments but essential for addressing the number one cause of mortality for children over five and young adults. The research project scraped 874,588 traffic related tweets in Nairobi, Kenya, applied a machine learning model to capture the occurrence of a crash, and developed an improved geoparsing algorithm to identify its location. We geolocate 32,991 crash reports in Twitter for 2012–2020 and cluster them into 22,872 unique crashes during this period. For a subset of crashes reported on Twitter, a motorcycle delivery service was dispatched in real-time to verify the crash and its location; the results show 92% accuracy. To our knowledge this is the first geolocated dataset of crashes for the city and allowed us to produce the first crash map for Nairobi. Using a spatial clustering algorithm, we are able to locate portions of the road network (

Suggested Citation

  • Sveta Milusheva & Robert Marty & Guadalupe Bedoya & Sarah Williams & Elizabeth Resor & Arianna Legovini, 2021. "Applying machine learning and geolocation techniques to social media data (Twitter) to develop a resource for urban planning," PLOS ONE, Public Library of Science, vol. 16(2), pages 1-12, February.
    • Handle: RePEc:plo:pone00:0244317
    DOI: 10.1371/journal.pone.0244317
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0244317
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0244317&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0244317?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Serajuddin,Umar & Uematsu,Hiroki & Wieser,Christina & Yoshida,Nobuo & Dabalen,Andrew L., 2015. "Data deprivation : another deprivation to end," Policy Research Working Paper Series 7252, The World Bank.
    2. Bernd Resch & Anja Summa & Peter Zeile & Michael Strube, 2016. "Citizen-Centric Urban Planning through Extracting Emotion Information from Twitter in an Interdisciplinary Space-Time-Linguistics Algorithm," Urban Planning, Cogitatio Press, vol. 1(2), pages 114-127.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Steffen Knoblauch & Simon Groß & Sven Lautenbach & Antonio Augusto de Aragão Rocha & Marta C González & Bernd Resch & Dorian Arifi & Thomas Jänisch & Ivonne Morales & Alexander Zipf, 2025. "Long-term validation of inner-urban mobility metrics derived from Twitter/X," Environment and Planning B, , vol. 52(6), pages 1310-1334, July.
    2. Gaukhar Aidarkhanova & Chingiz Zhumagulov & Gulnara Nyussupova & Veronika Kholina, 2025. "Assessing the Impact of Demographic Growth on the Educational Infrastructure for Sustainable Regional Development: Forecasting Demand for Preschool and Primary School Enrollment in Kazakhstan," Sustainability, MDPI, vol. 17(9), pages 1-18, May.
    3. Yijiang Zhao & Jing Luo & Daoan Zhang & Yizhi Liu & Zhuhua Liao, 2025. "A semi-supervised Chinese toponym recognition methods combining active learning and self-training," Journal of Geographical Systems, Springer, vol. 27(4), pages 555-583, October.
    4. Jin, Ting & Liang, Feiyan & Dong, Xiaoqi & Cao, Xiaojuan, 2023. "Research on land resource management integrated with support vector machine —Based on the perspective of green innovation," Resources Policy, Elsevier, vol. 86(PB).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yong Gao & Yuanyuan Chen & Lan Mu & Shize Gong & Pengcheng Zhang & Yu Liu, 2022. "Measuring urban sentiments from social media data: a dual-polarity metric approach," Journal of Geographical Systems, Springer, vol. 24(2), pages 199-221, April.
    2. Dang, Hai-Anh H & Carletto, Calogero, 2022. "Recall Bias Revisited: Measure Farm Labor Using Mixed-Mode Surveys and Multiple Imputation," IZA Discussion Papers 14997, IZA Network @ LISER.
    3. Huang, Wei & Luo, Mi & Ta, Yuqi & Wang, Boxian, 2024. "Land expropriation, household behaviors, and health outcomes: Evidence from China," Journal of Development Economics, Elsevier, vol. 171(C).
    4. Paul Makdissi & Walid Marrouch & Myra Yazbeck, 2025. "Monitoring Poverty in a Data‐Deprived Environment: The Case of Lebanon," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 71(1), February.
    5. Arena, Marika & Azzone, Giovanni & Dell’Agostino, Laura & Scotti, Francesco, 2022. "Precision policies and local content targets in resource-rich developing countries: The case of the oil and gas sector in Mozambique," Resources Policy, Elsevier, vol. 76(C).
    6. Cuesta, Jose & Chagalj, Cristian, 2019. "Measuring poverty with administrative data in data deprived contexts: The case of Nicaragua," Economics Letters, Elsevier, vol. 183(C), pages 1-1.
    7. Francis Rathinam & Sayak Khatua & Zeba Siddiqui & Manya Malik & Pallavi Duggal & Samantha Watson & Xavier Vollenweider, 2021. "Using big data for evaluating development outcomes: A systematic map," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(3), September.
    8. Dang, Hai-Anh H. & Serajuddin, Umar, 2020. "Tracking the sustainable development goals: Emerging measurement challenges and further reflections," World Development, Elsevier, vol. 127(C).
    9. Francisco H. G. Ferreira & Shaohua Chen & Andrew Dabalen & Yuri Dikhanov & Nada Hamadeh & Dean Jolliffe & Ambar Narayan & Espen Beer Prydz & Ana Revenga & Prem Sangraula & Umar Serajuddin & Nobuo Yosh, 2016. "A global count of the extreme poor in 2012: data issues, methodology and initial results," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 14(2), pages 141-172, June.
    10. Hai‐Anh H. Dang & Talip Kilic & Kseniya Abanokova & Calogero Carletto, 2025. "Poverty Imputation in Contexts Without Consumption Data: A Revisit With Further Refinements," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 71(1), February.
    11. Hai‐Anh H. Dang & Peter F. Lanjouw, 2023. "Measuring Poverty Dynamics with Synthetic Panels Based on Repeated Cross Sections," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 85(3), pages 599-622, June.
    12. Anna Kovacs-Gyori & Alina Ristea & Clemens Havas & Bernd Resch & Pablo Cabrera-Barona, 2018. "#London2012: Towards Citizen-Contributed Urban Planning Through Sentiment Analysis of Twitter Data," Urban Planning, Cogitatio Press, vol. 3(1), pages 75-99.
    13. Hai‐Anh Dang & Dean Jolliffe & Calogero Carletto, 2019. "Data Gaps, Data Incomparability, And Data Imputation: A Review Of Poverty Measurement Methods For Data‐Scarce Environments," Journal of Economic Surveys, Wiley Blackwell, vol. 33(3), pages 757-797, July.
    14. Atamanov, Aziz & Tandon, Sharad & Lopez-Acevedo, Gladys & Vergara Bahena, Mexico Alberto, 2020. "Measuring Monetary Poverty in the Middle East and North Africa (MENA) Region: Data Gaps and Different Options to Address Them," IZA Discussion Papers 13363, IZA Network @ LISER.
    15. Fernando Santa & Roberto Henriques & Joaquín Torres-Sospedra & Edzer Pebesma, 2019. "A Statistical Approach for Studying the Spatio-Temporal Distribution of Geolocated Tweets in Urban Environments," Sustainability, MDPI, vol. 11(3), pages 1-29, January.
    16. Leonardo Lucchetti & Paul Corral & Andrés Ham & Santiago Garriga, 2025. "An application of LASSO and multiple imputation techniques to income dynamics with cross‐sectional data," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 71(1), February.
    17. Ghasem Javadi & Mohammad Taleai, 2020. "Integration of User Generated Geo-contents and Official Data to Assess Quality of Life in Intra-national Level," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 152(1), pages 205-235, November.
    18. Ourania Kounadi & Bernd Resch & Andreas Petutschnig, 2018. "Privacy Threats and Protection Recommendations for the Use of Geosocial Network Data in Research," Social Sciences, MDPI, vol. 7(10), pages 1-17, October.
    19. Dang,Hai-Anh H., 2018. "To impute or not to impute ? a review of alternative poverty estimation methods in the context of unavailable consumption data," Policy Research Working Paper Series 8403, The World Bank.
    20. Kim, Daehwan & Seo, Ducksu & Kwon, Youngsang, 2021. "Novel trends in SNS customers in food and beverage patronage: An empirical study of metropolitan cities in South Korea," Land Use Policy, Elsevier, vol. 101(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0244317. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.