IDEAS home Printed from https://ideas.repec.org/a/kap/jgeosy/v26y2024i4d10.1007_s10109-023-00413-0.html
   My bibliography  Save this article

A structured comparison of causal machine learning methods to assess heterogeneous treatment effects in spatial data

Author

Listed:
  • Kevin Credit

    (Maynooth University)

  • Matthew Lehnert

    (Satelytics)

Abstract

The development of the “causal” forest by Wager and Athey (J Am Stat Assoc 113(523): 1228–1242, 2018) represents a significant advance in the area of explanatory/causal machine learning. However, this approach has not yet been widely applied to geographically referenced data, which present some unique issues: the random split of the test and training sets in the typical causal forest design fractures the spatial fabric of geographic data. To help solve this issue, we use a simulated dataset with known properties for average treatment effects and conditional average treatment effects to compare the performance of CF models across different definitions of the test/train split. We also develop a new “spatial” T-learner that can be implemented using predictive methods like random forest to provide estimates of heterogeneous treatment effects across all units. Our results show that all of the machine learning models outperform traditional ordinary least squares regression at identifying the true average treatment effect, but are not significantly different from one another. We then apply the preferred causal forest model in the context of analysing the treatment effect of the construction of the Valley Metro light rail (tram) system on on-road CO2 emissions per capita at the block group level in Maricopa County, Arizona, and find that the neighbourhoods most likely to benefit from treatment are those with higher pre-treatment proportions of transit and pedestrian commuting and lower proportions of auto commuting.

Suggested Citation

  • Kevin Credit & Matthew Lehnert, 2024. "A structured comparison of causal machine learning methods to assess heterogeneous treatment effects in spatial data," Journal of Geographical Systems, Springer, vol. 26(4), pages 483-510, October.
  • Handle: RePEc:kap:jgeosy:v:26:y:2024:i:4:d:10.1007_s10109-023-00413-0
    DOI: 10.1007/s10109-023-00413-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10109-023-00413-0
    File Function: Abstract
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s10109-023-00413-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Glaeser, Edward L. & Kahn, Matthew E., 2010. "The greenness of cities: Carbon dioxide emissions and urban development," Journal of Urban Economics, Elsevier, vol. 67(3), pages 404-418, May.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. Guerra, Erick & Cervero, Robert & Tischler, Daniel, 2011. "The Half-Mile Circle: Does It Represent Transit Station Catchments?," University of California Transportation Center, Working Papers qt0d84c2f4, University of California Transportation Center.
    4. Guerra, Erick & Cervero, Robert & Tischler, Daniel, 2011. "The Half-Mile Circle: Does It Best Represent Transit Station Catchments?," University of California Transportation Center, Working Papers qt9jd6r1t9, University of California Transportation Center.
    5. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    6. Anthony Strittmatter, 2018. "What Is the Value Added by Using Causal Machine Learning Methods in a Welfare Experiment Evaluation?," Papers 1812.06533, arXiv.org, revised Dec 2021.
    7. Jonathan M.V. Davis & Sara B. Heller, 2020. "Rethinking the Benefits of Youth Employment Programs: The Heterogeneous Effects of Summer Jobs," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 664-677, October.
    8. Reinhold Kosfeld & Timo Mitze & Johannes Rode & Klaus Wälde, 2021. "The Covid‐19 containment effects of public health measures: A spatial difference‐in‐differences approach," Journal of Regional Science, Wiley Blackwell, vol. 61(4), pages 799-825, September.
    9. Hoffman, Ian & Mast, Evan, 2019. "Heterogeneity in the effect of federal spending on local crime: Evidence from causal forests," Regional Science and Urban Economics, Elsevier, vol. 78(C).
    10. Reid Ewing & Shima Hamidi, 2014. "Longitudinal Analysis of Transit's Land Use Multiplier in Portland (OR)," Journal of the American Planning Association, Taylor & Francis Journals, vol. 80(2), pages 123-137, April.
    11. Kosfeld, Reinhold & Mitze, Timo & Rode, Johannes & Wälde, Klaus, 2021. "The Covid-19 containment effects of public health measures - A spatial difference-in-differences approach," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 126127, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    12. Guerra, Erick & Cervero, Robert & Tischler, Daniel, 2011. "The Half-Mile Circle: Does It Best Represent Transit Station Catchments?," University of California Transportation Center, Working Papers qt68r764df, University of California Transportation Center.
    13. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    14. Kevin Credit, 2018. "Transit-oriented economic development: The impact of light rail on new business starts in the Phoenix, AZ Region, USA," Urban Studies, Urban Studies Journal Limited, vol. 55(13), pages 2838-2862, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mark Kattenberg & Bas Scheer & Jurre Thiel, 2023. "Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences," CPB Discussion Paper 452, CPB Netherlands Bureau for Economic Policy Analysis.
    2. Cervero, Robert & Guerra, Erick, 2011. "Urban Densities and Transit: A Multi-dimensional Perspective," Institute of Transportation Studies, Research Reports, Working Papers, Proceedings qt3mb598qr, Institute of Transportation Studies, UC Berkeley.
    3. Credit, Kevin, 2025. "The Impact of Light Rail Construction on Regional On-Road CO2 Emissions Per Capita," OSF Preprints euyj6_v2, Center for Open Science.
    4. Vale, David S. & Viana, Cláudia M. & Pereira, Mauro, 2018. "The extended node-place model at the local scale: Evaluating the integration of land use and transport for Lisbon's subway network," Journal of Transport Geography, Elsevier, vol. 69(C), pages 282-293.
    5. Jyothi Chava & Peter Newman, 2016. "Stakeholder Deliberation on Developing Affordable Housing Strategies: Towards Inclusive and Sustainable Transit-Oriented Developments," Sustainability, MDPI, vol. 8(10), pages 1-21, October.
    6. Dohyung Kim & Yongjin Ahn & Simon Choi & Kwangkoo Kim, 2016. "Sustainable Mobility: Longitudinal Analysis of Built Environment on Transit Ridership," Sustainability, MDPI, vol. 8(10), pages 1-14, October.
    7. Heilmann, Kilian, 2018. "Transit access and neighborhood segregation. Evidence from the Dallas light rail system," Regional Science and Urban Economics, Elsevier, vol. 73(C), pages 237-250.
    8. Aston, Laura & Currie, Graham & Pavkova, Katerina, 2016. "Does transit mode influence the transit-orientation of urban development? – An empirical study," Journal of Transport Geography, Elsevier, vol. 55(C), pages 83-91.
    9. Jinbao Zhao & Wei Deng & Yan Song & Yueran Zhu, 2014. "Analysis of Metro ridership at station level and station-to-station level in Nanjing: an approach based on direct demand models," Transportation, Springer, vol. 41(1), pages 133-155, January.
    10. Aston, Laura & Currie, Graham & Kamruzzaman, Md. & Delbosc, Alexa & Teller, David, 2020. "Study design impacts on built environment and transit use research," Journal of Transport Geography, Elsevier, vol. 82(C).
    11. Li, Zekun & Han, Zixuan & Xin, Jing & Luo, Xin & Su, Shiliang & Weng, Min, 2019. "Transit oriented development among metro station areas in Shanghai, China: Variations, typology, optimization and implications for land use planning," Land Use Policy, Elsevier, vol. 82(C), pages 269-282.
    12. Mishra, Sabyasachee & Welch, Timothy F. & Jha, Manoj K., 2012. "Performance indicators for public transit connectivity in multi-modal transportation networks," Transportation Research Part A: Policy and Practice, Elsevier, vol. 46(7), pages 1066-1085.
    13. Singh, Yamini Jain & Lukman, Azhari & Flacke, Johannes & Zuidgeest, Mark & Van Maarseveen, M.F.A.M., 2017. "Measuring TOD around transit nodes - Towards TOD policy," Transport Policy, Elsevier, vol. 56(C), pages 96-111.
    14. Hiroyuki Egami & Md. Shafiur Rahman & Tsuyoshi Yamamoto & Chihiro Egami & Takahisa Wakabayashi, 2024. "Causal effect of video gaming on mental well-being in Japan 2020–2022," Nature Human Behaviour, Nature, vol. 8(10), pages 1943-1956, October.
    15. Kevin Credit, 2018. "Transit-oriented economic development: The impact of light rail on new business starts in the Phoenix, AZ Region, USA," Urban Studies, Urban Studies Journal Limited, vol. 55(13), pages 2838-2862, October.
    16. Arlie Adkins & Andrew Sanderford & Gary Pivo, 2017. "How Location Efficient Is LIHTC? Measuring and Explaining State-Level Achievement," Housing Policy Debate, Taylor & Francis Journals, vol. 27(3), pages 335-355, May.
    17. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    18. Shonosuke Sugasawa & Hisashi Noma, 2021. "Efficient screening of predictive biomarkers for individual treatment selection," Biometrics, The International Biometric Society, vol. 77(1), pages 249-257, March.
    19. Samuel de Haas & Georg Goetz & Sven Heim, 2021. "Measuring the effects of COVID-19-related night curfews: Empirical evidence from Germany," MAGKS Papers on Economics 202118, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    20. Michael C Knaus & Michael Lechner & Anthony Strittmatter, 2021. "Machine learning estimation of heterogeneous causal effects: Empirical Monte Carlo evidence," The Econometrics Journal, Royal Economic Society, vol. 24(1), pages 134-161.

    More about this item

    Keywords

    Causal forest; Heterogeneous treatment effects; Machine learning; Causal inference; Spatial; CO2 emissions; Transit;
    All these keywords.

    JEL classification:

    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C54 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Quantitative Policy Modeling
    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:jgeosy:v:26:y:2024:i:4:d:10.1007_s10109-023-00413-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.