IDEAS home Printed from https://ideas.repec.org/a/wly/japmet/v40y2025i6p591-607.html

A Random Forest–Based Panel Data Approach for Program Evaluation

Author

Listed:
  • Guannan Liu
  • Wei Long
  • Xuehong Luo

Abstract

It is challenging to conduct controlled experiments to assess the impacts of social policy. To address this, past studies propose a panel data approach using factor models to estimate average treatment effects. The selection of control units is a critical step to balance the goodness of fit within‐sample with the posttreatment forecasting error when the number of observed potential control units is large. In this study, we propose using random forests, an ensemble learning method, which offers robustness and requires fewer candidate models compared to existing methods. We demonstrate that our approach effectively selects almost all relevant control units, and we provide asymptotic normality results under the null of no average treatment effect and significance tests for policy interventions. Extensive simulations confirm the method's superior performance. In the empirical studies, we showcase the usefulness of the method by evaluating the impact of Brexit on the United Kingdom's GDP growth and China's anti‐corruption campaign on the importation of luxury watches.

Suggested Citation

  • Guannan Liu & Wei Long & Xuehong Luo, 2025. "A Random Forest–Based Panel Data Approach for Program Evaluation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 40(6), pages 591-607, September.
  • Handle: RePEc:wly:japmet:v:40:y:2025:i:6:p:591-607
    DOI: 10.1002/jae.3123
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/jae.3123
    Download Restriction: no

    File URL: https://libkey.io/10.1002/jae.3123?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2021. "The Augmented Synthetic Control Method," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1789-1803, October.
    2. Bai, ChongEn & Li, Qi & Ouyang, Min, 2014. "Property taxes and home prices: A tale of two cities," Journal of Econometrics, Elsevier, vol. 180(1), pages 1-15.
    3. Du, Zaichao & Zhang, Lin, 2015. "Home-purchase restriction, property tax and housing price in China: A counterfactual analysis," Journal of Econometrics, Elsevier, vol. 188(2), pages 558-568.
    4. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(1), pages 1-19.
    5. Li, Kathleen T. & Bell, David R., 2017. "Estimation of average treatment effects with panel data: Asymptotic theory and implementation," Journal of Econometrics, Elsevier, vol. 197(1), pages 65-75.
    6. Michael W. Robbins & Jessica Saunders & Beau Kilmer, 2017. "A Framework for Synthetic Control Methods With High-Dimensional, Micro-Level Data: Evaluating a Neighborhood-Specific Crime Intervention," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(517), pages 109-126, January.
    7. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-660, November.
    8. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    9. Fujiki, Hiroshi & Hsiao, Cheng, 2015. "Disentangling the effects of multiple treatments—Measuring the net economic impact of the 1995 great Hanshin-Awaji earthquake," Journal of Econometrics, Elsevier, vol. 186(1), pages 66-73.
    10. Cheng Hsiao & Qiankun Zhou, 2019. "Panel parametric, semiparametric, and nonparametric construction of counterfactuals," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 34(4), pages 463-481, June.
    11. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," Review of Finance, European Finance Association, vol. 33(5), pages 2223-2273.
    12. María C. Latorre & Zoryana Olekseyuk & Hidemichi Yonezawa, 2020. "Trade and foreign direct investment‐related impacts of Brexit," The World Economy, Wiley Blackwell, vol. 43(1), pages 2-32, January.
    13. Bruno Ferman, 2021. "On the Properties of the Synthetic Control Estimator with Many Periods and Many Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1764-1772, October.
    14. Kathleen T. Li, 2020. "Statistical Inference for Average Treatment Effects Estimated by Synthetic Control Methods," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(532), pages 2068-2083, December.
    15. Susan Athey & Guido W. Imbens, 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models," Econometrica, Econometric Society, vol. 74(2), pages 431-497, March.
    16. George Kapetanios & Massimiliano Marcellino, 2009. "A parametric estimation method for dynamic factor models of large dimensions," Journal of Time Series Analysis, Wiley Blackwell, vol. 30(2), pages 208-238, March.
    17. Xiaoyan Li & Jiazhu Pan & Anchao Song, 2023. "Geometric ergodicity and conditional self‐weighted M‐estimator of a GRCAR(p) model with heavy‐tailed errors," Journal of Time Series Analysis, Wiley Blackwell, vol. 44(4), pages 418-436, July.
    18. Gu, Shihao & Kelly, Bryan & Xiu, Dacheng, 2021. "Autoencoder asset pricing models," Journal of Econometrics, Elsevier, vol. 222(1), pages 429-450.
    19. Jushan Bai, 2003. "Inferential Theory for Factor Models of Large Dimensions," Econometrica, Econometric Society, vol. 71(1), pages 135-171, January.
    20. Shihao Gu & Bryan Kelly & Dacheng Xiu, 2020. "Empirical Asset Pricing via Machine Learning," The Review of Financial Studies, Society for Financial Studies, vol. 33(5), pages 2223-2273.
    21. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    22. Shi, Zhentao & Huang, Jingyi, 2023. "Forward-selected panel data approach for program evaluation," Journal of Econometrics, Elsevier, vol. 234(2), pages 512-535.
    23. Alberto Abadie & Jérémy L’Hour, 2021. "A Penalized Synthetic Control Estimator for Disaggregated Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1817-1834, October.
    24. Xiao Ke & Cheng Hsiao, 2022. "Economic impact of the most drastic lockdown during COVID‐19 pandemic—The experience of Hubei, China," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(1), pages 187-209, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shi, Zhentao & Huang, Jingyi, 2023. "Forward-selected panel data approach for program evaluation," Journal of Econometrics, Elsevier, vol. 234(2), pages 512-535.
    2. Hongyi Jiang & Xingyu Li & Yan Shen & Qiankun Zhou, 2025. "Prediction Intervals of Panel Data Approach for Programme Evaluation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 40(6), pages 655-668, September.
    3. Li, Xingyu & Shen, Yan & Zhou, Qiankun, 2024. "Confidence intervals of treatment effects in panel data models with interactive fixed effects," Journal of Econometrics, Elsevier, vol. 240(1).
    4. Luya Wang & Jeffrey S. Racine & Qiaoyu Wang, 2025. "Bootstrap inference on a factor model based average treatment effects estimator," Econometric Reviews, Taylor & Francis Journals, vol. 44(1), pages 80-89, January.
    5. Chen, Qiang & Xiao, Zhijie & Yao, Qingsong, 2025. "Quantile control via random forest," Journal of Econometrics, Elsevier, vol. 249(PA).
    6. Andrii Melnychuk, 2024. "Synthetic Controls with spillover effects: A comparative study," Papers 2405.01645, arXiv.org.
    7. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    8. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    9. Zhentao Shi & Jingyi Huang, 2019. "Forward-Selected Panel Data Approach for Program Evaluation," Papers 1908.05894, arXiv.org, revised Apr 2021.
    10. Viviano, Davide & Bradic, Jelena, 2023. "Synthetic Learner: Model-free inference on treatments over time," Journal of Econometrics, Elsevier, vol. 234(2), pages 691-713.
    11. Chengwang Liao & Zhentao Shi & Yapeng Zheng, 2025. "A Relaxation Approach to Synthetic Control," Papers 2508.01793, arXiv.org, revised May 2026.
    12. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
    13. Qi Li & Wei Long, 2018. "Do parole abolition and Truth-in-Sentencing deter violent crimes in Virginia?," Empirical Economics, Springer, vol. 55(4), pages 2027-2045, December.
    14. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    15. Li, Kathleen T. & Bell, David R., 2017. "Estimation of average treatment effects with panel data: Asymptotic theory and implementation," Journal of Econometrics, Elsevier, vol. 197(1), pages 65-75.
    16. Callaway, Brantly & Karami, Sonia, 2023. "Treatment effects in interactive fixed effects models with a small number of time periods," Journal of Econometrics, Elsevier, vol. 233(1), pages 184-208.
    17. Kathleen T. Li, 2024. "Frontiers: A Simple Forward Difference-in-Differences Method," Marketing Science, INFORMS, vol. 43(2), pages 267-279, March.
    18. Sayoni Roychowdhury & Indrila Ganguly & Abhik Ghosh, 2024. "Robust estimation of average treatment effects from panel data," Statistical Papers, Springer, vol. 65(1), pages 139-179, February.
    19. Keegan Harris & Anish Agarwal & Chara Podimata & Zhiwei Steven Wu, 2022. "Strategyproof Decision-Making in Panel Data Settings and Beyond," Papers 2211.14236, arXiv.org, revised Dec 2023.
    20. Fu, Yifan & Zhong, Shiquan & Ling, Shuai & He, Zhengbing, 2024. "Closing the loophole of vehicle ownership restriction: The impact of non-local vehicle restriction on new vehicle registrations and air pollution," Transportation Research Part A: Policy and Practice, Elsevier, vol. 185(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:japmet:v:40:y:2025:i:6:p:591-607. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.interscience.wiley.com/jpages/0883-7252/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.