IDEAS home Printed from https://ideas.repec.org/p/zbw/esprep/177390.html

Rethinking Policy Evaluation – Do Simple Neural Nets Bear Comparison with Synthetic Control Method?

Author

Listed:
  • Steinkraus, Arne

Abstract

With the advent of big data in economics machine learning algorithms become more and more appealing to economists. Despite some attempts of establishing artificial neural networks in in the early 1990s, only little is known about their ability of estimating causal effects in policy evaluation. We employ a simple forecasting neural network to analyze the effect of the construction of the Oresund bridge on the local economy. The outcome is compared to the causal effect estimated by the proven Synthetic Control Method. Our results suggest that – especially in so-called prediction policy problems – neural nets may outperform traditional approaches.

Suggested Citation

  • Steinkraus, Arne, 2018. "Rethinking Policy Evaluation – Do Simple Neural Nets Bear Comparison with Synthetic Control Method?," EconStor Preprints 177390, ZBW - Leibniz Information Centre for Economics.
    • Handle: RePEc:zbw:esprep:177390
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/177390/1/Rethinking%20Policy%20Evaluation2.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jon Kleinberg & Jens Ludwig & Sendhil Mullainathan & Ziad Obermeyer, 2015. "Prediction Policy Problems," American Economic Review, American Economic Association, vol. 105(5), pages 491-495, May.
    2. Alfredo M. Pereira & Jorge M. Andraz, 2013. "On The Economic Effects Of Public Infrastructure Investment: A Survey Of The International Evidence," Journal of Economic Development, Chung-Ang Unviersity, Department of Economics, vol. 38(4), pages 1-37, December.
    3. Munasib, Abdul & Rickman, Dan S., 2015. "Regional economic impacts of the shale gas and tight oil boom: A synthetic control analysis," Regional Science and Urban Economics, Elsevier, vol. 50(C), pages 1-17.
    4. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    5. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    6. Lee, Tae-Hwy & White, Halbert & Granger, Clive W. J., 1993. "Testing for neglected nonlinearity in time series models : A comparison of neural network methods and alternative tests," Journal of Econometrics, Elsevier, vol. 56(3), pages 269-290, April.
    7. David Alan Aschauer, 1989. "Back of the G-7 pack: public investment and productivity growth in the Group of Seven," Working Paper Series, Macroeconomic Issues 89-13, Federal Reserve Bank of Chicago.
    8. Aschauer, David Alan, 1989. "Does public capital crowd out private capital?," Journal of Monetary Economics, Elsevier, vol. 24(2), pages 171-188, September.
    9. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    10. Robert J. Barro, 1991. "Economic Growth in a Cross Section of Countries," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(2), pages 407-443.
    11. Thomas C. Buchmueller & John DiNardo & Robert G. Valletta, 2011. "The Effect of an Employer Health Insurance Mandate on Health Insurance Coverage and the Demand for Labor: Evidence from Hawaii," American Economic Journal: Economic Policy, American Economic Association, vol. 3(4), pages 25-51, November.
    12. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2008. "Nonparametric Tests for Treatment Effect Heterogeneity," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 389-405, August.
    13. Mercedes Gumbau Albert & Joaquín Maudos Villarroya & Pedro Cantos, 2002. "Transport Infrastructures And Regional Growth: Evidence Of The Spanish Case," Working Papers. Serie EC 2002-27, Instituto Valenciano de Investigaciones Económicas, S.A. (Ivie).
    14. Paolo Pinotti, 2015. "The Economic Costs of Organised Crime: Evidence from Southern Italy," Economic Journal, Royal Economic Society, vol. 125(586), pages 203-232, August.
    15. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    16. Kaul, Ashok & Klößner, Stefan & Pfeifer, Gregor & Schieler, Manuel, 2015. "Synthetic Control Methods: Never Use All Pre-Intervention Outcomes Together With Covariates," MPRA Paper 83790, University Library of Munich, Germany.
    17. Alfredo Pereira & Jorge Andraz, 2004. "Public highway spending and state spillovers in the USA," Applied Economics Letters, Taylor & Francis Journals, vol. 11(12), pages 785-788.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arne Steinkraus, 2019. "Estimating Treatment Effects With Artificial Neural Nets – A Comparison to Synthetic Control Method," Economics Bulletin, AccessEcon, vol. 39(4), pages 2778-2791.
    2. Sandra Achten & Lars Beyer & Antje-Mareike Dietrich & Dennis Ebeling & Christian Lessmann & Arne Steinkraus, 2019. "Large scale infrastructure investment and economic performance – a case study of Oresund," Applied Economics Letters, Taylor & Francis Journals, vol. 26(1), pages 21-26, January.
    3. Giulio Grossi & Marco Mariani & Alessandra Mattei & Patrizia Lattarulo & Ozge Oner, 2020. "Direct and spillover effects of a new tramway line on the commercial vitality of peripheral streets. A synthetic-control approach," Papers 2004.05027, arXiv.org, revised Nov 2023.
    4. Tianyuan Luo & Genti Kostandini, 2022. "Stringent immigration enforcement and responses of the immigrant‐intensive sector: Evidence from E‐Verify adoption in Arizona," American Journal of Agricultural Economics, John Wiley & Sons, vol. 104(4), pages 1411-1434, August.
    5. Samuel Verevis & Murat Üngör, 2021. "What has New Zealand gained from The FTA with China?: Two counterfactual analyses†," Scottish Journal of Political Economy, Scottish Economic Society, vol. 68(1), pages 20-50, February.
    6. repec:osf:osfxxx:s8ayp_v1 is not listed on IDEAS
    7. Kuosmanen, Timo & Zhou, Xun & Eskelinen, Juha & Malo, Pekka, 2021. "Design Flaw of the Synthetic Control Method," MPRA Paper 106328, University Library of Munich, Germany.
    8. David Audretsch & Paul P. Momtaz & Hanna Motuzenko & Silvio Vismara, 2023. "The Economic Costs of the Russia-Ukraine War: A Synthetic Control Study of (Lost) Entrepreneurship," Papers 2303.02773, arXiv.org.
    9. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    10. Bibek Adhikari, 2022. "A Guide to Using the Synthetic Control Method to Quantify the Effects of Shocks, Policies, and Shocking Policies," The American Economist, Sage Publications, vol. 67(1), pages 46-63, March.
    11. Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2021. "Decoupling synthetic control methods to ensure stability, accuracy and meaningfulness," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 12(4), pages 549-584, December.
    12. Thomas Barnebeck Andersen, 2023. "The Cost of a Currency Peg during the Great Recession," Open Economies Review, Springer, vol. 34(2), pages 255-279, April.
    13. Becker, Maike & Pfeifer, Gregor & Schweikert, Karsten, 2021. "Price Effects of the Austrian Fuel Price Fixing Act: A Synthetic Control Study," Energy Economics, Elsevier, vol. 97(C).
    14. Parast Layla & Hunt Priscillia & Griffin Beth Ann & Powell David, 2020. "When is a Match Sufficient? A Score-based Balance Metric for the Synthetic Control Method," Journal of Causal Inference, De Gruyter, vol. 8(1), pages 209-228, January.
    15. Mitja Kovac & Rok Spruk, 2025. "The Economic Dividends of Peace: Evidence from Arab-Israeli Normalization," Papers 2510.14517, arXiv.org, revised Oct 2025.
    16. Björn Falkenhall & Jonas Månsson & Sofia Tano, 2020. "Impact of VAT Reform on Swedish Restaurants: A Synthetic Control Group Approach," Scandinavian Journal of Economics, Wiley Blackwell, vol. 122(2), pages 824-850, April.
    17. Rodríguez-Puello, Gabriel, 2024. "Digging for Trouble? Uncovering the Link Between Mining Booms and Crime," OSF Preprints s8ayp, Center for Open Science.
    18. Gregor Pfeifer & Fabian Wahl & Martyna Marczak, 2018. "Illuminating the World Cup effect: Night lights evidence from South Africa," Journal of Regional Science, Wiley Blackwell, vol. 58(5), pages 887-920, November.
    19. repec:ran:wpaper:wr-1142 is not listed on IDEAS
    20. Billy, Alexander & Packard, Michael, 2022. "Crime and the Mariel Boatlift," International Review of Law and Economics, Elsevier, vol. 72(C).
    21. David B. Audretsch & Paul P. Momtaz & Hanna Motuzenko & Silvio Vismara, 2023. "War and Entrepreneurship: A Synthetic Control Study of the Russia-Ukraine Conflict," CESifo Working Paper Series 10466, CESifo.
    22. Yahaya, Shaibu, 2025. "Dutch Disease and Structural Transformation: Synthetic Control Evidence from Ghana's Oil Discovery," MPRA Paper 127064, University Library of Munich, Germany.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • O18 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Urban, Rural, Regional, and Transportation Analysis; Housing; Infrastructure

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:esprep:177390. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.