IDEAS home Printed from https://ideas.repec.org/a/sae/pubfin/v45y2017i1p10-43.html
   My bibliography  Save this article

Impact Evaluation Methods in Public Economics

Author

Listed:
  • Dina Pomeranz

Abstract

Recent years have seen a large expansion in the use of rigorous impact evaluation techniques. Increasingly, public administrations are collaborating with academic economists and other quantitative social scientists to apply such rigorous methods to the study of public finance. These developments allow for more reliable measurements of the effects of different policy options on the behavioral responses of citizens, firm owners, or public officials. They can help decision makers in tax administrations, public procurement offices, and other public agencies design programs informed by well-founded evidence. This article provides an introductory overview of the most frequently used impact evaluation methods. It is aimed at facilitating communication and collaboration between practitioners and academics by introducing key vocabulary and concepts used in rigorous impact evaluation methods, starting with randomized controlled trials and comparing them with other methods ranging from simple pre–post analysis to difference-in-differences, matching estimations, and regression discontinuity designs.

Suggested Citation

  • Dina Pomeranz, 2017. "Impact Evaluation Methods in Public Economics," Public Finance Review, , vol. 45(1), pages 10-43, January.
  • Handle: RePEc:sae:pubfin:v:45:y:2017:i:1:p:10-43
    DOI: 10.1177/1091142115614392
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/1091142115614392
    Download Restriction: no

    File URL: https://libkey.io/10.1177/1091142115614392?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Stephan Litschig & Yves Zamboni, 2011. "Audit risk and rent extraction: Evidence from a randomized evaluation in Brazil," Economics Working Papers 1270, Department of Economics and Business, Universitat Pompeu Fabra, revised Jun 2013.
    2. Sean Lewis-Faupel & Yusuf Neggers & Benjamin A. Olken & Rohini Pande, 2016. "Can Electronic Procurement Improve Infrastructure Provision? Evidence from Public Works in India and Indonesia," American Economic Journal: Economic Policy, American Economic Association, vol. 8(3), pages 258-283, August.
    3. Abhijit V. Banerjee & Shawn Cole & Esther Duflo & Leigh Linden, 2007. "Remedying Education: Evidence from Two Randomized Experiments in India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1235-1264.
    4. Dean Karlan & Sendhil Mullainathan & Margaret McConnell & Jonathan Zinman, 2010. "Getting to theTop of Mind: How Reminders Increase Saving," Working Papers id:2587, eSocialSciences.
    5. Esther Duflo, 2001. "Schooling and Labor Market Consequences of School Construction in Indonesia: Evidence from an Unusual Policy Experiment," American Economic Review, American Economic Association, vol. 91(4), pages 795-813, September.
    6. Henrik Jacobsen Kleven & Martin B. Knudsen & Claus Thustrup Kreiner & Søren Pedersen & Emmanuel Saez, 2011. "Unwilling or Unable to Cheat? Evidence From a Tax Audit Experiment in Denmark," Econometrica, Econometric Society, vol. 79(3), pages 651-692, May.
    7. Felipe Kast & Dina Pomeranz, 2013. "Saving More to Borrow Less: Experimental Evidence from Access to Formal Savings Accounts in Chile," Harvard Business School Working Papers 14-001, Harvard Business School, revised Jun 2014.
    8. Gerlinde Fellner & Rupert Sausgruber & Christian Traxler, 2009. "Testing Enforcement Strategies in the Field: Legal Threat, Moral Appeal and Social Information," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2009_31, Max Planck Institute for Research on Collective Goods.
    9. Castro, Lucio & Scartascini, Carlos, 2015. "Tax compliance and enforcement in the pampas evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 116(C), pages 65-82.
    10. Rachel Glennerster & Kudzai Takavarasha, 2013. "Running Randomized Evaluations: A Practical Guide," Economics Books, Princeton University Press, edition 1, number 10085.
    11. Lorenzo Casaburi & Ugo Troiano, 2016. "Ghost-House Busters: The Electoral Response to a Large Anti–Tax Evasion Program," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(1), pages 273-314.
    12. Abhijit V. Banerjee & Esther Duflo, 2009. "The Experimental Approach to Development Economics," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 151-178, May.
    13. Hallsworth, Michael & List, John A. & Metcalfe, Robert D. & Vlaev, Ivo, 2017. "The behavioralist as tax collector: Using natural field experiments to enhance tax compliance," Journal of Public Economics, Elsevier, vol. 148(C), pages 14-31.
    14. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    15. John Hasseldine & Peggy Hite & Simon James & Marika Toumi, 2007. "Persuasive Communications: Tax Compliance Enforcement Strategies for Sole Proprietors," Contemporary Accounting Research, John Wiley & Sons, vol. 24(1), pages 171-194, March.
    16. Sylvie Moulin & Michael Kremer & Paul Glewwe, 2009. "Many Children Left Behind? Textbooks and Test Scores in Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 1(1), pages 112-135, January.
    17. Coleman, Stephen, 1996. "The Minnesota income tax compliance experiment: State tax results," MPRA Paper 4827, University Library of Munich, Germany.
    18. Paul Carrillo & Dina Pomeranz & Monica Singhal, 2017. "Dodging the Taxman: Firm Misreporting and Limits to Tax Enforcement," American Economic Journal: Applied Economics, American Economic Association, vol. 9(2), pages 144-164, April.
    19. Slemrod, Joel & Blumenthal, Marsha & Christian, Charles, 2001. "Taxpayer response to an increased probability of audit: evidence from a controlled experiment in Minnesota," Journal of Public Economics, Elsevier, vol. 79(3), pages 455-483, March.
    20. Prina, Silvia, 2015. "Banking the poor via savings accounts: Evidence from a field experiment," Journal of Development Economics, Elsevier, vol. 115(C), pages 16-31.
    21. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    22. Benno Torgler, 2004. "Moral suasion: An alternative tax policy strategy? Evidence from a controlled field experiment in Switzerland," Economics of Governance, Springer, vol. 5(3), pages 235-253, November.
    23. Laura C. Haynes & Donald P. Green & Rory Gallagher & Peter John & David J. Torgerson, 2013. "Collection of Delinquent Fines: An Adaptive Randomized Trial to Assess the Effectiveness of Alternative Text Messages," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 32(4), pages 718-730, September.
    24. Gerlinde Fellner & Rupert Sausgruber & Christian Traxler, 2013. "Testing Enforcement Strategies In The Field: Threat, Moral Appeal And Social Information," Journal of the European Economic Association, European Economic Association, vol. 11(3), pages 634-660, June.
    25. Pascaline Dupas & Jonathan Robinson, 2013. "Why Don't the Poor Save More? Evidence from Health Savings Experiments," American Economic Review, American Economic Association, vol. 103(4), pages 1138-1171, June.
    26. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    27. Blumenthal, Marsha & Christian, Charles W. & Slemrod, Joel, 2001. "Do Normative Appeals Affect Tax Compliance? Evidence from a Controlled Experiment in Minnesota," National Tax Journal, National Tax Association, vol. 54(n. 1), pages 125-38, March.
    28. Ortega, Daniel & Sanguinetti, Pablo, 2013. "Deterrence and reciprocity effects on tax compliance: experimental evidence from Venezuela," Research Department working papers 253, CAF Development Bank Of Latinamerica.
    29. Wenzel, Michael, 2005. "Misperceptions of social norms about tax compliance: From theory to intervention," Journal of Economic Psychology, Elsevier, vol. 26(6), pages 862-883, December.
    30. Nicky J. Welton & Howard H. Z. Thom, 2015. "Value of Information," Medical Decision Making, , vol. 35(5), pages 564-566, July.
    31. Benno Torgler, 2013. "A Field Experiment in Moral Suasion and Tax Compliance Focusing on Underdeclaration and Overdeduction," FinanzArchiv: Public Finance Analysis, Mohr Siebeck, Tübingen, vol. 69(4), pages 393-411, December.
    32. Benno Torgler, 2004. "Moral Suasion: An alternative tax policy strategy? Evidence from a controlled field experiment in Switzerland," CREMA Working Paper Series 2004-01, Center for Research in Economics, Management and the Arts (CREMA).
    33. Paul J. Gertler & Sebastian Martinez & Patrick Premand & Laura B. Rawlings & Christel M. J. Vermeersch, . "Impact Evaluation in Practice, First Edition [La evaluación de impacto en la práctica]," World Bank Publications, The World Bank, number 2550, September.
    34. Blumenthal, Marsha & Christian, Charles W. & Slemrod, Joel, 2001. "Do Normative Appeals Affect Tax Compliance? Evidence From a Controlled Experiment in Minnesota," National Tax Journal, National Tax Association;National Tax Journal, vol. 54(1), pages 125-138, March.
    35. Joshua D. Angrist & Jörn-Steffen Pischke, 2015. "The path from cause to effect: mastering 'metrics," CentrePiece - The magazine for economic performance 442, Centre for Economic Performance, LSE.
    36. Saurabh Bhargava & Dayanand Manoli, 2015. "Psychological Frictions and the Incomplete Take-Up of Social Benefits: Evidence from an IRS Field Experiment," American Economic Review, American Economic Association, vol. 105(11), pages 3489-3529, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sagar Ratna Bajracharya & Narendra Raj Khanal & Pashupati Nepal & Sundar Kumar Rai & Pawan Kumar Ghimire & Neera Shrestha Pradhan, 2021. "Community Assessment of Flood Risks and Early Warning System in Ratu Watershed, Koshi Basin, Nepal," Sustainability, MDPI, vol. 13(6), pages 1-18, March.
    2. Fels, Katja M., 2021. "Who nudges whom? Field experiments with public partners," Ruhr Economic Papers 906, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    3. Vitaly L. Tambovtsev, 2021. "Targeted programs in the Russian Federation as a matter for evaluation," Upravlenets, Ural State University of Economics, vol. 12(2), pages 46-62, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dina Pomeranz & José Vila-Belda, 2019. "Taking State-Capacity Research to the Field: Insights from Collaborations with Tax Authorities," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 755-781, August.
    2. Biddle, Nicholas & Fels, Katja M. & Sinning, Mathias, 2018. "Behavioral insights on business taxation: Evidence from two natural field experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 18(C), pages 30-49.
    3. Biddle, Nicholas & Fels, Katja & Sinning, Mathias, 2017. "Behavioral insights and business taxation: Evidence from two randomized controlled trials," Ruhr Economic Papers 698, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    4. Dong, Sarah Xue & Sinning, Mathias, 2022. "Trying to Make a Good First Impression: A Natural Field Experiment to Engage New Entrants to the Tax System," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 100(C).
    5. Antinyan, Armenak & Asatryan, Zareh, 2019. "Nudging for tax compliance: A meta-analysis," ZEW Discussion Papers 19-055, ZEW - Leibniz Centre for European Economic Research.
    6. Castro, Lucio & Scartascini, Carlos, 2015. "Tax compliance and enforcement in the pampas evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 116(C), pages 65-82.
    7. Giulia Mascagni, 2018. "From The Lab To The Field: A Review Of Tax Experiments," Journal of Economic Surveys, Wiley Blackwell, vol. 32(2), pages 273-301, April.
    8. Hallsworth, Michael & List, John A. & Metcalfe, Robert D. & Vlaev, Ivo, 2017. "The behavioralist as tax collector: Using natural field experiments to enhance tax compliance," Journal of Public Economics, Elsevier, vol. 148(C), pages 14-31.
    9. Ortega, Daniel & Scartascini, Carlos, 2015. "Don't Blame the Messenger: A Field Experiment on Delivery Methods for Increasing Tax Compliance," IDB Publications (Working Papers) 7284, Inter-American Development Bank.
    10. Sinning, Mathias & Zhang, Yinjunjie, 2023. "Social norms or enforcement? A natural field experiment to improve traffic and parking fine compliance," Journal of Economic Behavior & Organization, Elsevier, vol. 210(C), pages 43-60.
    11. Daniel Ortega & Carlos Scartascini, 2015. "Don't Blame the Messenger: A Field Experiment on Delivery Methods for Increasing Tax Compliance," IDB Publications (Working Papers) 91741, Inter-American Development Bank.
    12. James Alm, 2019. "What Motivates Tax Compliance?," Journal of Economic Surveys, Wiley Blackwell, vol. 33(2), pages 353-388, April.
    13. Kristina M. Bott & Alexander W. Cappelen & Erik Ø. Sørensen & Bertil Tungodden, 2020. "You’ve Got Mail: A Randomized Field Experiment on Tax Evasion," Management Science, INFORMS, vol. 66(7), pages 2801-2819, July.
    14. James Alm & Laura Rosales Cifuentes & Carlos Mauricio Ortiz Niño & Diana Rocha, 2019. "Can Behavioral “Nudges” Improve Compliance? The Case of Colombia Social Protection Contributions," Games, MDPI, vol. 10(4), pages 1-23, October.
    15. Perez-Truglia, Ricardo & Troiano, Ugo, 2018. "Shaming tax delinquents," Journal of Public Economics, Elsevier, vol. 167(C), pages 120-137.
    16. Philipp Doerrenberg & Jan Schmitz, 2017. "Tax compliance and information provision. A field experiment with small firms," Journal of Behavioral Economics for Policy, Society for the Advancement of Behavioral Economics (SABE), vol. 1(1), pages 47-54, February.
    17. Matthew Collin & Vincenzo Di Maro & David K. Evans & Fredrick Manang, 2022. "Property Tax Compliance in Tanzania: Can Nudges Help?," Working Papers 621, Center for Global Development.
    18. James Alm & Lilith Burgstaller & Arrita Domi & Amanda März & Matthias Kasper, 2023. "Nudges, Boosts, and Sludge: Using New Behavioral Approaches to Improve Tax Compliance," Economies, MDPI, vol. 11(9), pages 1-22, September.
    19. Anne Brockmeyer & Spencer Smith & Marco Hernandez & Stewart Kettle, 2019. "Casting a Wider Tax Net: Experimental Evidence from Costa Rica," American Economic Journal: Economic Policy, American Economic Association, vol. 11(3), pages 55-87, August.
    20. Peter John & Toby Blume, 2018. "How best to nudge taxpayers? The impact of message simplification and descriptive social norms on payment rates in a central London local authority," Journal of Behavioral Public Administration, Center for Experimental and Behavioral Public Administration, vol. 1(1).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:pubfin:v:45:y:2017:i:1:p:10-43. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.