IDEAS home Printed from https://ideas.repec.org/p/iae/iaewps/wp2005n08.html
   My bibliography  Save this paper

Experimental and Quasi-Experimental Methods of Microeconomic Program and Policy Evaluation

Author

Listed:
  • Jeff Borland

    (Department of Economics, The University of Melbourne)

  • Yi-Ping Tseng

    (Melbourne Institute of Applied Economic and Social Research, The University of Melbourne)

  • Roger Wilkins

    (Melbourne Institute of Applied Economic and Social Research, The University of Melbourne)

Abstract

In this paper we review new empirical methods for evaluating microeconomic policies. Experimental and quasi-experimental evaluation measure the causal impact of a policy by comparing outcomes in the presence of the policy 'treatment' with outcomes in the absence of this treatment. For example, evaluation of a government program involves comparing outcomes associated with participation and non-participation in the program. We describe the motivation for the use of experimental and quasi-experimental methods, the types of policy effects that they can identify, and how they are implemented. Application of experimental and quasi-experimental methods is illustrated through a brief review of a variety of recent Australian studies that have evaluated microeconomic policies such as labour market programs, welfare payments policies, education policies, health policies and minimum wage laws.

Suggested Citation

  • Jeff Borland & Yi-Ping Tseng & Roger Wilkins, 2005. "Experimental and Quasi-Experimental Methods of Microeconomic Program and Policy Evaluation," Melbourne Institute Working Paper Series wp2005n08, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    • Handle: RePEc:iae:iaewps:wp2005n08
    as

    Download full text from publisher

    File URL: http://melbourneinstitute.unimelb.edu.au/downloads/working_paper_series/wp2005n08.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Carolyn Heinrich & Jeffrey Smith, 2002. "The Performance of Performance Standards," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 778-811.
    2. James Heckman, 2011. "Policies to foster human capital," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 3, pages 73-137.
    3. Richard Blundell & Alan Duncan & Costas Meghir, 1998. "Estimating Labor Supply Responses Using Tax Reforms," Econometrica, Econometric Society, vol. 66(4), pages 827-862, July.
    4. James J. Heckman, 2001. "Micro Data, Heterogeneity, and the Evaluation of Public Policy: Nobel Lecture," Journal of Political Economy, University of Chicago Press, vol. 109(4), pages 673-748, August.
    5. Linda L. Richardson, 2002. "Impact of the Mutual Obligation Initiative on the Exit Behaviour of Unemployment Benefit Recipients: The Threat of Additional Activities," The Economic Record, The Economic Society of Australia, vol. 78(243), pages 406-421, December.
    6. Anders Forslund & Alan B. Krueger, 1997. "An Evaluation of the Swedish Active Labor Market Policy: New and Received Wisdom," NBER Chapters, in: The Welfare State in Transition: Reforming the Swedish Model, pages 267-298, National Bureau of Economic Research, Inc.
    7. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    8. Lorraine Dearden & Alexandra Heath, 1996. "Income support and staying in school: what can we learn from Australia's AUSTUDY experiment?," Fiscal Studies, Institute for Fiscal Studies, vol. 17(4), pages 1-30, November.
    9. Richard Blundell & Monica Costa Dias, 2000. "Evaluation methods for non-experimental data," Fiscal Studies, Institute for Fiscal Studies, vol. 21(4), pages 427-468, January.
    10. Gerard J. van den Berg & Bas van der Klaauw & Jan C. van Ours, 2004. "Punitive Sanctions and the Transition Rate from Welfare to Work," Journal of Labor Economics, University of Chicago Press, vol. 22(1), pages 211-241, January.
    11. Richardson, Linda L, 2002. "Impact of the Mutual Obligation Initiative on the Exit Behaviour of Unemployment Benefit Recipients: The Threat of Additional Activities," The Economic Record, The Economic Society of Australia, vol. 78(243), pages 406-421, December.
    12. Heckman, James, 2001. "Accounting for Heterogeneity, Diversity and General Equilibrium in Evaluating Social Programmes," Economic Journal, Royal Economic Society, vol. 111(475), pages 654-699, November.
    13. James Heckman & Lance Lochner & Christopher Taber, 1998. "Explaining Rising Wage Inequality: Explanations With A Dynamic General Equilibrium Model of Labor Earnings With Heterogeneous Agents," Review of Economic Dynamics, Elsevier for the Society for Economic Dynamics, vol. 1(1), pages 1-58, January.
    14. Schmidt, Christoph M., 1999. "Knowing What Works: The Case for Rigorous Program Evaluation," IZA Discussion Papers 77, Institute of Labor Economics (IZA).
    15. Gruber, Jonathan, 1994. "The Incidence of Mandated Maternity Benefits," American Economic Review, American Economic Association, vol. 84(3), pages 622-641, June.
    16. Davidson, Carl & Woodbury, Stephen A, 1993. "The Displacement Effect of Reemployment Bonus Programs," Journal of Labor Economics, University of Chicago Press, vol. 11(4), pages 575-605, October.
    17. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records: Errata," American Economic Review, American Economic Association, vol. 80(5), pages 1284-1286, December.
    18. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    19. Alfons Palangkaraya & Jongsay Yong, 2007. "How effective is “lifetime health cover” in raising private health insurance coverage in Australia? An assessment using regression discontinuity," Applied Economics, Taylor & Francis Journals, vol. 39(11), pages 1361-1374.
    20. Besley, Timothy & Case, Anne, 2000. "Unnatural Experiments? Estimating the Incidence of Endogenous Policies," Economic Journal, Royal Economic Society, vol. 110(467), pages 672-694, November.
    21. Currie, Janet & Madrian, Brigitte C., 1999. "Health, health insurance and the labor market," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 50, pages 3309-3416, Elsevier.
    22. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    23. Angrist, Joshua D, 1990. "Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records," American Economic Review, American Economic Association, vol. 80(3), pages 313-336, June.
    24. Jeff Borland & Yi-Ping Tseng, 2003. "How Do Administrative Arrangements Affect Exit from Unemployment Payments? The Case of the Job Seeker Diary in Australia," Melbourne Institute Working Paper Series wp2003n27, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    25. Robert Breunig & Deborah A. Cobb‐Clark & Yvonne Dunlop & Marion Terrill, 2003. "Assisting the Long‐Term Unemployed: Results from a Randomised Trial," The Economic Record, The Economic Society of Australia, vol. 79(244), pages 84-102, March.
    26. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 136(III), pages 247-268, September.
    27. John Creedy & Alan Duncan, 2002. "Behavioural Microsimulation with Labour Supply Responses," Journal of Economic Surveys, Wiley Blackwell, vol. 16(1), pages 1-39, February.
    28. Andrew Leigh, 2004. "Employment Effects of Minimum Wages: Evidence from a Quasi-Experiment-Erratum," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 37(1), pages 102-105, March.
    29. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    30. Jeff Borland & Yi-Ping Tseng, 2004. "Does 'Work for the Dole' Work?," Melbourne Institute Working Paper Series wp2004n14, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    31. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-161, April.
    32. Andrew Leigh, 2003. "Employment Effects of Minimum Wages: Evidence from a Quasi‐Experiment," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 36(4), pages 361-373, December.
    33. James J. Heckman, 2000. "Causal Parameters and Policy Analysis in Economics: A Twentieth Century Retrospective," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 115(1), pages 45-97.
    34. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    35. Jaap H. Abbring & Gerard J. van den Berg, 2003. "The Nonparametric Identification of Treatment Effects in Duration Models," Econometrica, Econometric Society, vol. 71(5), pages 1491-1517, September.
    36. Joshua D. Angrist & Victor Lavy, 1997. "Using Maimonides' Rule to Estimate the Effect of Class Size on Student Achievement," NBER Working Papers 5888, National Bureau of Economic Research, Inc.
    37. Peter Bardsley, 2003. "Missing Environmental Markets and the Design of," Department of Economics - Working Papers Series 891, The University of Melbourne.
    38. Deborah A. Cobb‐Clark & Thomas Crossley, 2003. "Econometrics for Evaluations: An Introduction to Recent Developments," The Economic Record, The Economic Society of Australia, vol. 79(247), pages 491-511, December.
    39. Jeff Borland & Roger Wilkins, 2003. "Effects of Activity Test Arrangements on Exit from Payments: The 9-Month Intensive Review," Melbourne Institute Working Paper Series wp2003n25, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    40. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
    41. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 533-575.
    42. Denise J. Doiron, 2004. "Welfare Reform and the Labour Supply of Lone Parents in Australia: A Natural Experiment Approach," The Economic Record, The Economic Society of Australia, vol. 80(249), pages 157-176, June.
    43. Garry F. Barrett & Deborah A. Cobb-Clark, 2000. "The labour market plans of parenting payment recipients: information from a randomised social experiment," Australian Journal of Labour Economics (AJLE), Bankwest Curtin Economics Centre (BCEC), Curtin Business School, vol. 4(3), pages 192-205, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Massimiliano Bratti & Daniele Checchi & Guido De Blasio, 2008. "Does the Expansion of Higher Education Increase the Equality of Educational Opportunities? Evidence from Italy," LABOUR, CEIS, vol. 22(s1), pages 53-88, June.
    2. Wright, Christopher, 2006. "Estimating the Effect of the Chesapeake Bay Program on Application Rates for Enrollment in the Environmental Quality Incentive Program: A Case Study of Pennsylvania," 2006 Annual meeting, July 23-26, Long Beach, CA 21164, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Deborah A. Cobb‐Clark & Thomas Crossley, 2003. "Econometrics for Evaluations: An Introduction to Recent Developments," The Economic Record, The Economic Society of Australia, vol. 79(247), pages 491-511, December.
    3. van der Klaauw, Bas, 2014. "From micro data to causality: Forty years of empirical labor economics," Labour Economics, Elsevier, vol. 30(C), pages 88-97.
    4. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    5. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    6. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    7. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 136(III), pages 247-268, September.
    8. Markus Frölich, 2004. "Programme Evaluation with Multiple Treatments," Journal of Economic Surveys, Wiley Blackwell, vol. 18(2), pages 181-224, April.
    9. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    10. Guido W. Imbens, 2022. "Causality in Econometrics: Choice vs Chance," Econometrica, Econometric Society, vol. 90(6), pages 2541-2566, November.
    11. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    12. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-161, April.
    13. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    14. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    15. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    16. Angus Deaton, 2009. "Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development," Working Papers 1128, Princeton University, Woodrow Wilson School of Public and International Affairs, Center for Health and Wellbeing..
    17. Ravallion, Martin, 2008. "Evaluating Anti-Poverty Programs," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 59, pages 3787-3846, Elsevier.
    18. James J. Heckman, 2005. "Micro Data, Heterogeneity and the Evaluation of Public Policy Part 2," The American Economist, Sage Publications, vol. 49(1), pages 16-44, March.
    19. Smith, Jeffrey, 2000. "Evaluation aktiver Arbeitsmarktpolitik : Erfahrungen aus Nordamerika (Evaluating Avtive Labor Market Policies : Lessons from North America)," Mitteilungen aus der Arbeitsmarkt- und Berufsforschung, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany], vol. 33(3), pages 345-356.
    20. Greenstone, Michael & Gayer, Ted, 2009. "Quasi-experimental and experimental approaches to environmental economics," Journal of Environmental Economics and Management, Elsevier, vol. 57(1), pages 21-44, January.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iae:iaewps:wp2005n08. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sheri Carnegie (email available below). General contact details of provider: https://edirc.repec.org/data/mimelau.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.