IDEAS home Printed from https://ideas.repec.org/p/ecl/harjfk/rwp01-019.html
   My bibliography  Save this paper

Testing for Structural Breaks in the Evaluation of Programs

Author

Listed:
  • Cooper, Suzanne

    (Harvard U)

  • Piehl, Anne Morrison

    (Harvard U)

  • Braga, Anthony

    (Harvard U)

  • Kennedy, David

    (Harvard U)

Abstract

A youth homicide reduction initiative in Boston in the mid-1990s poses particular difficulties for program evaluation because it did not have a control group and the exact implementation date is unknown. A standard methodology in program evaluation is to use time series variation to compare pre- and post-program outcomes. Such an approach is not valid, however, when the timing of a potential break is unknown. To evaluate the Boston initiative, we adapt from the macroeconomics literature a test of unknown break point to test for a change in regime. Tests for parameter instability provide a flexible framework for testing a range of hypotheses commonly posed in program evaluation. These tests both pinpoint the timing of maximal break and provide a valid test of statistical significance. We evaluate the results of the estimation using the asymptotic results in the literature and with our own Monte Carlo analyses. We conclude there was a statistically significant discontinuity in youth homicide incidents (on the order of 60 percent) shortly after the intervention was unveiled.

Suggested Citation

  • Cooper, Suzanne & Piehl, Anne Morrison & Braga, Anthony & Kennedy, David, 2001. "Testing for Structural Breaks in the Evaluation of Programs," Working Paper Series rwp01-019, Harvard University, John F. Kennedy School of Government.
  • Handle: RePEc:ecl:harjfk:rwp01-019
    as

    Download full text from publisher

    File URL: https://research.hks.harvard.edu/publications/getFile.aspx?Id=11
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bai, Jushan, 1997. "Estimating Multiple Breaks One at a Time," Econometric Theory, Cambridge University Press, vol. 13(3), pages 315-352, June.
    2. Steven D. Levitt & Sudhir Alladi Venkatesh, 2000. "An Economic Analysis of a Drug-Selling Gang's Finances," The Quarterly Journal of Economics, Oxford University Press, vol. 115(3), pages 755-789.
    3. Cooper, Suzanne J, 1998. "Multiple Regimes in U.S. Output Fluctuations," Journal of Business & Economic Statistics, American Statistical Association, vol. 16(1), pages 92-100, January.
    4. Hamilton, James D, 1989. "A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle," Econometrica, Econometric Society, vol. 57(2), pages 357-384, March.
    5. Potter, Simon M, 1995. "A Nonlinear Approach to US GNP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 10(2), pages 109-125, April-Jun.
    6. Jushan Bai & Pierre Perron, 1998. "Estimating and Testing Linear Models with Multiple Structural Changes," Econometrica, Econometric Society, vol. 66(1), pages 47-78, January.
    7. Andrews, Donald W K & Ploberger, Werner, 1994. "Optimal Tests When a Nuisance Parameter Is Present Only under the Alternative," Econometrica, Econometric Society, vol. 62(6), pages 1383-1414, November.
    8. Quintos, Carmela E, 1995. "Sustainability of the Deficit Process with Structural Shifts," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(4), pages 409-417, October.
    9. Banerjee, Anindya & Lumsdaine, Robin L & Stock, James H, 1992. "Recursive and Sequential Tests of the Unit-Root and Trend-Break Hypotheses: Theory and International Evidence," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(3), pages 271-287, July.
    10. Andrews, Donald W K, 1993. "Tests for Parameter Instability and Structural Change with Unknown Change Point," Econometrica, Econometric Society, vol. 61(4), pages 821-856, July.
    11. Budd, J.W. & Nho, Y., 1995. "Testing for a Structurak Change in U.S. Wage Determination," Papers 95-03, Minnesota - Industrial Relations Center.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Uctum, Remzi, 2007. "Économétrie des modèles à changement de régimes : un essai de synthèse," L'Actualité Economique, Société Canadienne de Science Economique, vol. 83(4), pages 447-482, décembre.
    2. Yu, Ping & Phillips, Peter C.B., 2018. "Threshold regression with endogeneity," Journal of Econometrics, Elsevier, vol. 203(1), pages 50-68.
    3. Gonzalo, Jesus & Pitarakis, Jean-Yves, 2002. "Estimation and model selection based inference in single and multiple threshold models," Journal of Econometrics, Elsevier, vol. 110(2), pages 319-352, October.
    4. Travaglini, Guido, 2007. "The U.S. Dynamic Taylor Rule With Multiple Breaks, 1984-2001," MPRA Paper 3419, University Library of Munich, Germany, revised 15 Jun 2007.
    5. Abhijit Sharma & Kelvin G Balcombe & Iain M Fraser, 2009. "Non-renewable resource prices: Structural breaks and long term trends," Economics Bulletin, AccessEcon, vol. 29(2), pages 805-819.
    6. Somayeh Mardaneh, 2012. "How Do Oil Shocks A¤ect the Structural Stability of Hybrid New Keynesian Phillips Curve?," Discussion Papers in Economics 12/20, Division of Economics, School of Business, University of Leicester.
    7. Jushan Bai & Pierre Perron, 2003. "Computation and analysis of multiple structural change models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 1-22.
    8. Carrasco, Marine, 2002. "Misspecified Structural Change, Threshold, and Markov-switching models," Journal of Econometrics, Elsevier, vol. 109(2), pages 239-273, August.
    9. P Muthuramu & T Uma Maheswari, 2019. "Tests for Structural Breaks in Time Series Analysis: A Review of Recent Development," Shanlax International Journal of Economics, Shanlax Journals, vol. 7(4), pages 66-79, September.
    10. Juan F. Jimeno & Esther Moral & Lorena Saiz, 2006. "Structural breaks in labor productivity growth: the United States vs. the European Union," Working Papers 0625, Banco de España.
    11. Hansen,B.E., 1999. "Testing for linearity," Working papers 7, Wisconsin Madison - Social Systems.
    12. Bruce E. Hansen, 2001. "The New Econometrics of Structural Change: Dating Breaks in U.S. Labour Productivity," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 117-128, Fall.
    13. Chauvet, Marcelle & Senyuz, Zeynep, 2016. "A dynamic factor model of the yield curve components as a predictor of the economy," International Journal of Forecasting, Elsevier, vol. 32(2), pages 324-343.
    14. Paraskevi Katsiampa & Kyriaki Begiazi, 2019. "An empirical analysis of the Scottish housing market by property type," Scottish Journal of Political Economy, Scottish Economic Society, vol. 66(4), pages 559-583, September.
    15. Njindan Iyke, Bernard, 2015. "Asymmetries, Structural Breaks, and Nonlinear Persistence: Evidence and Implications for Uncovering the Energy-Growth Nexus in Selected African Countries," MPRA Paper 67163, University Library of Munich, Germany.
    16. Chauvet, Marcelle & Senyuz, Zeynep, 2008. "A Joint Dynamic Bi-Factor Model of the Yield Curve and the Economy as a Predictor of Business Cycles," MPRA Paper 15076, University Library of Munich, Germany, revised Apr 2009.
    17. Marcelle Chauvet & Zeynep Senyuz, 2012. "A Dynamic Factor Model of the Yield Curve as a Predictor of the Economy," Finance and Economics Discussion Series 2012-32, Board of Governors of the Federal Reserve System (U.S.).
    18. Benati, Luca, 2007. "Drift and breaks in labor productivity," Journal of Economic Dynamics and Control, Elsevier, vol. 31(8), pages 2847-2877, August.
    19. Gómez-Puig, Marta & Sosvilla-Rivero, Simón, 2014. "Causality and contagion in EMU sovereign debt markets," International Review of Economics & Finance, Elsevier, vol. 33(C), pages 12-27.
    20. Devi, P. Indira & Shanmugam, K.R. & Jayasree, M.G., 2012. "Compensating Wages for Occupational Risks of Farm Workers in India," Indian Journal of Agricultural Economics, Indian Society of Agricultural Economics, vol. 67(2), pages 1-12.

    More about this item

    JEL classification:

    • C22 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes
    • K42 - Law and Economics - - Legal Procedure, the Legal System, and Illegal Behavior - - - Illegal Behavior and the Enforcement of Law

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecl:harjfk:rwp01-019. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/ksharus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (email available below). General contact details of provider: https://edirc.repec.org/data/ksharus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.