IDEAS home Printed from https://ideas.repec.org/a/ucp/jaerec/doi10.1086-689868.html
   My bibliography  Save this article

Panel Data Designs and Estimators as Substitutes for Randomized Controlled Trials in the Evaluation of Public Programs

Author

Listed:
  • Paul J. Ferraro
  • Juan José Miranda

Abstract

In the evaluation of public programs, experimental designs are rare. Researchers instead rely on observational designs. Observational designs that use panel data are widely portrayed as superior to time-series or cross-sectional designs because they provide opportunities to control for observable and unobservable variables correlated with outcomes and exposure to a program. The most popular panel data evaluation designs use linear, fixed-effects estimators with additive individual and time effects. To assess the ability of observational designs to replicate results from experimental designs, scholars use design replications. No such replications have assessed popular, fixed-effects panel data models that exploit repeated observations before and after treatment assignment. We implement such a study using, as a benchmark, results from a randomized environmental program that included effective and ineffective treatments. The popular linear, fixed-effects estimator fails to generate impact estimates or statistical inferences similar to the experimental estimator. Applying common flexible model specifications or trimming procedures also fail to yield accurate estimates or inferences. However, following best practices for selecting a nonexperimental comparison group and combining matching methods with panel data estimators, we replicate the experimental benchmarks. We demonstrate how the combination of panel and matching methods mitigates common concerns about specifying the correct functional form, the nature of treatment effect heterogeneity, and the way in which time enters the model. Our results are consistent with recent claims that design trumps methods in estimating treatment effects and that combining designs is more likely to approximate a randomized controlled trial than applying a single design.

Suggested Citation

  • Paul J. Ferraro & Juan José Miranda, 2017. "Panel Data Designs and Estimators as Substitutes for Randomized Controlled Trials in the Evaluation of Public Programs," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(1), pages 281-317.
  • Handle: RePEc:ucp:jaerec:doi:10.1086/689868
    DOI: 10.1086/689868
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1086/689868
    Download Restriction: Access to the online full text or PDF requires a subscription.

    File URL: http://dx.doi.org/10.1086/689868
    Download Restriction: Access to the online full text or PDF requires a subscription.

    File URL: https://libkey.io/10.1086/689868?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ucp:jaerec:doi:10.1086/689868. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Journals Division (email available below). General contact details of provider: https://www.journals.uchicago.edu/JAERE .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.