IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v151y2009i1p33-46.html
   My bibliography  Save this article

A specification test for the propensity score using its distribution conditional on participation

Author

Listed:
  • Shaikh, Azeem M.
  • Simonsen, Marianne
  • Vytlacil, Edward J.
  • Yildiz, Nese

Abstract

Propensity score matching has become a popular method for the estimation of average treatment effects. In empirical applications, researchers almost always impose a parametric model for the propensity score. This practice raises the possibility that the model for the propensity score is misspecified and therefore the propensity score matching estimator of the average treatment effect may be inconsistent. We show that the common practice of calculating estimates of the densities of the propensity score conditional on the participation decision provides a means for examining whether the propensity score is misspecified. In particular, we derive a restriction between the density of the propensity score among participants and the density among nonparticipants. We show that this restriction between the two conditional densities is equivalent to a particular orthogonality restriction and derive a formal test based upon it. The resulting test is shown via a simulation study to have dramatically greater power than competing tests for many alternatives. The principal disadvantage of this approach is loss of power against some alternatives.

Suggested Citation

  • Shaikh, Azeem M. & Simonsen, Marianne & Vytlacil, Edward J. & Yildiz, Nese, 2009. "A specification test for the propensity score using its distribution conditional on participation," Journal of Econometrics, Elsevier, vol. 151(1), pages 33-46, July.
  • Handle: RePEc:eee:econom:v:151:y:2009:i:1:p:33-46
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304-4076(09)00049-9
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Michael Lechner, 2002. "Program Heterogeneity And Propensity Score Matching: An Application To The Evaluation Of Active Labor Market Policies," The Review of Economics and Statistics, MIT Press, vol. 84(2), pages 205-220, May.
    2. Michael Lechner, 2000. "An Evaluation of Public-Sector-Sponsored Continuous Vocational Training Programs in East Germany," Journal of Human Resources, University of Wisconsin Press, vol. 35(2), pages 347-375.
    3. Smith, Jeffrey & Todd, Petra, 2005. "Rejoinder," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 365-375.
    4. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    5. Barbara Sianesi, 2004. "An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 133-155, February.
    6. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    7. Horowitz, Joel L & Spokoiny, Vladimir G, 2001. "An Adaptive, Rate-Optimal Test of a Parametric Mean-Regression Model against a Nonparametric Alternative," Econometrica, Econometric Society, vol. 69(3), pages 599-631, May.
    8. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    9. Hong, Yongmiao & White, Halbert, 1995. "Consistent Specification Testing via Nonparametric Series Regression," Econometrica, Econometric Society, vol. 63(5), pages 1133-1159, September.
    10. Michael Lechner, 1999. "Nonparametric bounds on employment and income effects of continuous vocational training in East Germany," Econometrics Journal, Royal Economic Society, vol. 2(1), pages 1-28.
    11. Powell, James L & Stock, James H & Stoker, Thomas M, 1989. "Semiparametric Estimation of Index Coefficients," Econometrica, Econometric Society, vol. 57(6), pages 1403-1430, November.
    12. Lechner, Michael, 1999. "Earnings and Employment Effects of Continuous Off-the-Job Training in East Germany after Unification," Journal of Business & Economic Statistics, American Statistical Association, vol. 17(1), pages 74-90, January.
    13. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    14. John Xu Zheng, 1996. "A consistent test of functional form via nonparametric estimation techniques," Journal of Econometrics, Elsevier, vol. 75(2), pages 263-289, December.
    15. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(2), pages 261-294.
    16. Lars Skipper & Marianne Simonsen, 2006. "The costs of motherhood: an analysis using matching estimators," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(7), pages 919-934.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rajeev Dehejia, 2013. "The Porous Dialectic: Experimental and Non-Experimental Methods in Development Economics," WIDER Working Paper Series wp-2013-011, World Institute for Development Economic Research (UNU-WIDER).
    2. Dehejia, Rajeev, 2013. "The Porous Dialectic: Experimental and Non-Experimental Methods in Development Economics," WIDER Working Paper Series 011, World Institute for Development Economic Research (UNU-WIDER).
    3. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    4. Steven Lehrer & Gregory Kordas, 2013. "Matching using semiparametric propensity scores," Empirical Economics, Springer, vol. 44(1), pages 13-45, February.
    5. Sant’Anna, Pedro H.C. & Song, Xiaojun, 2019. "Specification tests for the propensity score," Journal of Econometrics, Elsevier, vol. 210(2), pages 379-404.
    6. Martin Huber, 2011. "Testing for covariate balance using quantile regression and resampling methods," Journal of Applied Statistics, Taylor & Francis Journals, vol. 38(12), pages 2881-2899, February.
    7. Martin Huber, 2012. "Identification of Average Treatment Effects in Social Experiments Under Alternative Forms of Attrition," Journal of Educational and Behavioral Statistics, , vol. 37(3), pages 443-474, June.
    8. Wang-Sheng Lee, 2013. "Propensity score matching and variations on the balancing test," Empirical Economics, Springer, vol. 44(1), pages 47-80, February.
    9. Li Liang & Greene Tom, 2013. "A Weighting Analogue to Pair Matching in Propensity Score Analysis," The International Journal of Biostatistics, De Gruyter, vol. 9(2), pages 215-234, July.
    10. Pedro H. C. Sant'Anna & Xiaojun Song, 2020. "Specification tests for generalized propensity scores using double projections," Papers 2003.13803, arXiv.org, revised Apr 2023.
    11. Huber, Martin, 2012. "Identifying causal mechanisms in experiments (primarily) based on inverse probability weighting," Economics Working Paper Series 1213, University of St. Gallen, School of Economics and Political Science, revised May 2013.
    12. Russell Davidson & Victoria Zinde‐Walsh, 2017. "Advances in specification testing," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 50(5), pages 1595-1631, December.
    13. von Eije, Henk & Goyal, Abhinav & Muckley, Cal B., 2014. "Does the information content of payout initiations and omissions influence firm risks?," Journal of Econometrics, Elsevier, vol. 183(2), pages 222-229.
    14. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    15. Dehejia Rajeev, 2015. "Experimental and Non-Experimental Methods in Development Economics: A Porous Dialectic," Journal of Globalization and Development, De Gruyter, vol. 6(1), pages 47-69, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2010. "How to Control for Many Covariates? Reliable Estimators Based on the Propensity Score," IZA Discussion Papers 5268, Institute of Labor Economics (IZA).
    2. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.
    3. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355.
    4. Michael Lechner & Ruth Miquel & Conny Wunsch, 2011. "Long‐Run Effects Of Public Sector Sponsored Training In West Germany," Journal of the European Economic Association, European Economic Association, vol. 9(4), pages 742-784, August.
    5. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    6. Dettmann, Eva & Becker, Claudia & Schmeißer, Christian, 2010. "Is there a Superior Distance Function for Matching in Small Samples?," IWH Discussion Papers 3/2010, Halle Institute for Economic Research (IWH).
    7. Jochen Kluve & Boris Augurzky, 2007. "Assessing the performance of matching algorithms when selection into treatment is strong," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(3), pages 533-557.
    8. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    9. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    10. Huber, Martin & Lechner, Michael & Wunsch, Conny, 2013. "The performance of estimators based on the propensity score," Journal of Econometrics, Elsevier, vol. 175(1), pages 1-21.
    11. Aschhoff, Birgit, 2009. "The effect of subsidies on R&D investment and success: do subsidy history and size matter?," ZEW Discussion Papers 09-032, ZEW - Leibniz Centre for European Economic Research.
    12. Hämäläinen, Kari & Ollikainen, Virve, 2004. "Differential Effects of Active Labour Market Programmes in the Early Stages of Young People's Unemployment," Research Reports 115, VATT Institute for Economic Research.
    13. Dirk Czarnitzki & Cindy Lopes-Bento, 2014. "Innovation Subsidies: Does the Funding Source Matter for Innovation Intensity and Performance? Empirical Evidence from Germany," Industry and Innovation, Taylor & Francis Journals, vol. 21(5), pages 380-409, July.
    14. Volpe Martincus, Christian & Carballo, Jerónimo, 2010. "Beyond the average effects: The distributional impacts of export promotion programs in developing countries," Journal of Development Economics, Elsevier, vol. 92(2), pages 201-214, July.
    15. Lechner Michael & Miquel Ruth & Wunsch Conny, 2007. "The Curse and Blessing of Training the Unemployed in a Changing Economy: The Case of East Germany After Unification," German Economic Review, De Gruyter, vol. 8(4), pages 468-509, December.
    16. Markus Frölich, 2004. "Programme Evaluation with Multiple Treatments," Journal of Economic Surveys, Wiley Blackwell, vol. 18(2), pages 181-224, April.
    17. Czarnitzki, Dirk & Lopes-Bento, Cindy, 2013. "Value for money? New microeconometric evidence on public R&D grants in Flanders," Research Policy, Elsevier, vol. 42(1), pages 76-89.
    18. Czarnitzki, Dirk & Hanel, Petr & Rosa, Julio Miguel, 2011. "Evaluating the impact of R&D tax credits on innovation: A microeconometric study on Canadian firms," Research Policy, Elsevier, vol. 40(2), pages 217-229, March.
    19. James Alm & Yongzheng Liu, 2014. "China's Tax-for-Fee Reform and Village Inequality," Oxford Development Studies, Taylor & Francis Journals, vol. 42(1), pages 38-64, March.
    20. Hottenrott, Hanna & Lopes-Bento, Cindy, 2014. "(International) R&D collaboration and SMEs: The effectiveness of targeted public R&D support schemes," Research Policy, Elsevier, vol. 43(6), pages 1055-1066.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:151:y:2009:i:1:p:33-46. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.