IDEAS home Printed from https://ideas.repec.org/a/spr/empeco/v44y2013i1p47-80.html
   My bibliography  Save this article

Propensity score matching and variations on the balancing test

Author

Listed:
  • Wang-Sheng Lee

    ()

Abstract

Balancing tests are diagnostics designed for use with propensity score methods, a widely used non-experimental approach in the evaluation literature. Such tests provide useful information on whether plausible counterfactuals have been created. Currently, multiple balancing tests exist in the literature but it is unclear which is the most useful. This article highlights the poor size properties of commonly employed balancing tests and attempts to shed some light on the link between the results of balancing tests and bias of the evaluation estimator. The simulation results suggest that in scenarios where the conditional independence assumption holds, a permutation version of the balancing test described in Dehejia and Wahba (Rev Econ Stat 84:151–161, 2002 ) can be useful in applied study. The proposed test has good size properties. In addition, the test appears to have good power for detecting a misspecification in the link function and some power for detecting an omission of relevant non-linear terms involving variables that are included at a lower order. Copyright Springer-Verlag 2013

Suggested Citation

  • Wang-Sheng Lee, 2013. "Propensity score matching and variations on the balancing test," Empirical Economics, Springer, vol. 44(1), pages 47-80, February.
  • Handle: RePEc:spr:empeco:v:44:y:2013:i:1:p:47-80 DOI: 10.1007/s00181-011-0481-0
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s00181-011-0481-0
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
    2. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, pages 1161-1189.
    3. Busso, Matias & DiNardo, John & McCrary, Justin, 2009. "New Evidence on the Finite Sample Properties of Propensity Score Matching and Reweighting Estimators," IZA Discussion Papers 3998, Institute for the Study of Labor (IZA).
    4. Bryan S. Graham & Cristine Campos De Xavier Pinto & Daniel Egel, 2012. "Inverse Probability Tilting for Moment Condition Models with Missing Data," Review of Economic Studies, Oxford University Press, vol. 79(3), pages 1053-1079.
    5. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    6. repec:adr:anecst:y:2008:i:91-92:p:10 is not listed on IDEAS
    7. Koenker, Roger & Yoon, Jungmo, 2009. "Parametric links for binary choice models: A Fisherian-Bayesian colloquy," Journal of Econometrics, Elsevier, vol. 152(2), pages 120-130, October.
    8. Millimet, Daniel L. & Tchernis, Rusty, 2009. "On the Specification of Propensity Scores, With Applications to the Analysis of Trade Policies," Journal of Business & Economic Statistics, American Statistical Association, vol. 27(3), pages 397-415.
    9. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    10. Heckman, J.J. & Hotz, V.J., 1988. "Choosing Among Alternative Nonexperimental Methods For Estimating The Impact Of Social Programs: The Case Of Manpower Training," University of Chicago - Economics Research Center 88-12, Chicago - Economics Research Center.
    11. Sekhon, Jasjeet S., 2011. "Multivariate and Propensity Score Matching Software with Automated Balance Optimization: The Matching package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i07).
    12. Harvey, A C, 1976. "Estimating Regression Models with Multiplicative Heteroscedasticity," Econometrica, Econometric Society, vol. 44(3), pages 461-465, May.
    13. Zhao, Zhong, 2008. "Sensitivity of propensity score methods to the specifications," Economics Letters, Elsevier, vol. 98(3), pages 309-319, March.
    14. Shaikh, Azeem M. & Simonsen, Marianne & Vytlacil, Edward J. & Yildiz, Nese, 2009. "A specification test for the propensity score using its distribution conditional on participation," Journal of Econometrics, Elsevier, vol. 151(1), pages 33-46, July.
    15. Abadie A., 2002. "Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 284-292, March.
    16. Jose C. Galdo & Jeffrey Smith & Dan Black, 2008. "Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced Data," Annals of Economics and Statistics, GENES, issue 91-92, pages 189-216.
    17. Sascha O. Becker & Andrea Ichino, 2002. "Estimation of average treatment effects based on propensity scores," Stata Journal, StataCorp LP, vol. 2(4), pages 358-377, November.
    18. Juan Jose Diaz & Sudhanshu Handa, 2006. "An Assessment of Propensity Score Matching as a Nonexperimental Impact Estimator: Evidence from Mexico’s PROGRESA Program," Journal of Human Resources, University of Wisconsin Press, vol. 41(2).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Junaid Ahmed & Mazhar Mughal, 2014. "How do consumption patterns of foreign and domestic remittance recipients and non recipients compare? Evidence from Pakistan," Courant Research Centre: Poverty, Equity and Growth - Discussion Papers 160, Courant Research Centre PEG.
    2. Dolton, Peter & Smith, Jeffrey A., 2011. "The Impact of the UK New Deal for Lone Parents on Benefit Receipt," IZA Discussion Papers 5491, Institute for the Study of Labor (IZA).
    3. Browning, Martin & Crossley, Thomas F., 2008. "The long-run cost of job loss as measured by consumption changes," Journal of Econometrics, Elsevier, pages 109-120.
    4. Ogutu, Sylvester Ochieng & Okello, Julius Juma & Otieno, David Jakinda, 2014. "Impact of Information and Communication Technology-Based Market Information Services on Smallholder Farm Input Use and Productivity: The Case of Kenya," World Development, Elsevier, vol. 64(C), pages 311-321.
    5. Tebkieta Alexandra TAPSOBA, 2017. "Poverty, disasters and remittances: do remittances and past disasters influence households’ resilience?," Working Papers 201708, CERDI.
    6. Martin Biewen & Bernd Fitzenberger & Aderonke Osikominu & Marie Paul, 2014. "The Effectiveness of Public-Sponsored Training Revisited: The Importance of Data and Methodological Choices," Journal of Labor Economics, University of Chicago Press, vol. 32(4), pages 837-897.
    7. Simone Bertoli & Francesca Marchetta, 2014. "Migration, Remittances and Poverty in Ecuador," Journal of Development Studies, Taylor & Francis Journals, pages 1067-1089.
    8. Michele Campolieti & Morley Gunderson & Jeffrey Smith, 2014. "The effect of vocational rehabilitation on the employment outcomes of disability insurance beneficiaries: new evidence from Canada," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 3(1), pages 1-29, December.
    9. Veronika Vynaryk & Aoife Hanley, 2015. "Effects of the Qms ISO 9000 Certification on Russian Manufacturing Companies," HSE Working papers WP BRP 39/MAN/2015, National Research University Higher School of Economics.
    10. Carlos A. Flores & Oscar A. Mitnik, 2013. "Comparing Treatments across Labor Markets: An Assessment of Nonexperimental Multiple-Treatment Strategies," The Review of Economics and Statistics, MIT Press, pages 1691-1707.
    11. Schreinemachers, Pepijn & Wu, Mei-huey & Uddin, Md. Nasir & Ahmad, Shahabuddin & Hanson, Peter, 2016. "Farmer training in off-season vegetables: Effects on income and pesticide use in Bangladesh," Food Policy, Elsevier, vol. 61(C), pages 132-140.
    12. Wanglin Ma & Awudu Abdulai, 2016. "Linking apple farmers to markets: Determinants and impacts of marketing contracts in China," China Agricultural Economic Review, Emerald Group Publishing, vol. 8(1), pages 2-21, February.
    13. repec:spr:astaws:v:11:y:2017:i:3:d:10.1007_s11943-017-0212-x is not listed on IDEAS
    14. Guo, Jianxin & Jin, Songqing & Chen, Lei & Wang, Min & Zhang, Junfeng & Sun, Sufen, 2015. "The Effects of Distance Education on Agricultural Performance and Household Income: Evidence from Suburban Beijing," 2015 AAEA & WAEA Joint Annual Meeting, July 26-28, San Francisco, California 205754, Agricultural and Applied Economics Association;Western Agricultural Economics Association.
    15. Kabunga, Nassul, 2014. "Adoption and Impact of Improved Cow Breeds on Household Welfare and Child Nutrition Outcomes: Empirical Evidence from Uganda," 88th Annual Conference, April 9-11, 2014, AgroParisTech, Paris, France 170517, Agricultural Economics Society.
    16. Sarah Hamersma & Carolyn Heinrich, 2008. "Temporary Help Service Firms' Use of Employer Tax Credits: Implications for Disadvantaged Workers' Labor Market Outcomes," Southern Economic Journal, Southern Economic Association, vol. 74(4), pages 1123-1148, April.
    17. John F. Helliwell & Shun Wang, 2015. "How was the Weekend? How the Social Context Underlies Weekend Effects in Happiness and other Emotions for US Workers," NBER Working Papers 21374, National Bureau of Economic Research, Inc.
    18. Gueorgui Kambourov & Iourii Manovskii & Miana Plesca, 2012. "Occupational Mobility and the Returns to Training," Working Papers tecipa-444, University of Toronto, Department of Economics.
    19. Dan Pan, 2014. "The Impact of Agricultural Extension on Farmer Nutrient Management Behavior in Chinese Rice Production: A Household-Level Analysis," Sustainability, MDPI, Open Access Journal, vol. 6(10), pages 1-22, September.
    20. Sánchez, Elmer, 2013. "Grado de inversión y flujos de inversión directa extranjera a economías emergentes," Working Papers 2013-010, Banco Central de Reserva del Perú.
    21. Tebkieta Alexandra Tapsoba, 2017. "Poverty, disasters and remittances: do remittances and past disasters influence households’ resilience?," Working Papers halshs-01512716, HAL.
    22. Alexandru MINEA & Jean-Louis COMBES & Weneyam Hippolyte BALIMA, 2015. "Sovereign Debt Risk in Emerging Countries: Does Inflation Targeting Adoption Make Any Difference?," Working Papers 201504, CERDI.
    23. Jose C. Galdo & Jeffrey Smith & Dan Black, 2008. "Bandwidth Selection and the Estimation of Treatment Effects with Unbalanced Data," Annals of Economics and Statistics, GENES, issue 91-92, pages 189-216.
    24. Jessica Aschemann-Witzel & Tino Bech-Larsen & Sara Capacci, 2016. "Do Target Groups Appreciate Being Targeted? An Exploration of Healthy Eating Policy Acceptance," Journal of Consumer Policy, Springer, vol. 39(3), pages 285-306, September.
    25. Sánchez, Elmer, 2013. "Grado de inversión y flujos de inversión directa extranjera a economías emergentes," Revista Estudios Económicos, Banco Central de Reserva del Perú, issue 26, pages 61-79.

    More about this item

    Keywords

    Matching; Propensity score; Balancing test; Permutation test; Monte Carlo simulation; C14; C99;

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C99 - Mathematical and Quantitative Methods - - Design of Experiments - - - Other

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:empeco:v:44:y:2013:i:1:p:47-80. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Sonal Shukla) or (Rebekah McClure). General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.