IDEAS home Printed from https://ideas.repec.org/a/spr/aphecp/v14y2016i4d10.1007_s40258-016-0249-y.html
   My bibliography  Save this article

Difference-in-Differences Method in Comparative Effectiveness Research: Utility with Unbalanced Groups

Author

Listed:
  • Huanxue Zhou

    (KMK Consulting, Inc.)

  • Christopher Taber

    (University of Wisconsin)

  • Steve Arcona

    (Novartis Pharmaceuticals Corporation)

  • Yunfeng Li

    (Novartis Pharmaceuticals Corporation)

Abstract

Background Comparative effectiveness research (CER) often includes observational studies utilizing administrative data. Multiple conditioning methods can be used for CER to adjust for group differences, including difference-in-differences (DiD) estimation. Objective This study presents DiD and demonstrates how to apply this conditioning method to estimate treatment outcomes in the CER setting by utilizing the MarketScan® Databases for multiple sclerosis (MS) patients receiving different therapies. Methods The sample included 6762 patients, with 363 in the Test Cohort [glatiramer acetate (GA) switched to fingolimod (FTY)] and 6399 in the Control Cohort (GA only, no switch) from a US administrative claims database. A trend analysis was conducted to rule out concerns regarding regression to the mean and to compare relapse rates among treatment cohorts. DiD analysis was used to enable comparisons among the Test and Control Cohorts. Logistic regression was used to estimate the probability of relapse after switching from GA to FTY, and to compare group differences in the pre- and post-index periods. Results Crude DiD analysis showed that in the pre-index period more patients in the Test Cohort experienced an MS relapse and had a higher mean number of relapses than in the Control Cohort. During the pre-index period, numeric and relative data for MS relapses in patients in the Test Cohort were significantly higher than in the Control Cohort, while no significant between-group differences emerged during the post-index period. Generalized linear modeling with DiD regression estimation showed that the mean number of MS relapses decreased significantly in the post-index period among patients in the Test Cohort compared with patients in the Control Cohort. Conclusion In this study, an MS population was utilized to demonstrate how DiD can be applied to estimate treatment effects in a heterogeneous population, where the Test and Control Cohorts varied greatly. The results show that DiD offers a robust method for comparing diverse cohorts when other risk-adjustment methods may not be adequate.

Suggested Citation

  • Huanxue Zhou & Christopher Taber & Steve Arcona & Yunfeng Li, 2016. "Difference-in-Differences Method in Comparative Effectiveness Research: Utility with Unbalanced Groups," Applied Health Economics and Health Policy, Springer, vol. 14(4), pages 419-429, August.
    • Handle: RePEc:spr:aphecp:v:14:y:2016:i:4:d:10.1007_s40258-016-0249-y
    DOI: 10.1007/s40258-016-0249-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40258-016-0249-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40258-016-0249-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    2. James J. Heckman & Jeffrey A. Smith, 1999. "The Pre-Program Earnings Dip and the Determinants of Participation in a Social Program: Implications for Simple Program Evaluation Strategies," NBER Working Papers 6983, National Bureau of Economic Research, Inc.
    3. Heckman, James J & Smith, Jeffrey A, 1999. "The Pre-programme Earnings Dip and the Determinants of Participation in a Social Programme. Implications for Simple Programme Evaluation Strategies," Economic Journal, Royal Economic Society, vol. 109(457), pages 313-348, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Li Hao, 2022. "Impact of Relaxing the Hukou Constraints on Return Migration Intentions: Evidence from China," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 41(2), pages 583-607, April.
    2. Miraldo, Marisa & Propper, Carol & Williams, Rachael I., 2018. "The impact of publicly subsidised health insurance on access, behavioural risk factors and disease management," Social Science & Medicine, Elsevier, vol. 217(C), pages 135-151.
    3. Song, Mingjie & Wu, Yuling & Chen, Lu, 2020. "Does the land titling program promote rural housing land transfer in China? Evidence from household surveys in Hubei Province," Land Use Policy, Elsevier, vol. 97(C).
    4. Pécastaing, Nicolas & Chávez, Carlos, 2020. "The impact of El Niño phenomenon on dry forest-dependent communities' welfare in the northern coast of Peru," Ecological Economics, Elsevier, vol. 178(C).
    5. Vy Thi Nhat Nguyen & Takashi Zaitsu & Akiko Oshiro & Tai Tan Tran & Yen Hoang Thi Nguyen & Yoko Kawaguchi & Jun Aida, 2021. "Impact of School-Based Oral Health Education on Vietnamese Adolescents: A 6-Month Study," IJERPH, MDPI, vol. 18(5), pages 1-12, March.
    6. Jessica Yu & Paul Gustafson & Martino Tran & Michael Brauer, 2022. "Assessing Trade-Offs and Optimal Ranges of Density for Life Expectancy and 12 Causes of Mortality in Metro Vancouver, Canada, 1990–2016," IJERPH, MDPI, vol. 19(5), pages 1-17, March.
    7. Ambarish Dutta & Sarthak Pattanaik & Rajendra Choudhury & Pritish Nanda & Suvanand Sahu & Rajendra Panigrahi & Bijaya K Padhi & Krushna Chandra Sahoo & P R Mishra & Pinaki Panigrahi & Daisy Lekharu & , 2018. "Impact of involvement of non-formal health providers on TB case notification among migrant slum-dwelling populations in Odisha, India," PLOS ONE, Public Library of Science, vol. 13(5), pages 1-16, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabrielle Wills, 2016. "Principal leadership changes in South Africa: Investigating their consequences for school performance," Working Papers 01/2016, Stellenbosch University, Department of Economics.
    2. Peter Bäckström & Erika Sandow & Olle Westerlund, 2016. "Commuting and timing of retirement," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 56(1), pages 125-152, January.
    3. Stenberg, Anders & Westerlund, Olle, 2016. "Flexibility at a cost – Should governments stimulate tertiary education for adults?," The Journal of the Economics of Ageing, Elsevier, vol. 7(C), pages 69-86.
    4. Katherine Baicker & Theodore Svoronos, 2019. "Testing the Validity of the Single Interrupted Time Series Design," NBER Working Papers 26080, National Bureau of Economic Research, Inc.
    5. Anders Stenberg & Xavier Luna & Olle Westerlund, 2014. "Does Formal Education for Older Workers Increase Earnings? — Evidence Based on Rich Data and Long-term Follow-up," LABOUR, CEIS, vol. 28(2), pages 163-189, June.
    6. Regner, Hakan, 2002. "A nonexperimental evaluation of training programs for the unemployed in Sweden," Labour Economics, Elsevier, vol. 9(2), pages 187-206, April.
    7. Lechner Michael & Miquel Ruth & Wunsch Conny, 2007. "The Curse and Blessing of Training the Unemployed in a Changing Economy: The Case of East Germany After Unification," German Economic Review, De Gruyter, vol. 8(4), pages 468-509, December.
    8. Robert S. Chirinko & Daniel J. Wilson, 2023. "Job Creation Tax Credits, Fiscal Foresight, and Job Growth: Evidence from US States," National Tax Journal, University of Chicago Press, vol. 76(3), pages 481-523.
    9. Reinowski, Eva & Schultz, Birgit & Wiemers, Jürgen, 2004. "Evaluation of Further Training Programmes in Saxony from 1989 to 2001," IWH Discussion Papers 188/2004, Halle Institute for Economic Research (IWH), revised 2004.
    10. Raphael W. Bostic & Allen C. Prohofsky, 2006. "Enterprise Zones and Individual Welfare: A Case Study of California," Journal of Regional Science, Wiley Blackwell, vol. 46(2), pages 175-203, May.
    11. Mario Liebensteiner, 2014. "Estimating the Income Gain of Seasonal Labor Migration," Review of Development Economics, Wiley Blackwell, vol. 18(4), pages 667-680, November.
    12. Martins, Pedro S., 2017. "Can Non-Cognitive Skills Programs Improve Achievement? Quasi-Experimental Evidence from EPIS," GLO Discussion Paper Series 105, Global Labor Organization (GLO).
    13. Robert S. Chirinko & Daniel J. Wilson, 2023. "Fiscal Foresight and Perverse Distortions to Firm Behavior: Anticipatory Dips and Compensating Rebounds," Working Paper Series 2021-15, Federal Reserve Bank of San Francisco.
    14. Apeti, Ablam Estel & N’Doua, Bossoma Doriane, 2023. "The impact of timber regulations on timber and timber product trade," Ecological Economics, Elsevier, vol. 213(C).
    15. Knut Løyland & Oddbjørn Raaum & Gaute Torsvik & Arnstein Øvrum, 2024. "Evaluating compliance gains of expanding tax enforcement," Economica, London School of Economics and Political Science, vol. 91(361), pages 142-162, January.
    16. Alpert, Abby, 2016. "The anticipatory effects of Medicare Part D on drug utilization," Journal of Health Economics, Elsevier, vol. 49(C), pages 28-45.
    17. Hagen, Tobias, 2016. "Econometric evaluation of a placement coaching program for recipients of disability insurance benefits in Switzerland," Working Paper Series 10, Frankfurt University of Applied Sciences, Faculty of Business and Law.
    18. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    19. Gueorgui Kambourov & Iourii Manovskii & Miana Plesca, 2020. "Occupational mobility and the returns to training," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 53(1), pages 174-211, February.
    20. Andersson, Fredrik W. & Holzer, Harry J. & Lane, Julia & Rosenblum, David & Smith, Jeffrey A., 2013. "Does Federally-Funded Job Training Work? Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms," IZA Discussion Papers 7621, Institute of Labor Economics (IZA).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:aphecp:v:14:y:2016:i:4:d:10.1007_s40258-016-0249-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.