IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1803.09015.html
   My bibliography  Save this paper

Difference-in-Differences with Multiple Time Periods

Author

Listed:
  • Brantly Callaway
  • Pedro H. C. Sant'Anna

Abstract

In this article, we consider identification, estimation, and inference procedures for treatment effect parameters using Difference-in-Differences (DiD) with (i) multiple time periods, (ii) variation in treatment timing, and (iii) when the "parallel trends assumption" holds potentially only after conditioning on observed covariates. We show that a family of causal effect parameters are identified in staggered DiD setups, even if differences in observed characteristics create non-parallel outcome dynamics between groups. Our identification results allow one to use outcome regression, inverse probability weighting, or doubly-robust estimands. We also propose different aggregation schemes that can be used to highlight treatment effect heterogeneity across different dimensions as well as to summarize the overall effect of participating in the treatment. We establish the asymptotic properties of the proposed estimators and prove the validity of a computationally convenient bootstrap procedure to conduct asymptotically valid simultaneous (instead of pointwise) inference. Finally, we illustrate the relevance of our proposed tools by analyzing the effect of the minimum wage on teen employment from 2001--2007. Open-source software is available for implementing the proposed methods.

Suggested Citation

  • Brantly Callaway & Pedro H. C. Sant'Anna, 2018. "Difference-in-Differences with Multiple Time Periods," Papers 1803.09015, arXiv.org, revised Dec 2020.
  • Handle: RePEc:arx:papers:1803.09015
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1803.09015
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Bruno Ferman & Cristine Pinto, 2019. "Inference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 452-467, July.
    3. MacKinnon, James G. & Webb, Matthew D., 2020. "Randomization inference for difference-in-differences with few treated clusters," Journal of Econometrics, Elsevier, vol. 218(2), pages 435-450.
    4. Escanciano, J. Carlos, 2006. "A Consistent Diagnostic Test For Regression Models Using Projections," Econometric Theory, Cambridge University Press, vol. 22(6), pages 1030-1051, December.
    5. Victor Chernozhukov & Iván Fernández‐Val & Ye Luo, 2018. "The Sorted Effects Method: Discovering Heterogeneous Effects Beyond Their Averages," Econometrica, Econometric Society, vol. 86(6), pages 1911-1938, November.
    6. Jonathan Meer & Jeremy West, 2016. "Effects of the Minimum Wage on Employment Dynamics," Journal of Human Resources, University of Wisconsin Press, vol. 51(2), pages 500-522.
    7. Arindrajit Dube & T. William Lester & Michael Reich, 2016. "Minimum Wage Shocks, Employment Flows, and Labor Market Frictions," Journal of Labor Economics, University of Chicago Press, vol. 34(3), pages 663-704.
    8. James G. MacKinnon & Matthew D. Webb, 2018. "The wild bootstrap for few (treated) clusters," Econometrics Journal, Royal Economic Society, vol. 21(2), pages 114-135, June.
    9. Ai, Chunrong & Chen, Xiaohong, 2012. "The semiparametric efficiency bound for models of sequential moment restrictions containing unknown functions," Journal of Econometrics, Elsevier, vol. 170(2), pages 442-457.
    10. Irene Botosaru & Federico H. Gutierrez, 2018. "Difference‐in‐differences when the treatment status is observed in only one period," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 33(1), pages 73-90, January.
    11. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    12. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    13. Murphy S.A. & van der Laan M.J. & Robins J.M., 2001. "Marginal Mean Models for Dynamic Regimes," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1410-1423, December.
    14. Wenceslao González-Manteiga & Rosa Crujeiras, 2013. "An updated review of Goodness-of-Fit tests for regression models," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 22(3), pages 361-411, September.
    15. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    16. Timothy G. Conley & Christopher R. Taber, 2011. "Inference with "Difference in Differences" with a Small Number of Policy Changes," The Review of Economics and Statistics, MIT Press, vol. 93(1), pages 113-125, February.
    17. Jacobson, Louis S & LaLonde, Robert J & Sullivan, Daniel G, 1993. "Earnings Losses of Displaced Workers," American Economic Review, American Economic Association, vol. 83(4), pages 685-709, September.
    18. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(1), pages 1-19.
    19. Wooldridge, Jeffrey M., 2007. "Inverse probability weighted estimation for general missing data problems," Journal of Econometrics, Elsevier, vol. 141(2), pages 1281-1301, December.
    20. David H. Autor & William R. Kerr & Adriana D. Kugler, 2007. "Do Employment Protections Reduce Productivity? Evidence from U.S. States," NBER Working Papers 12860, National Bureau of Economic Research, Inc.
    21. Sant’Anna, Pedro H.C. & Song, Xiaojun, 2019. "Specification tests for the propensity score," Journal of Econometrics, Elsevier, vol. 210(2), pages 379-404.
    22. Heckman, James J. & Humphries, John Eric & Veramendi, Gregory, 2016. "Dynamic treatment effects," Journal of Econometrics, Elsevier, vol. 191(2), pages 276-292.
    23. Newey, Whitney K, 1994. "The Asymptotic Variance of Semiparametric Estimators," Econometrica, Econometric Society, vol. 62(6), pages 1349-1382, November.
    24. Herman J. Bierens & Werner Ploberger, 1997. "Asymptotic Theory of Integrated Conditional Moment Tests," Econometrica, Econometric Society, vol. 65(5), pages 1129-1152, September.
    25. S Yang & P Ding, 2018. "Asymptotic inference of causal effects with observational studies trimmed by the estimated propensity scores," Biometrika, Biometrika Trust, vol. 105(2), pages 487-493.
    26. David Neumark & William Wascher, 1992. "Employment Effects of Minimum and Subminimum Wages: Panel Data on State Minimum Wage Laws," ILR Review, Cornell University, ILR School, vol. 46(1), pages 55-81, October.
    27. Xiaohong Chen & Oliver Linton & Ingrid Van Keilegom, 2003. "Estimation of Semiparametric Models when the Criterion Function Is Not Smooth," Econometrica, Econometric Society, vol. 71(5), pages 1591-1608, September.
    28. Carlos Escanciano, J., 2008. "Joint and marginal specification tests for conditional mean and variance models," Journal of Econometrics, Elsevier, vol. 143(1), pages 74-87, March.
    29. Alberto Abadie & Susan Athey & Guido W Imbens & Jeffrey M Wooldridge, 2023. "When Should You Adjust Standard Errors for Clustering?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(1), pages 1-35.
    30. Matias Busso & John DiNardo & Justin McCrary, 2014. "New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators," The Review of Economics and Statistics, MIT Press, vol. 96(5), pages 885-897, December.
    31. Richard Blundell & Monica Costa Dias & Costas Meghir & John Van Reenen, 2004. "Evaluating the Employment Impact of a Mandatory Job Search Program," Journal of the European Economic Association, MIT Press, vol. 2(4), pages 569-606, June.
    32. Kline Patrick & Santos Andres, 2012. "A Score Based Approach to Wild Bootstrap Inference," Journal of Econometric Methods, De Gruyter, vol. 1(1), pages 23-41, August.
    33. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    34. Malani, Anup & Reif, Julian, 2015. "Interpreting pre-trends as anticipation: Impact on estimated treatment effects from tort reform," Journal of Public Economics, Elsevier, vol. 124(C), pages 1-17.
    35. James G. MacKinnon & Matthew D. Webb, 2019. "Wild Bootstrap Randomization Inference for Few Treated Clusters," Advances in Econometrics, in: The Econometrics of Complex Survey Data, volume 39, pages 61-85, Emerald Group Publishing Limited.
    36. A. Belloni & V. Chernozhukov & I. Fernández‐Val & C. Hansen, 2017. "Program Evaluation and Causal Inference With High‐Dimensional Data," Econometrica, Econometric Society, vol. 85, pages 233-298, January.
    37. Laporte, Audrey & Windmeijer, Frank, 2005. "Estimation of panel data models with binary indicators when treatment effects are not constant over time," Economics Letters, Elsevier, vol. 88(3), pages 389-396, September.
    38. Cheng, Guang & Yu, Zhuqing & Huang, Jianhua Z., 2013. "The cluster bootstrap consistency in generalized estimating equations," Journal of Multivariate Analysis, Elsevier, vol. 115(C), pages 33-47.
    39. S. A. Murphy, 2003. "Optimal dynamic treatment regimes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(2), pages 331-355, May.
    40. Ekaterina Jardim & Mark C. Long & Robert Plotnick & Emma van Inwegen & Jacob Vigdor & Hilary Wething, 2017. "Minimum Wage Increases, Wages, and Low-Wage Employment: Evidence from Seattle," NBER Working Papers 23532, National Bureau of Economic Research, Inc.
    41. Philip Oreopoulos & Till von Wachter & Andrew Heisz, 2012. "The Short- and Long-Term Career Effects of Graduating in a Recession," American Economic Journal: Applied Economics, American Economic Association, vol. 4(1), pages 1-29, January.
    42. Susan Athey & Guido W. Imbens, 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models," Econometrica, Econometric Society, vol. 74(2), pages 431-497, March.
    43. Shakeeb Khan & Elie Tamer, 2010. "Irregular Identification, Support Conditions, and Inverse Weight Estimation," Econometrica, Econometric Society, vol. 78(6), pages 2021-2042, November.
    44. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    45. Freyberger, Joachim & Rai, Yoshiyasu, 2018. "Uniform confidence bands: Characterization and optimality," Journal of Econometrics, Elsevier, vol. 204(1), pages 119-130.
    46. Jeffrey M. Wooldridge, 2003. "Cluster-Sample Methods in Applied Econometrics," American Economic Review, American Economic Association, vol. 93(2), pages 133-138, May.
    47. James Heckman & Hidehiko Ichimura & Jeffrey Smith & Petra Todd, 1998. "Characterizing Selection Bias Using Experimental Data," Econometrica, Econometric Society, vol. 66(5), pages 1017-1098, September.
    48. Donald, Stephen G. & Hsu, Yu-Chin, 2014. "Estimation and inference for distribution functions and quantile functions in treatment effect models," Journal of Econometrics, Elsevier, vol. 178(P3), pages 383-397.
    49. Bryan S. Graham & Cristine Campos De Xavier Pinto & Daniel Egel, 2012. "Inverse Probability Tilting for Moment Condition Models with Missing Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 1053-1079.
    50. Victor Chernozhukov & Iván Fernández‐Val & Jinyong Hahn & Whitney Newey, 2013. "Average and Quantile Effects in Nonseparable Panel Models," Econometrica, Econometric Society, vol. 81(2), pages 535-580, March.
    51. Justin McCrary, 2007. "The Effect of Court-Ordered Hiring Quotas on the Composition and Quality of Police," American Economic Review, American Economic Association, vol. 97(1), pages 318-353, March.
    52. Martha J. Bailey & Andrew Goodman-Bacon, 2015. "The War on Poverty's Experiment in Public Medicine: Community Health Centers and the Mortality of Older Americans," American Economic Review, American Economic Association, vol. 105(3), pages 1067-1104, March.
    53. Michelle Marcus & Pedro H. C. Sant’Anna, 2021. "The Role of Parallel Trends in Event Study Settings: An Application to Environmental Economics," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 8(2), pages 235-275.
    54. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    55. Barbara Sianesi, 2004. "An Evaluation of the Swedish System of Active Labor Market Programs in the 1990s," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 133-155, February.
    56. Stéphane Bonhomme & Ulrich Sauder, 2011. "Recovering Distributions in Difference-in-Differences Models: A Comparison of Selective and Comprehensive Schooling," The Review of Economics and Statistics, MIT Press, vol. 93(2), pages 479-494, May.
    57. Anna, Petrenko, 2016. "Мaркування готової продукції як складова частина інформаційного забезпечення маркетингової діяльності підприємств овочепродуктового підкомплексу," Agricultural and Resource Economics: International Scientific E-Journal, Agricultural and Resource Economics: International Scientific E-Journal, vol. 2(1), March.
    58. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    59. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    60. Brantly Callaway & Tong Li, 2019. "Quantile treatment effects in difference in differences models with panel data," Quantitative Economics, Econometric Society, vol. 10(4), pages 1579-1618, November.
    61. Dale Belman & Paul J. Wolfson, 2014. "What Does the Minimum Wage Do?," Books from Upjohn Press, W.E. Upjohn Institute for Employment Research, number wdmwd, December.
    62. Stinchcombe, Maxwell B. & White, Halbert, 1998. "Consistent Specification Testing With Nuisance Parameters Present Only Under The Alternative," Econometric Theory, Cambridge University Press, vol. 14(3), pages 295-325, June.
    63. William Wascher & David Neumark, 2000. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania: Comment," American Economic Review, American Economic Association, vol. 90(5), pages 1362-1396, December.
    64. Ai, Chunrong & Chen, Xiaohong, 2007. "Estimation of possibly misspecified semiparametric conditional moment restriction models with different conditioning variables," Journal of Econometrics, Elsevier, vol. 141(1), pages 5-43, November.
    65. Jaap H. Abbring & Gerard J. van den Berg, 2003. "The Nonparametric Identification of Treatment Effects in Duration Models," Econometrica, Econometric Society, vol. 71(5), pages 1491-1517, September.
    66. Chunrong Ai & Xiaohong Chen, 2003. "Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions," Econometrica, Econometric Society, vol. 71(6), pages 1795-1843, November.
    67. John Schmitt, 2013. "Why Does the Minimum Wage Have No Discernible Effect on Employment?," CEPR Reports and Issue Briefs 2013-04, Center for Economic and Policy Research (CEPR).
    68. Jeffrey M. Wooldridge, 2005. "Fixed-Effects and Related Estimators for Correlated Random-Coefficient and Treatment-Effect Panel Data Models," The Review of Economics and Statistics, MIT Press, vol. 87(2), pages 385-390, May.
    69. Jing Qin & And Biao Zhang, 2008. "Empirical‐likelihood‐based difference‐in‐differences estimators," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(2), pages 329-349, April.
    70. Henry S. Farber, 2017. "Employment, Hours, and Earnings Consequences of Job Loss: US Evidence from the Displaced Workers Survey," Journal of Labor Economics, University of Chicago Press, vol. 35(S1), pages 235-272.
    71. Wooldridge, Jeffrey M., 2005. "Violating Ignorability Of Treatment By Controlling For Too Many Factors," Econometric Theory, Cambridge University Press, vol. 21(5), pages 1026-1028, October.
    72. David H. Autor & William R. Kerr & Adriana D. Kugler, 2007. "Does Employment Protection Reduce Productivity? Evidence From US States," Economic Journal, Royal Economic Society, vol. 117(521), pages 189-217, June.
    73. Arindrajit Dube & T. William Lester & Michael Reich, 2010. "Minimum Wage Effects Across State Borders: Estimates Using Contiguous Counties," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 945-964, November.
    74. Callaway, Brantly & Li, Tong & Oka, Tatsushi, 2018. "Quantile treatment effects in difference in differences models under dependence restrictions and with only two time periods," Journal of Econometrics, Elsevier, vol. 206(2), pages 395-413.
    75. Bierens, Herman J., 1982. "Consistent model specification tests," Journal of Econometrics, Elsevier, vol. 20(1), pages 105-134, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brantly Callaway & Pedro H. C. Sant'Anna, 2018. "Difference-in-Differences with Multiple Time Periods and an Application on the Minimum Wage and Employment," DETU Working Papers 1804, Department of Economics, Temple University.
    2. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    3. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    4. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    5. Sant’Anna, Pedro H.C. & Song, Xiaojun, 2019. "Specification tests for the propensity score," Journal of Econometrics, Elsevier, vol. 210(2), pages 379-404.
    6. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    7. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    8. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    9. Brantly Callaway, 2022. "Difference-in-Differences for Policy Evaluation," Papers 2203.15646, arXiv.org.
    10. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    11. Bruno Ferman, 2023. "Inference in difference‐in‐differences: How much should we trust in independent clusters?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(3), pages 358-369, April.
    12. Callaway, Brantly & Karami, Sonia, 2023. "Treatment effects in interactive fixed effects models with a small number of time periods," Journal of Econometrics, Elsevier, vol. 233(1), pages 184-208.
    13. Pedro H. C. Sant'Anna, 2016. "Program Evaluation with Right-Censored Data," Papers 1604.02642, arXiv.org.
    14. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    15. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    16. Arindrajit Dube & Attila Lindner, 2024. "Minimum Wages in the 21st Century," RF Berlin - CReAM Discussion Paper Series 2524, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    17. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    18. Schumann, Mathias, 2017. "The effects of minimum wages on firm-financed apprenticeship training," Labour Economics, Elsevier, vol. 47(C), pages 163-181.
    19. Michelle Marcus & Pedro H. C. Sant’Anna, 2021. "The Role of Parallel Trends in Event Study Settings: An Application to Environmental Economics," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 8(2), pages 235-275.
    20. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.

    More about this item

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • J23 - Labor and Demographic Economics - - Demand and Supply of Labor - - - Labor Demand
    • J38 - Labor and Demographic Economics - - Wages, Compensation, and Labor Costs - - - Public Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1803.09015. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.