IDEAS home Printed from https://ideas.repec.org/a/bla/jecsur/v32y2018i3p878-915.html
   My bibliography  Save this article

Causal Inference On Education Policies: A Survey Of Empirical Studies Using Pisa, Timss And Pirls

Author

Listed:
  • José M. Cordero
  • Víctor Cristóbal
  • Daniel Santín

Abstract

The identification of the causal effects of educational policies is the top priority in recent education economics literature. As a result, a shift can be observed in the strategies of empirical studies. They have moved from the use of standard multivariate statistical methods, which identify correlations or associations between variables only, to more complex econometric strategies, which can help to identify causal relationships. However, exogenous variations in databases have to be identified in order to apply causal inference techniques. This is a far from straightforward task. For this reason, this paper provides an extensive and comprehensive overview of the literature using quasi†experimental techniques applied to three well†known international large†scale comparative assessments, such as PISA, PIRLS or TIMSS, over the period 2004–2016. In particular, we review empirical studies employing instrumental variables, regression discontinuity designs, difference in differences and propensity score matching to the above databases. Additionally, we provide a detailed summary of estimation strategies, issues treated and profitability in terms of the quality of publications to encourage further potential evaluations. The paper concludes with some operational recommendations for prospective researchers in the field.

Suggested Citation

  • José M. Cordero & Víctor Cristóbal & Daniel Santín, 2018. "Causal Inference On Education Policies: A Survey Of Empirical Studies Using Pisa, Timss And Pirls," Journal of Economic Surveys, Wiley Blackwell, vol. 32(3), pages 878-915, July.
  • Handle: RePEc:bla:jecsur:v:32:y:2018:i:3:p:878-915
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/joes.12217
    Download Restriction: no

    References listed on IDEAS

    as
    1. Andreas Ammermueller & Jörn-Steffen Pischke, 2009. "Peer Effects in European Primary Schools: Evidence from the Progress in International Reading Literacy Study," Journal of Labor Economics, University of Chicago Press, vol. 27(3), pages 315-348, July.
    2. Gamboa, Luis Fernando & Rodríguez Acosta, Mauricio & García Suaza, Andrés, 2013. "Differences in motivations and academic achievement," Revista Lecturas de Economía, Universidad de Antioquia - CIE, issue 78, pages 9-44, March.
    3. Oliver Falck & Ludger Woessmann, 2013. "School competition and students’ entrepreneurial intentions: international evidence using historical Catholic roots of private schooling," Small Business Economics, Springer, vol. 40(2), pages 459-478, February.
    4. Brindusa Anghel & Antonio Cabrales & Jorge Sainz & Ismael Sanz, 2015. "Publicizing the results of standardized external tests: does it have an effect on school outcomes?," IZA Journal of European Labor Studies, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-20, December.
    5. Martin Schlotter & Guido Schwerdt & Ludger Woessmann, 2011. "Econometric methods for causal evaluation of education policies and practices: a non-technical guide," Education Economics, Taylor & Francis Journals, vol. 19(2), pages 109-137.
    6. Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.), 2011. "Handbook of the Economics of Education," Handbook of the Economics of Education, Elsevier, edition 1, volume 4, number 4, June.
    7. Isphording, Ingo E. & Piopiunik, Marc & Rodríguez-Planas, Núria, 2016. "Speaking in numbers: The effect of reading performance on math performance among immigrants," Economics Letters, Elsevier, vol. 139(C), pages 52-56.
    8. Eric A. Hanushek & Ludger Wössmann, 2006. "Does Educational Tracking Affect Performance and Inequality? Differences- in-Differences Evidence Across Countries," Economic Journal, Royal Economic Society, vol. 116(510), pages 63-76, March.
    9. Schwerdt, Guido & Wuppermann, Amelie C., 2011. "Is traditional teaching really all that bad? A within-student between-subject approach," Economics of Education Review, Elsevier, vol. 30(2), pages 365-379, April.
    10. Wo[ss]mann, Ludger & West, Martin, 2006. "Class-size effects in school systems around the world: Evidence from between-grade variation in TIMSS," European Economic Review, Elsevier, vol. 50(3), pages 695-736, April.
    11. David Kiss, 2013. "Are immigrants and girls graded worse? Results of a matching approach," Education Economics, Taylor & Francis Journals, vol. 21(5), pages 447-463, December.
    12. Martin R. West & Ludger Woessmann, 2010. "'Every Catholic Child in a Catholic School': Historical Resistance to State Schooling, Contemporary Private Competition and Student Achievement across Countries," Economic Journal, Royal Economic Society, vol. 120(546), pages 229-255, August.
    13. James J. Heckman, 1976. "The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models," NBER Chapters,in: Annals of Economic and Social Measurement, Volume 5, number 4, pages 475-492 National Bureau of Economic Research, Inc.
    14. Edwards, Sebastian & Garcia Marin, Alvaro, 2015. "Constitutional rights and education: An international comparative study," Journal of Comparative Economics, Elsevier, vol. 43(4), pages 938-955.
    15. Kosuke Imai & David A. van Dyk, 2004. "Causal Inference With General Treatment Regimes: Generalizing the Propensity Score," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 854-866, January.
    16. Piopiunik, Marc, 2014. "The effects of early tracking on student performance: Evidence from a school reform in Bavaria," Economics of Education Review, Elsevier, vol. 42(C), pages 12-33.
    17. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    18. repec:spr:portec:v:1:y:2002:i:2:d:10.1007_s10258-002-0010-3 is not listed on IDEAS
    19. Vandenberghe, V. & Robin, S., 2004. "Evaluating the effectiveness of private education across countries: a comparison of methods," Labour Economics, Elsevier, vol. 11(4), pages 487-506, August.
    20. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, February.
    21. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    22. Shahidur R. Khandker & Gayatri B. Koolwal & Hussain A. Samad, 2010. "Handbook on Impact Evaluation : Quantitative Methods and Practices," World Bank Publications, The World Bank, number 2693.
    23. Steven G. Rivkin & Jeffrey C. Schiman, 2015. "Instruction Time, Classroom Quality, and Academic Achievement," Economic Journal, Royal Economic Society, vol. 125(588), pages 425-448, November.
    24. Paul R. Rosenbaum, 1986. "Dropping out of High School in the United States: An Observational Study," Journal of Educational and Behavioral Statistics, , vol. 11(3), pages 207-224, September.
    25. Sergio Perelman & Daniel Santin, 2011. "Measuring educational efficiency at student level with parametric stochastic distance functions: an application to Spanish PISA results," Education Economics, Taylor & Francis Journals, vol. 19(1), pages 29-49.
    26. Dronkers, Jaap & Avram, S, 2009. "A cross-national analysis of the relations between school choice and effectiveness differences between private-dependent and public schools," MPRA Paper 23911, University Library of Munich, Germany.
    27. Hanushek, Eric A. & Link, Susanne & Woessmann, Ludger, 2013. "Does school autonomy make sense everywhere? Panel estimates from PISA," Journal of Development Economics, Elsevier, vol. 104(C), pages 212-232.
    28. Victor Lavy, 2015. "Do Differences in Schools' Instruction Time Explain International Achievement Gaps? Evidence from Developed and Developing Countries," Economic Journal, Royal Economic Society, vol. 125(588), pages 397-424, November.
    29. Ponzo, Michela & Scoppa, Vincenzo, 2014. "The long-lasting effects of school entry age: Evidence from Italian students," Journal of Policy Modeling, Elsevier, vol. 36(3), pages 578-599.
    30. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, March.
    31. Eric A. Hanushek, 1979. "Conceptual and Empirical Issues in the Estimation of Educational Production Functions," Journal of Human Resources, University of Wisconsin Press, vol. 14(3), pages 351-388.
    32. West, Martin R. & Woessmann, Ludger, 2006. "Which school systems sort weaker students into smaller classes? International evidence," European Journal of Political Economy, Elsevier, vol. 22(4), pages 944-968, December.
    33. Julia Kuzmina & Martin Carnoy, 2016. "The effectiveness of vocational versus general secondary education: Evidence from the PISA 2012 for countries with early tracking," International Journal of Manpower, Emerald Group Publishing, vol. 37(1), pages 2-24, April.
    34. Kelly Bedard & Elizabeth Dhuey, 2006. "The Persistence of Early Childhood Maturity: International Evidence of Long-Run Age Effects," The Quarterly Journal of Economics, Oxford University Press, vol. 121(4), pages 1437-1472.
    35. Denny, Kevin & Oppedisano, Veruska, 2013. "The surprising effect of larger class sizes: Evidence using two identification strategies," Labour Economics, Elsevier, vol. 23(C), pages 57-65.
    36. Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.), 2011. "Handbook of the Economics of Education," Handbook of the Economics of Education, Elsevier, edition 1, volume 3, number 3, June.
    37. Betz, Timm, 2013. "Robust Estimation with Nonrandom Measurement Error and Weak Instruments," Political Analysis, Cambridge University Press, vol. 21(01), pages 86-96, December.
    38. Ponzo, Michela, 2013. "Does bullying reduce educational achievement? An evaluation using matching estimators," Journal of Policy Modeling, Elsevier, vol. 35(6), pages 1057-1078.
    39. Ludger Wößmann, 2005. "Educational production in Europe," Economic Policy, CEPR;CES;MSH, vol. 20(43), pages 445-504, July.
    40. Eva Crespo-Cebada & Francisco Pedraja-Chaparro & Daniel Santín, 2014. "Does school ownership matter? An unbiased efficiency comparison for regions of Spain," Journal of Productivity Analysis, Springer, vol. 41(1), pages 153-172, February.
    41. Elizabeth Ty Wilde & Robinson Hollister, 2007. "How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(3), pages 455-477.
    42. Cho, Insook, 2012. "The effect of teacher–student gender matching: Evidence from OECD countries," Economics of Education Review, Elsevier, vol. 31(3), pages 54-67.
    43. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, Oxford University Press, vol. 119(1), pages 249-275.
    44. Hanushek, Eric A. & Piopiunik, Marc & Wiederhold, Simon, 2014. "The Value of Smarter Teachers: International Evidence on Teacher Cognitive Skills and Student Performance," Discussion Papers in Economics 22032, University of Munich, Department of Economics.
    45. Petra E. Todd & Kenneth I. Wolpin, 2003. "On The Specification and Estimation of The Production Function for Cognitive Achievement," Economic Journal, Royal Economic Society, vol. 113(485), pages 3-33, February.
    46. Wößmann, Ludger, 2007. "International evidence on school competition, autonomy, and accountability: A review," Munich Reprints in Economics 19649, University of Munich, Department of Economics.
    47. Bietenbeck, Jan, 2014. "Teaching practices and cognitive skills," Labour Economics, Elsevier, vol. 30(C), pages 143-153.
    48. Francisco Pedraja-Chaparro & Daniel Santín & Rosa Simancas, 2016. "The impact of immigrant concentration in schools on grade retention in Spain: a difference-in-differences approach," Applied Economics, Taylor & Francis Journals, vol. 48(21), pages 1978-1990, May.
    49. Jensen, Peter & Rasmussen, Astrid Würtz, 2011. "The effect of immigrant concentration in schools on native and immigrant children's reading and math skills," Economics of Education Review, Elsevier, vol. 30(6), pages 1503-1515.
    50. Joshua D. Angrist & Jörn-Steffen Pischke, 2014. "Mastering ’Metrics: The Path from Cause to Effect," Economics Books, Princeton University Press, edition 1, number 10363, December.
    51. Dinand Webbink, 2005. "Causal Effects in Education," Journal of Economic Surveys, Wiley Blackwell, vol. 19(4), pages 535-560, September.
    52. Steven G. Rivkin & Jeffrey C. Schiman, 2015. "Instruction Time, Classroom Quality, and Academic Achievement," Economic Journal, Royal Economic Society, vol. 0(588), pages 425-448, November.
    53. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, Oxford University Press, vol. 114(2), pages 533-575.
    54. Vardardottir, Arna, 2015. "The impact of classroom peers in a streaming system," Economics of Education Review, Elsevier, vol. 49(C), pages 110-128.
    55. Maciej Jakubowski, 2015. "Latent variables and propensity score matching: a simulation study with application to data from the Programme for International Student Assessment in Poland," Empirical Economics, Springer, vol. 48(3), pages 1287-1325, May.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jecsur:v:32:y:2018:i:3:p:878-915. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley Content Delivery). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0950-0804 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.