IDEAS home Printed from https://ideas.repec.org/a/bla/jecsur/v32y2018i3p878-915.html
   My bibliography  Save this article

Causal Inference On Education Policies: A Survey Of Empirical Studies Using Pisa, Timss And Pirls

Author

Listed:
  • José M. Cordero
  • Víctor Cristóbal
  • Daniel Santín

Abstract

The identification of the causal effects of educational policies is the top priority in recent education economics literature. As a result, a shift can be observed in the strategies of empirical studies. They have moved from the use of standard multivariate statistical methods, which identify correlations or associations between variables only, to more complex econometric strategies, which can help to identify causal relationships. However, exogenous variations in databases have to be identified in order to apply causal inference techniques. This is a far from straightforward task. For this reason, this paper provides an extensive and comprehensive overview of the literature using quasi†experimental techniques applied to three well†known international large†scale comparative assessments, such as PISA, PIRLS or TIMSS, over the period 2004–2016. In particular, we review empirical studies employing instrumental variables, regression discontinuity designs, difference in differences and propensity score matching to the above databases. Additionally, we provide a detailed summary of estimation strategies, issues treated and profitability in terms of the quality of publications to encourage further potential evaluations. The paper concludes with some operational recommendations for prospective researchers in the field.

Suggested Citation

  • José M. Cordero & Víctor Cristóbal & Daniel Santín, 2018. "Causal Inference On Education Policies: A Survey Of Empirical Studies Using Pisa, Timss And Pirls," Journal of Economic Surveys, Wiley Blackwell, vol. 32(3), pages 878-915, July.
  • Handle: RePEc:bla:jecsur:v:32:y:2018:i:3:p:878-915
    DOI: 10.1111/joes.12217
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/joes.12217
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Andreas Ammermueller & Jörn-Steffen Pischke, 2009. "Peer Effects in European Primary Schools: Evidence from the Progress in International Reading Literacy Study," Journal of Labor Economics, University of Chicago Press, vol. 27(3), pages 315-348, July.
    2. Gamboa, Luis Fernando & Rodríguez Acosta, Mauricio & García Suaza, Andrés, 2013. "Differences in motivations and academic achievement," Revista Lecturas de Economía, Universidad de Antioquia - CIE, issue 78, pages 9-44, March.
    3. Oliver Falck & Ludger Woessmann, 2013. "School competition and students’ entrepreneurial intentions: international evidence using historical Catholic roots of private schooling," Small Business Economics, Springer, vol. 40(2), pages 459-478, February.
    4. Steven G. Rivkin & Eric A. Hanushek & John F. Kain, 2005. "Teachers, Schools, and Academic Achievement," Econometrica, Econometric Society, vol. 73(2), pages 417-458, March.
    5. Brindusa Anghel & Antonio Cabrales & Jorge Sainz & Ismael Sanz, 2015. "Publicizing the results of standardized external tests: does it have an effect on school outcomes?," IZA Journal of European Labor Studies, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-20, December.
    6. Maria A. Cattaneo & Chantal Oggenfuss & Stefan C. Wolter, 2017. "The more, the better? The impact of instructional time on student performance," Education Economics, Taylor & Francis Journals, vol. 25(5), pages 433-445, September.
    7. Martin Schlotter & Guido Schwerdt & Ludger Woessmann, 2011. "Econometric methods for causal evaluation of education policies and practices: a non-technical guide," Education Economics, Taylor & Francis Journals, vol. 19(2), pages 109-137.
    8. Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.), 2011. "Handbook of the Economics of Education," Handbook of the Economics of Education, Elsevier, edition 1, volume 4, number 4, June.
    9. Isphording, Ingo E. & Piopiunik, Marc & Rodríguez-Planas, Núria, 2016. "Speaking in numbers: The effect of reading performance on math performance among immigrants," Economics Letters, Elsevier, vol. 139(C), pages 52-56.
    10. Hanushek, Eric A. & Woessmann, Ludger, 2011. "The Economics of International Differences in Educational Achievement," Handbook of the Economics of Education, in: Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.),Handbook of the Economics of Education, edition 1, volume 3, chapter 2, pages 89-200, Elsevier.
    11. J. Ignacio Garc, 2014. "Does grade retention affect students' achievement? Some evidence from Spain," Applied Economics, Taylor & Francis Journals, vol. 46(12), pages 1373-1392, April.
    12. Eric A. Hanushek & Ludger Wössmann, 2006. "Does Educational Tracking Affect Performance and Inequality? Differences- in-Differences Evidence Across Countries," Economic Journal, Royal Economic Society, vol. 116(510), pages 63-76, March.
    13. Schwerdt, Guido & Wuppermann, Amelie C., 2011. "Is traditional teaching really all that bad? A within-student between-subject approach," Economics of Education Review, Elsevier, vol. 30(2), pages 365-379, April.
    14. Wo[ss]mann, Ludger & West, Martin, 2006. "Class-size effects in school systems around the world: Evidence from between-grade variation in TIMSS," European Economic Review, Elsevier, vol. 50(3), pages 695-736, April.
    15. David Kiss, 2013. "Are immigrants and girls graded worse? Results of a matching approach," Education Economics, Taylor & Francis Journals, vol. 21(5), pages 447-463, December.
    16. Martin R. West & Ludger Woessmann, 2010. "'Every Catholic Child in a Catholic School': Historical Resistance to State Schooling, Contemporary Private Competition and Student Achievement across Countries," Economic Journal, Royal Economic Society, vol. 120(546), pages 229-255, August.
    17. James J. Heckman, 1976. "The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models," NBER Chapters, in: Annals of Economic and Social Measurement, Volume 5, number 4, pages 475-492, National Bureau of Economic Research, Inc.
    18. Edwards, Sebastian & Garcia Marin, Alvaro, 2015. "Constitutional rights and education: An international comparative study," Journal of Comparative Economics, Elsevier, vol. 43(4), pages 938-955.
    19. Kosuke Imai & David A. van Dyk, 2004. "Causal Inference With General Treatment Regimes: Generalizing the Propensity Score," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 854-866, January.
    20. Piopiunik, Marc, 2014. "The effects of early tracking on student performance: Evidence from a school reform in Bavaria," Economics of Education Review, Elsevier, vol. 42(C), pages 12-33.
    21. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    22. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    23. Vandenberghe, V. & Robin, S., 2004. "Evaluating the effectiveness of private education across countries: a comparison of methods," Labour Economics, Elsevier, vol. 11(4), pages 487-506, August.
    24. James Heckman & Salvador Navarro-Lozano, 2004. "Using Matching, Instrumental Variables, and Control Functions to Estimate Economic Choice Models," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 30-57, February.
    25. Patrick Puhani & Andrea Weber, 2007. "Does the early bird catch the worm?," Empirical Economics, Springer, vol. 32(2), pages 359-386, May.
    26. Shahidur R. Khandker & Gayatri B. Koolwal & Hussain A. Samad, . "Handbook on Impact Evaluation : Quantitative Methods and Practices," World Bank Publications, The World Bank, number 2693, Juni.
    27. Steven G. Rivkin & Jeffrey C. Schiman, 2015. "Instruction Time, Classroom Quality, and Academic Achievement," Economic Journal, Royal Economic Society, vol. 125(588), pages 425-448, November.
    28. Hendrik Jürges & Kerstin Schneider & Felix Büchel, 2005. "The Effect Of Central Exit Examinations On Student Achievement: Quasi-Experimental Evidence From TIMSS Germany," Journal of the European Economic Association, MIT Press, vol. 3(5), pages 1134-1155, September.
    29. Paul R. Rosenbaum, 1986. "Dropping out of High School in the United States: An Observational Study," Journal of Educational and Behavioral Statistics, , vol. 11(3), pages 207-224, September.
    30. Kristof de Witte & Laura López-Torres, 2015. "Efficiency in Education. A Review of Literature and a Way Forward," Working Papers 1501, Departament Empresa, Universitat Autònoma de Barcelona, revised Apr 2015.
    31. Sergio Perelman & Daniel Santin, 2011. "Measuring educational efficiency at student level with parametric stochastic distance functions: an application to Spanish PISA results," Education Economics, Taylor & Francis Journals, vol. 19(1), pages 29-49.
    32. Dronkers, Jaap & Avram, S, 2009. "A cross-national analysis of the relations between school choice and effectiveness differences between private-dependent and public schools," MPRA Paper 23911, University Library of Munich, Germany.
    33. Hanushek, Eric A. & Link, Susanne & Woessmann, Ludger, 2013. "Does school autonomy make sense everywhere? Panel estimates from PISA," Journal of Development Economics, Elsevier, vol. 104(C), pages 212-232.
    34. Victor Lavy, 2015. "Do Differences in Schools' Instruction Time Explain International Achievement Gaps? Evidence from Developed and Developing Countries," Economic Journal, Royal Economic Society, vol. 125(588), pages 397-424, November.
    35. Ponzo, Michela & Scoppa, Vincenzo, 2014. "The long-lasting effects of school entry age: Evidence from Italian students," Journal of Policy Modeling, Elsevier, vol. 36(3), pages 578-599.
    36. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    37. Eric A. Hanushek, 1979. "Conceptual and Empirical Issues in the Estimation of Educational Production Functions," Journal of Human Resources, University of Wisconsin Press, vol. 14(3), pages 351-388.
    38. Ruhose, Jens & Schwerdt, Guido, 2016. "Does early educational tracking increase migrant-native achievement gaps? Differences-in-differences evidence across countries," Economics of Education Review, Elsevier, vol. 52(C), pages 134-154.
    39. West, Martin R. & Woessmann, Ludger, 2006. "Which school systems sort weaker students into smaller classes? International evidence," European Journal of Political Economy, Elsevier, vol. 22(4), pages 944-968, December.
    40. Julia Kuzmina & Martin Carnoy, 2016. "The effectiveness of vocational versus general secondary education: Evidence from the PISA 2012 for countries with early tracking," International Journal of Manpower, Emerald Group Publishing, vol. 37(1), pages 2-24, April.
    41. Kelly Bedard & Elizabeth Dhuey, 2006. "The Persistence of Early Childhood Maturity: International Evidence of Long-Run Age Effects," The Quarterly Journal of Economics, Oxford University Press, vol. 121(4), pages 1437-1472.
    42. Paul J. Gertler & Sebastian Martinez & Patrick Premand & Laura B. Rawlings & Christel M. J. Vermeersch, 2016. "Impact Evaluation in Practice, Second Edition," World Bank Publications, The World Bank, number 25030, Juni.
    43. Oecd, 2016. "Fields of education, gender and the labour market," Education Indicators in Focus 45, OECD Publishing.
    44. Denny, Kevin & Oppedisano, Veruska, 2013. "The surprising effect of larger class sizes: Evidence using two identification strategies," Labour Economics, Elsevier, vol. 23(C), pages 57-65.
    45. Erik Hanushek & Stephen Machin & Ludger Woessmann (ed.), 2011. "Handbook of the Economics of Education," Handbook of the Economics of Education, Elsevier, edition 1, volume 3, number 3, June.
    46. Christina Felfe & Natalia Nollenberger & Núria Rodríguez-Planas, 2015. "Can’t buy mommy’s love? Universal childcare and children’s long-term cognitive development," Journal of Population Economics, Springer;European Society for Population Economics, vol. 28(2), pages 393-422, April.
    47. Elaine L. Hill & David J. G. Slusky, 2017. "Birth Spacing and Educational Outcomes," Advances in Health Economics and Health Services Research, in: Kristian Bolin & Björn Lindgren & Michael Grossman & Dorte Gyrd-Hansen & Tor Iversen & Robert Kaestn (ed.),Human Capital and Health Behavior, volume 25, pages 3-29, Emerald Publishing Ltd.
    48. Betz, Timm, 2013. "Robust Estimation with Nonrandom Measurement Error and Weak Instruments," Political Analysis, Cambridge University Press, vol. 21(1), pages 86-96, January.
    49. Ponzo, Michela, 2013. "Does bullying reduce educational achievement? An evaluation using matching estimators," Journal of Policy Modeling, Elsevier, vol. 35(6), pages 1057-1078.
    50. Eric A. Hanushek & Marc Piopiunik & Simon Wiederhold, 2019. "The Value of Smarter Teachers: International Evidence on Teacher Cognitive Skills and Student Performance," Journal of Human Resources, University of Wisconsin Press, vol. 54(4), pages 857-899.
    51. Charles T. Clotfelter & Helen F. Ladd & Jacob L. Vigdor, 2010. "Teacher Credentials and Student Achievement in High School: A Cross-Subject Analysis with Student Fixed Effects," Journal of Human Resources, University of Wisconsin Press, vol. 45(3).
    52. Ludger Wößmann, 2005. "Educational production in Europe," Economic Policy, CEPR;CES;MSH, vol. 20(43), pages 445-504, July.
    53. Oecd, 2016. "What influences spending on education?," Education Indicators in Focus 46, OECD Publishing.
    54. Eva Crespo-Cebada & Francisco Pedraja-Chaparro & Daniel Santín, 2014. "Does school ownership matter? An unbiased efficiency comparison for regions of Spain," Journal of Productivity Analysis, Springer, vol. 41(1), pages 153-172, February.
    55. Elizabeth Ty Wilde & Robinson Hollister, 2007. "How close is close enough? Evaluating propensity score matching using data from a class size reduction experiment," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 26(3), pages 455-477.
    56. Cho, Insook, 2012. "The effect of teacher–student gender matching: Evidence from OECD countries," Economics of Education Review, Elsevier, vol. 31(3), pages 54-67.
    57. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, Oxford University Press, vol. 119(1), pages 249-275.
    58. Petra E. Todd & Kenneth I. Wolpin, 2003. "On The Specification and Estimation of The Production Function for Cognitive Achievement," Economic Journal, Royal Economic Society, vol. 113(485), pages 3-33, February.
    59. Wößmann, Ludger, 2007. "International evidence on school competition, autonomy, and accountability: A review," Munich Reprints in Economics 19649, University of Munich, Department of Economics.
    60. ., 2016. "Demographics, education, health, and labor," Chapters, in: Economic Reform in Asia, chapter 8, pages 141-157, Edward Elgar Publishing.
    61. Clotfelter, Charles T. & Ladd, Helen F. & Vigdor, Jacob L., 2007. "Teacher credentials and student achievement: Longitudinal analysis with student fixed effects," Economics of Education Review, Elsevier, vol. 26(6), pages 673-682, December.
    62. Bietenbeck, Jan, 2014. "Teaching practices and cognitive skills," Labour Economics, Elsevier, vol. 30(C), pages 143-153.
    63. Francisco Pedraja-Chaparro & Daniel Santín & Rosa Simancas, 2016. "The impact of immigrant concentration in schools on grade retention in Spain: a difference-in-differences approach," Applied Economics, Taylor & Francis Journals, vol. 48(21), pages 1978-1990, May.
    64. Jensen, Peter & Rasmussen, Astrid Würtz, 2011. "The effect of immigrant concentration in schools on native and immigrant children's reading and math skills," Economics of Education Review, Elsevier, vol. 30(6), pages 1503-1515.
    65. Joshua D. Angrist & Jörn-Steffen Pischke, 2014. "Mastering ’Metrics: The Path from Cause to Effect," Economics Books, Princeton University Press, edition 1, number 10363.
    66. Dinand Webbink, 2005. "Causal Effects in Education," Journal of Economic Surveys, Wiley Blackwell, vol. 19(4), pages 535-560, September.
    67. ., 2016. "Education, individualism and inequality," Chapters, in: Individualism and Inequality, chapter 8, pages 126-144, Edward Elgar Publishing.
    68. Hendrik Jürges & Kerstin Schneider, 2004. "International Differences in Student Achievement: An Economic Perspective," German Economic Review, Verein für Socialpolitik, vol. 5(3), pages 357-380, August.
    69. Steven G. Rivkin & Jeffrey C. Schiman, 2015. "Instruction Time, Classroom Quality, and Academic Achievement," Economic Journal, Royal Economic Society, vol. 0(588), pages 425-448, November.
    70. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, Oxford University Press, vol. 114(2), pages 533-575.
    71. Vardardottir, Arna, 2015. "The impact of classroom peers in a streaming system," Economics of Education Review, Elsevier, vol. 49(C), pages 110-128.
    72. Maciej Jakubowski, 2015. "Latent variables and propensity score matching: a simulation study with application to data from the Programme for International Student Assessment in Poland," Empirical Economics, Springer, vol. 48(3), pages 1287-1325, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Daniel Santín & Gabriela Sicilia, 2018. "Using DEA for measuring teachers’ performance and the impact on students’ outcomes: evidence for Spain," Journal of Productivity Analysis, Springer, vol. 49(1), pages 1-15, February.
    2. Cordero, Jose M. & Polo, Cristina & Santín, Daniel & Simancas, Rosa, 2018. "Efficiency measurement and cross-country differences among schools: A robust conditional nonparametric analysis," Economic Modelling, Elsevier, vol. 74(C), pages 45-60.
    3. Adeniran, Adedeji & Ishaku, Joseph & Akanni, Lateef Olawale, 2020. "Is Nigeria experiencing a learning crisis: Evidence from curriculum-matched learning assessment," International Journal of Educational Development, Elsevier, vol. 77(C).
    4. Schnepf, Sylke, 2018. "Insights into survey errors of large scale educational achievement surveys," Working Papers 2018-05, Joint Research Centre, European Commission (Ispra site).
    5. Gil-Izquierdo, María & Cordero, José Manuel, 2017. "Guidelines for data fusion with international large scale assessments: Insights from the TALIS-PISA link," MPRA Paper 79781, University Library of Munich, Germany.

    More about this item

    JEL classification:

    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jecsur:v:32:y:2018:i:3:p:878-915. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley Content Delivery). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0950-0804 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.