IDEAS home Printed from https://ideas.repec.org/p/qss/dqsswp/1704.html

What Happens When Econometrics and Psychometrics Collide? An Example Using PISA Data

Author

Listed:
  • John Jerrim

    (Department of Social Science, UCL Institute of Education, University College London)

  • Luis Alejandro Lopez-Agudo

    (Departamento de Economía Aplicada (Estadística y Econometría). Facultad de Ciencias Económicas y Empresariales. Universidad de Málaga)

  • Oscar D. Marcenaro-Gutierrez

    (Departamento de Economía Aplicada (Estadística y Econometría). Facultad de Ciencias Económicas y Empresariales. Universidad de Málaga)

  • Nikki Shure

    (Department of Social Science, UCL Institute of Education and Institute of Labor Economics)

Abstract

International large-scale assessments such as PISA are increasingly being used to benchmark the academic performance of young people across the world. Yet many of the technicalities underpinning these datasets are miss-understood by applied researchers, who sometimes fail to take into account their complex survey and test designs. The aim of this paper is to generate a better understanding amongst economists about how such databases are created, and what this implies for the empirical methodologies one should or should not apply. In doing so, we explain how some of the modelling strategies preferred by economists is at odds with the design of these studies. In doing so, we hope to generate a better understanding of international large-scale education datasets, and promote better practice in their use.

Suggested Citation

  • John Jerrim & Luis Alejandro Lopez-Agudo & Oscar D. Marcenaro-Gutierrez & Nikki Shure, 2017. "What Happens When Econometrics and Psychometrics Collide? An Example Using PISA Data," DoQSS Working Papers 17-04, Quantitative Social Science - UCL Social Research Institute, University College London.
  • Handle: RePEc:qss:dqsswp:1704
    as

    Download full text from publisher

    File URL: https://repec.ucl.ac.uk/REPEc/pdf/qsswp1704.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Andreas Ammermueller, 2007. "Poor Background or Low Returns? Why Immigrant Students in Germany Perform so Poorly in the Programme for International Student Assessment," Education Economics, Taylor & Francis Journals, vol. 15(2), pages 215-230.
    2. David Kiss, 2013. "Are immigrants and girls graded worse? Results of a matching approach," Education Economics, Taylor & Francis Journals, vol. 21(5), pages 447-463, December.
    3. Lant Pritchett & Martina Viarengo, 2015. "Does public sector control reduce variance in school quality?," Education Economics, Taylor & Francis Journals, vol. 23(5), pages 557-576, October.
    4. Andreas Ammermueller, 2007. "PISA: What makes the difference?," Empirical Economics, Springer, vol. 33(2), pages 263-287, September.
    5. Veruska Oppedisano & Gilberto Turati, 2015. "What are the causes of educational inequality and of its evolution over time in Europe? Evidence from PISA," Education Economics, Taylor & Francis Journals, vol. 23(1), pages 3-24, February.
    6. John Micklewright & Sylke V. Schnepf & Pedro N. Silva, 2010. "Peer effects and measurement error: the impact of sampling variation in school survey data," DoQSS Working Papers 10-13, Quantitative Social Science - UCL Social Research Institute, University College London.
    7. Polidano, Cain & Tabasso, Domenico, 2014. "Making it real: The benefits of workplace learning in upper-secondary vocational education and training courses," Economics of Education Review, Elsevier, vol. 42(C), pages 130-146.
    8. Maciej Jakubowski, 2015. "Latent variables and propensity score matching: a simulation study with application to data from the Programme for International Student Assessment in Poland," Empirical Economics, Springer, vol. 48(3), pages 1287-1325, May.
    9. Barbara Mueller & Stefan Wolter, 2014. "The role of hard-to-obtain information on ability for the school-to-work transition," Empirical Economics, Springer, vol. 46(4), pages 1447-1471, June.
    10. Herrero, Carmen & Mendez, Ildefonso & Villar, Antonio, 2014. "Analysis of group performance with categorical data when agents are heterogeneous: The evaluation of scholastic performance in the OECD through PISA," Economics of Education Review, Elsevier, vol. 40(C), pages 140-151.
    11. S. Mahuteau & K. Mavromaras, 2014. "An analysis of the impact of socio-economic disadvantage and school quality on the probability of school dropout," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 389-411, August.
    12. Maciej Jakubowski & Harry Anthony Patrinos & Emilio Ernesto Porta & Jerzy Wiśniewski, 2016. "The effects of delaying tracking in secondary school: evidence from the 1999 education reform in Poland," Education Economics, Taylor & Francis Journals, vol. 24(6), pages 557-572, November.
    13. Nonoyama-Tarumi, Yuko & Willms, J. Douglas, 2010. "The relative and absolute risks of disadvantaged family background and low levels of school resources on student literacy," Economics of Education Review, Elsevier, vol. 29(2), pages 214-224, April.
    14. Lounkaew, Kiatanantha, 2013. "Explaining urban–rural differences in educational achievement in Thailand: Evidence from PISA literacy data," Economics of Education Review, Elsevier, vol. 37(C), pages 213-225.
    15. Marina Murat & Patrizio Frederic, 2015. "Institutions, culture and background: the school performance of immigrant students," Education Economics, Taylor & Francis Journals, vol. 23(5), pages 612-630, October.
    16. Stefanie Hof, 2014. "Does private tutoring work? The effectiveness of private tutoring: a nonparametric bounds analysis," Education Economics, Taylor & Francis Journals, vol. 22(4), pages 347-366, August.
    17. Asako Ohinata & Jan C. van Ours, 2013. "How Immigrant Children Affect the Academic Achievement of Native Dutch Children," Economic Journal, Royal Economic Society, vol. 0, pages 308-331, August.
    18. Harry Anthony Patrinos, 2013. "Private education provision and public finance: the Netherlands," Education Economics, Taylor & Francis Journals, vol. 21(4), pages 392-414, September.
    19. Eric A. Hanushek & Ludger Wössmann, 2006. "Does Educational Tracking Affect Performance and Inequality? Differences- in-Differences Evidence Across Countries," Economic Journal, Royal Economic Society, vol. 116(510), pages 63-76, March.
    20. Tommaso Agasisti, 2013. "The efficiency of Italian secondary schools and the potential role of competition: a data envelopment analysis using OECD-PISA2006 data," Education Economics, Taylor & Francis Journals, vol. 21(5), pages 520-544, December.
    21. Cain Polidano & Barbara Hanel & Hielke Buddelmeyer, 2013. "Explaining the socio-economic status school completion gap," Education Economics, Taylor & Francis Journals, vol. 21(3), pages 230-247, July.
    22. Jensen, Peter & Rasmussen, Astrid Würtz, 2011. "The effect of immigrant concentration in schools on native and immigrant children's reading and math skills," Economics of Education Review, Elsevier, vol. 30(6), pages 1503-1515.
    23. Micklewright, John & Schnepf, Sylke V. & Silva, Pedro N., 2012. "Peer effects and measurement error: The impact of sampling variation in school survey data (evidence from PISA)," Economics of Education Review, Elsevier, vol. 31(6), pages 1136-1142.
    24. Jan van Ours, 2008. "When do children read books?," Education Economics, Taylor & Francis Journals, vol. 16(4), pages 313-328.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giannelli, Gianna Claudia & Rapallini, Chiara, 2016. "Immigrant student performance in Math: Does it matter where you come from?," Economics of Education Review, Elsevier, vol. 52(C), pages 291-304.
    2. José M. Cordero & Víctor Cristóbal & Daniel Santín, 2018. "Causal Inference On Education Policies: A Survey Of Empirical Studies Using Pisa, Timss And Pirls," Journal of Economic Surveys, Wiley Blackwell, vol. 32(3), pages 878-915, July.
    3. Marina Murat, 2012. "Do Immigrant Students Succeed? Evidence from Italy and France," Global Economy Journal (GEJ), World Scientific Publishing Co. Pte. Ltd., vol. 12(3), pages 1-22, August.
    4. Maria Cattaneo & Stefan Wolter, 2015. "Better migrants, better PISA results: Findings from a natural experiment," IZA Journal of Migration and Development, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-19, December.
    5. Bonin, Holger, 2017. "The Potential Economic Benefits of Education of Migrants in the EU," IZA Research Reports 75, Institute of Labor Economics (IZA).
    6. Jens Ruhose, 2015. "Microeconometric Analyses on Economic Consequences of Selective Migration," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 61, July.
    7. De Paola, Maria & Brunello, Giorgio, 2016. "Education as a Tool for the Economic Integration of Migrants," IZA Discussion Papers 9836, Institute of Labor Economics (IZA).
    8. Frattini, Tommaso & Meschi, Elena, 2019. "The effect of immigrant peers in vocational schools," European Economic Review, Elsevier, vol. 113(C), pages 1-22.
    9. Marina Murat & Davide Ferrari & Patrizio Frederic, 2012. "Immigrant students and educational systems. Cross-country evidence from PISA 2006," Center for Economic Research (RECent) 080, University of Modena and Reggio E., Dept. of Economics "Marco Biagi".
    10. Paul Anand & Jere R. Behrman & Hai-Anh H. Dang & Sam Jones, 2018. "Inequality of opportunity in education: Accounting for the contributions of Sibs, schools and sorting across East Africa," Working Papers 480, ECINEQ, Society for the Study of Economic Inequality.
    11. Clifton-Sprigg, Joanna, 2014. "Educational spillovers and parental migration," 2007 Annual Meeting, July 29-August 1, 2007, Portland, Oregon TN 2015-46, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    12. Cattaneo, Maria Alejandra & Wolter, Stefan C., 2012. "Migration Policy Can Boost PISA Results: Findings from a Natural Experiment," IZA Discussion Papers 6300, Institute of Labor Economics (IZA).
    13. Seah, Kelvin, 2016. "The Impact of Immigrant Peers on Native Students' Academic Achievement in Countries Where Parents of Immigrants Are Relatively Skilled," IZA Discussion Papers 10065, Institute of Labor Economics (IZA).
    14. Marina Murat, 2011. "Do immigrant students succeed? Evidence from Italy and France based on PISA 2006," Department of Economics 0670, University of Modena and Reggio E., Faculty of Economics "Marco Biagi".
    15. Kyle Peyton & Chris Ryan & Justin van de Ven, 2016. "What Can We Learn from Student Attitudes for International Achievement Tests?," Melbourne Institute Working Paper Series wp2016n22, Melbourne Institute of Applied Economic and Social Research, The University of Melbourne.
    16. Ohinata, A. & van Ours, J.C., 2013. "Spillover Effects of Studying with Immigrant Students : A Quantile Regression Approach," Discussion Paper 2013-058, Tilburg University, Center for Economic Research.
    17. Björn NILSSON, 2019. "Education and migration: insights for policymakers," Working Paper 23ca9c54-061a-4d60-967c-f, Agence française de développement.
    18. David Figlio & Umut Özek, 2019. "Unwelcome Guests? The Effects of Refugees on the Educational Outcomes of Incumbent Students," Journal of Labor Economics, University of Chicago Press, vol. 37(4), pages 1061-1096.
    19. Timothy M. Diette & Ruth Uwaifo Oyelere, 2017. "Do limited English students jeopardize the education of other students? Lessons from the North Carolina public school system," Education Economics, Taylor & Francis Journals, vol. 25(5), pages 446-461, September.
    20. Andreas Ammermueller, 2013. "Institutional Features of Schooling Systems and Educational Inequality: Cross-Country Evidence From PIRLS and PISA," German Economic Review, Verein für Socialpolitik, vol. 14(2), pages 190-213, May.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C10 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - General
    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:qss:dqsswp:1704. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Dr Neus Bover Fonts (email available below). General contact details of provider: https://edirc.repec.org/data/dqioeuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.