IDEAS home Printed from https://ideas.repec.org/p/oec/eduaab/87-en.html
   My bibliography  Save this paper

Analysis of the Predictive Power of PISA Test Items

Author

Listed:
  • Maciej Jakubowski

    (Warsaw University)

Abstract

The predictive power of the PISA test items for future student success is examined based on data from the Longitudinal Surveys of Australian Youth (LSAY) for the PISA 2003 cohort. This working paper analyses how students’ responses to mathematics and problem-solving items in PISA 2003 are related to the students’ qualifications in education in 2007 and 2010. The results show that items do differ in their predictive power, depending on some of their deep qualities. PISA mathematics and problem-solving items are grouped into various classifications according to their qualities. This paper proposes 16 new classifications of items. Among mathematics-specific item classifications, two are found to be significantly related to future student success: those that assess knowledge, understanding, and application of statistics; and those related to rates, ratios, proportions, and/or percent. These items frequently require students to apply common mathematical concepts to solve multi-step, non-routine problems, think flexibly, and understand and interpret information presented in an unfamiliar format or context. Among classifications that are not specific to mathematics, items that were classified as using reverse or flexible thinking are found to be related to student qualifications in both mathematics and problem solving. These items require students to be able to think through a solution at various points during the problem-solving process, not just at the start. L’efficacité prédictive des items de l’enquête PISA pour la réussite future des élèves est examinée à partir de données collectées par l’enquête longitudinale australienne LSAY (Longitudinal Surveys of Australian Youth) sur l’échantillon d’élèves évalués lors du cycle PISA 2003. Le présent document de travail analyse dans quelle mesure les réponses des élèves aux items de mathématiques et de résolution de problèmes de l’enquête PISA 2003 sont liées à leur niveau de formation en 2007 et 2010. Les résultats montrent que l’efficacité prédictive des items varie en fonction de certaines de leurs qualités profondes. Les items PISA de mathématiques et de résolution de problèmes sont classés dans différentes catégories selon leurs qualités. Le présent document propose 16 nouvelles catégories d’items. Parmi les catégories d’items spécifiques à l’évaluation des compétences en mathématiques, deux ont été identifiées comme étant liées de façon significative à la réussite future des élèves : les types d’items qui évaluent les connaissances, la compréhension et l’application des statistiques, et ceux qui ont trait aux taux, ratios, proportions et/ou pourcentages. Ces items requièrent généralement des élèves l’application de concepts mathématiques de base pour résoudre des problèmes non routiniers et comportant plusieurs étapes, un raisonnement flexible, ainsi que la compréhension et l’interprétation d’informations présentées sous un format ou dans un contexte non familiers. Parmi les catégories qui ne sont pas spécifiques aux mathématiques, il apparaît que les items définis comme faisant appel à un raisonnement rétrospectif ou flexible présentent une corrélation avec les qualifications des élèves à la fois en mathématiques et en résolution de problèmes. Ces items supposent de la part des élèves une capacité à réfléchir à une solution à différentes étapes du processus de résolution du problème, et pas uniquement au début de ce processus.

Suggested Citation

  • Maciej Jakubowski, 2013. "Analysis of the Predictive Power of PISA Test Items," OECD Education Working Papers 87, OECD Publishing.
  • Handle: RePEc:oec:eduaab:87-en
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1787/5k4bx47268g5-en
    Download Restriction: no

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Deb, Kaveri & Sengupta, Bodhisattva, 2016. "On Empirical Distribution of RCA Indices," MPRA Paper 74087, University Library of Munich, Germany.
    2. Wang, Qi & Zhang, Chunyu & Ding, Yi & Xydis, George & Wang, Jianhui & Østergaard, Jacob, 2015. "Review of real-time electricity markets for integrating Distributed Energy Resources and Demand Response," Applied Energy, Elsevier, vol. 138(C), pages 695-706.
    3. Michele Pellizzari & Anne Fichen, 2013. "A New Measure of Skills Mismatch: Theory and Evidence from the Survey of Adult Skills (PIAAC)," OECD Social, Employment and Migration Working Papers 153, OECD Publishing.
    4. Chen, Xin & Mu, Hailin & Li, Huanan & Gui, Shusen, 2014. "Using stockpile delegation to improve China׳s strategic oil policy: A multi-dimension stochastic dynamic programming approach," Energy Policy, Elsevier, vol. 69(C), pages 28-42.
    5. Michele Pellizzari & Anne Fichen, 2017. "A new measure of skill mismatch: theory and evidence from PIAAC," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 6(1), pages 1-30, December.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oec:eduaab:87-en. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/deoecfr.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.