IDEAS home Printed from
   My bibliography  Save this paper

A revised PIRLS 2011 to 2016 trend for South Africa and the importance of analysing the underlying microdata


  • Martin Gustafsson

    () (ReSEP, Stellenbosch University and Department of Basic Education)


Given South Africa’s weak performance in international testing programmes, there is a strong interest in gauging improvements within these programmes. The finding that South Africa saw no progress between 2011 and 2016 in the Progress in International Reading Literacy Study (PIRLS) programme, which tests Grade 4 reading competencies, was inconsistent with considerable progress seen in a couple of other testing programmes. Moreover, an earlier PIRLS average score for Grade 4 from 2006 suggested that the 2011 mean score used to determine the flat 2011 to 2016 flat trend was problematic. The current paper uses the underlying microdata for PIRLS 2011 and 2016, which are publicly available, to examine the trend. It is clear that the 2011 mean score used by the international PIRLS analysts to arrive at the flat trend cannot be correct. It should be considerably lower. It should be noted that the 2011 mean for South Africa involved an unusual process. South Africa was the only country for which an original mean on an easier scale, prePIRLS, had to be recalibrated to the main PIRLS scale. This was because South Africa was the only country participating in the easier prePIRLS in 2011 and in some form of PIRLS in 2016. There was clearly something wrong with the 2011 recalibration. In correspondence, the International Association for the Evaluation of Educational Achievement (IEA), the body that conducts PIRLS globally, acknowledged, in part on the basis of a preliminary version of the current paper, that the originally published 2011 to 2016 South Africa trend should now not be considered reliable. They also confirmed that the classical score gains for South Africa reflected in the paper are correct. The method used in the paper is essentially to examine classical score gains between 2011 and 2016 with respect to common items, and then to recalibrate that to the main PIRLS scale. As an additional verification, the paper checks that gains remain after one controls for socio-economic status. The paper concludes that South Africa in fact saw a large gain between 2011 and 2016, equal to around 0.05 standard deviations a year. This is a fast rate of improvement by international standards. Of the 43 countries with 2011 to 2016 trends in PIRLS, South Africa displayed the third-steepest improvement, after Morocco and Oman.

Suggested Citation

  • Martin Gustafsson, 2020. "A revised PIRLS 2011 to 2016 trend for South Africa and the importance of analysing the underlying microdata," Working Papers 02/2020, Stellenbosch University, Department of Economics.
  • Handle: RePEc:sza:wpaper:wpapers337

    Download full text from publisher

    File URL:
    File Function: First version, 2020
    Download Restriction: no


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Martin Gustafsson & Carol Nuga Deliwe, 2020. "How is the COVID-19 pandemic affecting educational quality in South Africa? Evidence to date and future risks," Working Papers 23/2020, Stellenbosch University, Department of Economics.

    More about this item


    South Africa; PIRLS; international testing systems; reading;
    All these keywords.

    JEL classification:

    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C89 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sza:wpaper:wpapers337. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Melt van Schoor). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.