Policy reflections

In late 2024 I decided to add this page to my blog, the intention being to reflect personally on policies affecting education in South Africa, in other countries, and even the global organisations, for instance in relation to standardised reporting on learning. In these personal reflections I want to focus strongly on how the things we do in education contribute, or do not contribute, towards better learning, especially at the primary level. This is a key concern in the Sustainable Development Goals (SDGs), for good reason. I’ve expressed the typical logic for this in, for instance, this report (see section 2.2). My aim is also to focus on how data and statistics can tighten up the policy discussions. I want to be honest, while avoiding sensationalism and emotional mud-slinging, of which there is quite an abundance in the education policy debates. My framework draws strongly from the economics of education – that’s my current profession – and the history behind what we see today – once upon a time I qualified as history teacher.

Lacking prescriptions on how to measure educational progress in countries?

30 June 2025

Much of the debate around the comparability of national learning statistics focusses on comparing countries, as opposed to comparing a country’s progress over time. The two are related, yet different. I’m going to focus on the latter, which is obviously crucial.

Descriptions of what learners should be able to do in specific grades are fundamental for national measures of proficiency. Here the situation is not too bad. Any decent national school curriculum describes expected competencies in some detail. Globally, there are agreed upon descriptions, though UNESCO could have been better at differentiating what is official from what is indicative. While this ‘draft’ seems to capture UNESCO’s most recent position, earlier documents here, here and here could easily be interpreted as final and official with (I’ve saved locally, in the interests of a historical perspective, in case UNESCO moves them).

An important concern is that there has been too little focus on precursor skills, in particular expected levels of reading fluency. Under the Early Grade Reading Assessment (EGRA) banner, reading fluency testing was used in one or two countries to gauge national improvements. EGRA involves getting children to read a passage in a one-on-one assessment. The assessor counts words read correctly in a minute. Perhaps more sustainable, given the demise of EGRA’s main funder USAID, is UNICEF’s reading fluency test within the Multiple Indicator Cluster Surveys (MICS). Soon MICS should have enough data to gauge national trends over time.

Had the work on precursor skills been as rich as it is now in the years preceding the 2015 launch of the SDGs, such skills may have been a prominent SDG feature. Instead, SDG indicators are defined largely with more traditional tests, from the end of lower primary level and above, in mind. This explains why SDG policies have engaged with precursor skills fairly tangentially.

Things get complex when it comes to comparability over time. National examinations, such as South Africa’s ‘Matric’, easily create the impression that with the help of some statistical adjustment the results of two separate examinations are sufficiently comparable. In fact, these examinations can perform critical tasks well: signalling to learners what post-school studies they can apply for, and giving universities a ranking of learners allowing them to select who is admitted into, say, engineering. But considerably more rigorous comparability is required to gauge whether a whole schooling system is improving, especially given that progress is at best painfully slow.

Here sufficient comparability – it’s always a question of sufficient fitness-for-purpose, never perfection – requires anchor (or common) items, questions repeated over time and not leaked outside the testing process. The approach is perhaps most authoritatively laid out in this AERA guide, which explains the associated item response theory (IRT) statistical methods for obtaining comparable scores (a nice layman’s explanation can be found here).

Unfortunately, good IRT software plus a good textbook (Stata, ConQuest, R textbook) can be expensive, complicating sorely needed capacity building across countries.

The technical documentation aside, what does the more official standards documentation, for instance relating to the SDGs, say? The situation is not ideal. That latest UNESCO draft briefly points to the need for IRT, but detailed guidance informed in part by what is actually happening in the field would be useful. A crucial requirement, stated in the UNESCO draft, is that approaches towards standardising results should be well-documented.

It should be noted that IRT is useful for multi-question tests, including but not limited to multiple-choice tests. However, for reading fluency testing with words correct counts IRT is not appropriate, nor is it needed. A words-correct-per-minute measure is fairly comparable without any adjustment within one language.

Where things get messy is in the implementation of IRT. Lant Pritchett has complained that education administrations too often pursue ‘isomorphic mimicry’, superficial compliance in terms of labels, but without substance. SEACMEQ, which claims to use IRT, has released almost no documentation on this, and its microdata are not consistent with proper IRT use. Western Cape’s provincial testing system and Kenya’s national assessment seem to use elements of IRT, but the impression created that the technique was fully implemented seems dubious and is impossible to confirm with the available documentation.

Inaccuracies unfortunately extend to UNESCO’s own products. UNESCO’s intention to monitor the monitoring systems is an excellent one. However, the most recent version of this indicates through a ‘quality index’ that just under half of some 10,000 subject-specific assessments across the world used IRT, presumably in an appropriate manner, when this is clearly not true in many instances. To cite just one major anomaly, South Africa’s Annual National Assessments (ANA) did not employ IRT and the official reports were clear that comparability was very limited.

Apart from good psychometrics (including IRT), the second pillar underpinning comparability is good sampling. A sample-based approach is often inescapable for reasons of cost and to facilitate security around anchor items. Poor sampling as a destroyer of comparability could receive more attention. Here it is useful to include the timing of the assessment among the sampling challenges – learners at the start of grade 3 are a different population to learners at the end of grade 3 in terms of what has been learnt. Incomparable sampling famously distorted political debates around educational quality in Britain. Patterns of electricity access in the PASEC data suggest that for some countries the data may not be nationally representative. Even in mature assessment programmes such as TIMSS, country-specific details on exactly how sampling occurred is lacking.

National prescriptions on how to measure progress in learning, prescriptions that ministry of education officials pay special attention to, are often weak, ignoring the limitations of national examinations for measuring progress, and focussing somewhat blindly on formats developed by auditor-general staff, whose knowledge of non-financial monitoring is often weak.

While good policies and prescriptions are no guarantee of good practice, if they are non-existent, superficial or unclear, that can be detrimental. There remains a lot that can be done to improve them, at the global and national levels.

What the BELA Act does and does not do

30 November 2024

What is the significance of the various parts of the Basic Education Laws Amendment (BELA) Act of 2024? What are the key financial and non-financial numbers we should keep in mind? How might the Act affect how learners learn?

There are 40 sections in the Act amending the South African Schools Act (SASA), while 13 amend the Employment of Educators Act (EEA). Unfortunately, integrated SASA and EEA texts with all amendments up to 2024 (there have been many before 2024) are not publicly available yet. Given past trends, this might take a couple of years.

There is one hugely significant, and widely embraced, SASA amendment: making the grade before Grade 1, Grade R, compulsory (section 2 of BELA). Then there are several SASA amendments which, at least in a legal sense, increase the powers of the head of the provincial department, mostly relative to the school governing body (SGB). These amendments deal with: the languages of learning and teaching (LOLTs) used by the school (section 5); the admission of learners (4); and home schooling (35). Finally, having certain schools focus on specifically talented learners is facilitated (12). These are for me the substantive amendments.

Data behind this report indicate that some 80% of children get to participate in Grade R in a school. The amended SASA requires that to be 100%. An obvious solution is to have more Grade R places in public schools. A less obvious solution is to re-classify pre-schools currently providing Grade R as schools in the legal sense, even if they do not offer Grade 1 and above. These pre-schools are nearly all private, and some are subsidised by the state. Data I have looked at indicate that the middle class is over twice as likely not to have their children attend Grade R in a school, mainly because they are in a private pre-school. Analysis from some years back suggests this is not a new pattern. If BELA is to be implemented in a pro-poor fashion, funding and management effort should be directed at the approximately 100,000 children who are ‘missing’ from Grade R in the poorer quintiles 1 to 3 schools. How many of these 100,000 are already in subsidised centres as opposed to being un-enrolled is difficult to ascertain. But the number would not be insignificant.

It is widely understood that if Grade R becomes compulsory, its per learner funding in public schools would become comparable to that in grades 1 and above. My reading of the various policies is that there is no clear legal requirement to increase Grade R per learner funding, even if this seems desirable. Clearly, a critical question is what the gap currently is between per learner public funding in Grade R and in Grade 1. It is sometimes assumed that because the policy requires Grade R to be funded at 70% of the Grade 1 level, that is what is actually happening. I suspect the actual gap is much wider – the 2022 World Bank report on ECD in South Africa points to Grade R enjoying only around 35% of Grade 1 funding (see Figure 29).

But will increasing schools-based Grade R attendance in poorer communities from around 85% to 100% help boost learning across all grades? Reference is often made to this excellent study that finds having done Grade R in a school barely changes one’s learning prospects in later grades. The problem with this study is that it uses data from a time when Grade R was even more under-resourced and under-prioritised than it is today. The bottom line is that we actually know little about the impact of Grade R as currently offered.

The BELA amendments aimed at elevating the powers of heads of provincial departments are the ones that have sparked controversy. The ANC has argued that SDG powers are not reduced, but are simply harmonised with those of the department. It has moreover emphasised that BELA will reduce the use of a school’s language policy as a means of excluding learners. On BELA, the EFF’s and MKP’s positions have largely been in line with those of the ANC. The DA’s strong opposition to BELA has officially focussed on the undesirability of reducing SDG powers. Indeed, the NDP is supportive of strong SDGs as part of ‘citizen participation’.

In my view, simmering beneath the surface of the BELA debates around SGB powers are historically rooted tensions between, on the one hand, those who recall that the imposition of Afrikaans in schools was a catalyst for the broader struggle against apartheid and, on the other hand, white Afrikaner nationalists. Relevant and historically sensitive research on this topic could help. Among primary-level learners, some 7% are taught in Afrikaans. But a sub-set of this, 2% of primary learners, are in some 300 schools which are, firstly, historically white and, secondly, use no LOLT other than Afrikaans. Within these 300 schools, only 4% of learners speak an African language at home. Some research exists on discriminatory practices in these schools, and similar secondary schools, but we would benefit from some more if we are to understand how BELA might change the situation.

In short, the BELA Act provides the foundations for the universalisation of Grade R, but is not meant to deal with the critical matter of the quality of Grade R, and is essentially silent on key resourcing questions. While the Act does appear to strengthen the powers of the head of the provincial department, it is striking how many complex checks and balances there are on the new powers, to the extent that I wonder whether the Act will in fact alter the balance of power between the department and SGBs, particularly the SGBs of historically advantaged schools. There is a lot of work, in terms of support to schools and school accountability, which must still be done at the national and provincial levels to realise the system envisaged in the NDP. That is work which essentially falls outside the ambit of the BELA Act.