As an avid Podcast listener, the More or Less Podcast has long been a weekly essential. The Podcast looks to explain the numbers and statistics used in political debate, the news and everyday life. I was particularly captivated by a recent episode, in which the Podcast investigated the truth behind claims made by The Times that “48% of A level results are wrong”.
To explore these assertions the More or Less team met with Dennis Sherwood, an Independent Consultant who runs the Silver Bullet Machine consultancy. Sherwood started the conversation by suggesting that there is an overall one in four chance that a grade is going to be graded differently when marked by a senior examiner.
The source of Sherwood’s claims is Marking Consistency Metrics, which was published by Ofqual (The Office of Qualifications and Examinations Regulation) in 2018. This study was carried out by Ofqual over several years in which many school exams were blind double-marked. In each pair, one mark was given by a general examiner and the other by a senior examiner. The senior examiners mark was used as the measure of what was the ‘right’ mark. This study was cohort-wide across whole subjects.
During the short Podcast interview Sherwood mentions a few of the key findings from the report, giving a few specific subjects as examples:
Maths
- 96% marks remained the same
- 4% marks were different
Economics
- 74% marks remained the same
- 26% marks were different
English Language
- 61% marks remained the same
- 39% marks were different
For a subject like History, the probability of a grade being higher or lower was found to be 50%.
Of course, if a candidate is unhappy with their final grade, they can always request a re-mark. Rather terrifyingly the stat given during the Podcast suggested that out of 58,000 re-mark requests last year, 12,100 were re-graded – thus, 21% of re-marks altered the end grade.
In this article, Sherwood notes that, if, on average, about 1 grade in 4 is indeed wrong, there are some important consequences. For example:
- Every candidate sitting 4 A levels is likely to be awarded 1 wrong grade
- Every candidate sitting 8 GCSEs is likely to be awarded 2 wrong grades
Sherwood also suggests that drawing directly on the data shown in the chart:
- For every 100 candidates sitting A level maths, further maths and physics, about 81 receive a certificate on which all 3 grades are right, whilst about 19 are awarded at least 1 wrong grade.
-
For every 100 candidates sitting A level English language, English literature and history, about 20 receive a certificate on which all 3 grades are right, whilst about 80 are awarded at least 1 wrong grade.
Is there a solution?
Sherwood suggested dispensing with the current grade system entirely. Suggesting that examiners could award a given mark, which would be paired with a range that would reflect the ‘fussiness of the subject’.
For example:
- Maths (mark +/- 3)
- History (mark +/- 10)
Where are we now?
The question remains, what is the fairest system of delivering exams and results. Given that so many important decisions and opportunities stem from an individual’s performance at GCSE and A level, is it not essential to form a system that minimises the range of potential results dependent on who is marking the exam? Is there room to keep changing the system, and if so, how can we do so to enable a more consistent marking classification?
References
https://www.hepi.ac.uk/2019/01/15/1-school-exam-grade-in-4-is-wrong-does-this-matter/
https://www.thetimes.co.uk/article/revealed-a-level-results-are-48-wrong-xsj33jvnh
Harriet Blomefield is an Education Consultant at Keystone Tutors
Follow Keystone on Twitter @Keystone_Tutors