As the ritual annual self-flagellation over A level results reaches fever pitch, my mind turned to a story that I came across in the Telegraph a couple of weeks ago. It turns out that universities, amongst the main critics of the rising A level pass rate, may themselves be guilty of ‘dumbing down'.
According to the Telegraph, over the last ten years or so the proportion of students gaining firsts almost doubled, with upper seconds rising by a more modest 8%. The proportion of students therefore gaining ‘good’ degrees rose from 52.2% of the student body in 1997 to 61.4% last year, an 18% rise. The increase is even more marked if we go back to the 1980s when only a third of students gained the top two degree classifications.
In many ways this seems surprising. Universities are constantly bemoaning the declining skills of undergraduates caused by ‘satnav’ A level qualifications. Reports allege that students "lack the basic ability to express themselves in writing", and that there has been an “alarming decline” in numeracy standards. At the same time, the expansion of university education (numbers rising from 23% of under 21s in 1991 towards the government’s target of 50%) must surely imply some dilution of quality. In this light, one would surely expect a reduction in the number of 'good' degrees rather than the large increase we have seen.
It is possible, of course that universities are simply getting better at preparing undergraduates for their final degrees, but the evidence is that more and more universities are making use of postgraduates for teaching rather than their ‘big name’ academics. At the same time an increasingly market-driven environment is also leading to inexorable upward pressure on degree classes.
This pressure comes from two main sources, the external market and the internal one. Firstly, as discussed in an earlier post, university league tables include the proportion of good degrees as part of their ranking – with students being increasingly aware of the tables, a good ranking is important for recruitment and, by extension, funding. A leaked email from Manchester Metropolitan university last year showed how these pressures were brought to bear. The email, to Maths and Computing staff, says: "As a university we do not award as many Firsts and 2:1s as other comparable institutions so there is an understandable desire to increase the proportion of such awards. Please bear this in mind when setting your second and final year assessments, especially the latter." As hints go, this is a pretty broad one.
The internal market also exerts pressures – universities will close down courses that are not ‘economically viable’, essentially meaning having lots of students on them. If you want to keep your job, you need to run a course, and students are unlikely to sign up for courses in which high grades are difficult to acquire, leading to a beauty contest between different courses within degree programmes.
As a consequence, universities' protests at A level grade inflation making life difficult for them begin to ring a little hollow, as the same criticisms are now being levelled at them by prospective employers. Indeed, there have been some reports suggesting that degrees have become so devalued that employers prefer to look at university graduates' A level grades as being a better discriminator than degree class! A worrying development, given the rising cost of studying.
It is perhaps surprising, therefore, that at the very time when A level grades are being benchmarked against the students' GCSE background, to try to mitigate against grade inflation, universities are resisting any attempts to introduce greater rigour into their own procedures - since the demise of the CNAA (after polytechnics became universities back in 1992) each university has set its own standards with no over-arching supervision. Pots and kettles come to mind.
Showing posts with label pass rates. Show all posts
Showing posts with label pass rates. Show all posts
Friday, 21 August 2009
Wednesday, 8 July 2009
A* won’t turn the tide
In response to the concerns of the rising tide of top grades at A level (discussed in an earlier post), the government has decided to introduce a new A* grade at A level from June next year. The grade will be awarded to those scoring 90% in the second year exams (the current A grade requires an average of 80% over the two years). The aims of this are probably two-fold:
* To separate out the top performers for university selection
* To prevent students who perform very well in the easier AS year from being able to coast in the second year.
Whilst it may well achieve the latter of these two goals, I have doubts about its efficacy in terms of the former. Essentially, it is little more than a stop-gap. Much in the way that Canute tried to turn back the sea 1000 years ago, so too the A* will eventually become overwhelmed. We need only look at GCSEs to see the future of A level. When A* was first introduced to GCSEs back in 1994, 2.8% of candidates nationally achieved it. In 2008, the figure was 6.8%, almost a 150% increase over 15 years. The same inexorable fate must surely befall A levels over time.
Nevertheless, one might argue that in the short run, it will allow universities to identify the very best of all candidates. True, provided that we can rely on the quality of marking of scripts at the very top end. Public exams are inevitably marked by large teams of examiners, and there is consequently scope for variation within the marks that they award.
The standard tolerance for an examiner in a team is 5% of the total mark for the paper. In other words, if the exam is out of 60 marks, and I would give the script 37/60, an examiner within my team is within tolerance if they award a mark between 34 and 40. It doesn’t take a genius to work out that 34 and 40 out of 60 would lead to very different grading outcomes. The big problem for A* is that in my experience, examiners are far more consistent on the marks awarded to standard scripts lying between the E and A borderlines than outside them because they see far more of those scripts. It is clearly the case that there will be far fewer of the very best and very worst scripts. Therefore, because examiners see fewer of the best scripts, the scope for variation is probably greater, which in my opinion seriously undermines the reliability of the proposed A* grade.
What does this mean for my own suggestion of publishing the raw mark achieved by candidates? The same potential for error would still be there, but the starkness of the gap between A* and A would not be. If, as a result of poor marking, I score 88% instead of the 91% that I deserve, then that is unfortunate, but far less unfortunate than being given an A rather than an A* - at least the university can see clearly that I am not a borderline A/B candidate, and if they are aware of the degree of uncertainty inherent within A level marking (as some of the research done on marking suggests they should be), then maybe I have more of a chance.
On this subject, one of my colleagues (in the comments on my earlier blog) took issue with my sporting metaphor, arguing that whereas there is no limit to performance in the long jump, exams are all ultimately limited to 100%, and therefore even my proposals would eventually fail, as all top candidates end up scoring between 99.9 and 100% (much like figure skating, in fact). Whilst I might take issue with the argument that there are no limits to human performance (in the 2212 Olympics, the gaps between the top long-jumpers may require a micrometer to measure, as they all cluster around 11 metres!), I nevertheless take his point. Even raw scores are subject ultimately to grade inflation, and this perhaps is why many universities have started administering their own entrance tests.
In many ways, of course, this is a throw-back – Oxford and Cambridge used written entrance tests until 1995 and 1987 respectively – but their introduction is now becoming far more widespread – Imperial has introduced its own admissions tests, and BMAT, LNAT, HAT, TSA are now compulsory acronyms for top performing students. One of the reasons that Oxford and Cambridge abandoned entrance tests is that they were felt to give an unfair advantage to applicants from the Independent sector, who would be better prepared. Ultimately, though this argument doesn’t stack up – better preparation leads to higher A level grades in the Independent sector (50.4% A grades at independent schools in 2008 compared with a national average of 25.9%) giving those candidates an automatic advantage in applications anyway.
What we all want is for the best universities to admit the best students, and if universities running their own admissions tests can assist in this goal, then we should welcome it as a way of increasing fairness into university admissions that the introduction of an A* grade is, in my view, unlikely to do.
* To separate out the top performers for university selection
* To prevent students who perform very well in the easier AS year from being able to coast in the second year.
Whilst it may well achieve the latter of these two goals, I have doubts about its efficacy in terms of the former. Essentially, it is little more than a stop-gap. Much in the way that Canute tried to turn back the sea 1000 years ago, so too the A* will eventually become overwhelmed. We need only look at GCSEs to see the future of A level. When A* was first introduced to GCSEs back in 1994, 2.8% of candidates nationally achieved it. In 2008, the figure was 6.8%, almost a 150% increase over 15 years. The same inexorable fate must surely befall A levels over time.
Nevertheless, one might argue that in the short run, it will allow universities to identify the very best of all candidates. True, provided that we can rely on the quality of marking of scripts at the very top end. Public exams are inevitably marked by large teams of examiners, and there is consequently scope for variation within the marks that they award.
The standard tolerance for an examiner in a team is 5% of the total mark for the paper. In other words, if the exam is out of 60 marks, and I would give the script 37/60, an examiner within my team is within tolerance if they award a mark between 34 and 40. It doesn’t take a genius to work out that 34 and 40 out of 60 would lead to very different grading outcomes. The big problem for A* is that in my experience, examiners are far more consistent on the marks awarded to standard scripts lying between the E and A borderlines than outside them because they see far more of those scripts. It is clearly the case that there will be far fewer of the very best and very worst scripts. Therefore, because examiners see fewer of the best scripts, the scope for variation is probably greater, which in my opinion seriously undermines the reliability of the proposed A* grade.
What does this mean for my own suggestion of publishing the raw mark achieved by candidates? The same potential for error would still be there, but the starkness of the gap between A* and A would not be. If, as a result of poor marking, I score 88% instead of the 91% that I deserve, then that is unfortunate, but far less unfortunate than being given an A rather than an A* - at least the university can see clearly that I am not a borderline A/B candidate, and if they are aware of the degree of uncertainty inherent within A level marking (as some of the research done on marking suggests they should be), then maybe I have more of a chance.
On this subject, one of my colleagues (in the comments on my earlier blog) took issue with my sporting metaphor, arguing that whereas there is no limit to performance in the long jump, exams are all ultimately limited to 100%, and therefore even my proposals would eventually fail, as all top candidates end up scoring between 99.9 and 100% (much like figure skating, in fact). Whilst I might take issue with the argument that there are no limits to human performance (in the 2212 Olympics, the gaps between the top long-jumpers may require a micrometer to measure, as they all cluster around 11 metres!), I nevertheless take his point. Even raw scores are subject ultimately to grade inflation, and this perhaps is why many universities have started administering their own entrance tests.
In many ways, of course, this is a throw-back – Oxford and Cambridge used written entrance tests until 1995 and 1987 respectively – but their introduction is now becoming far more widespread – Imperial has introduced its own admissions tests, and BMAT, LNAT, HAT, TSA are now compulsory acronyms for top performing students. One of the reasons that Oxford and Cambridge abandoned entrance tests is that they were felt to give an unfair advantage to applicants from the Independent sector, who would be better prepared. Ultimately, though this argument doesn’t stack up – better preparation leads to higher A level grades in the Independent sector (50.4% A grades at independent schools in 2008 compared with a national average of 25.9%) giving those candidates an automatic advantage in applications anyway.
What we all want is for the best universities to admit the best students, and if universities running their own admissions tests can assist in this goal, then we should welcome it as a way of increasing fairness into university admissions that the introduction of an A* grade is, in my view, unlikely to do.
Tuesday, 7 July 2009
Are A levels getting easier?
Back in the mists of time when I took my A levels (1984), exactly 30% of candidates failed. The same was true in 1985 and 1986. The reason? Only 70% of candidates were allowed to pass – the system was used to rank students, not to measure standards. It didn’t matter whether everyone was good or everyone was bad – 10% would score ‘A’ grades, 15% ‘B’ grades, 10% ‘C’s, 15% ‘D’s, 20% ‘E’s and 30% would fail. This had been true every year since 1963.
In 1987, the world changed. Students needed only to gain a certain mark to achieve an ‘A’ grade or a pass. A standard was defined for an ‘A’ grade, and so long as you hit that standard, the ‘A’ grade was yours. Fast forwarding to 2008 97.2% of candidates passed, and 25.9% scored ‘A’ grades. At the same time, the numbers of students taking A levels have doubled, so that today 5 times as many students get ‘A’ grades as was the case in the 1980s.
Does this mean that A levels are easier? Ofqual, the Government’s exams watchdog, would say not – their analysis of standards over time suggests that the quality needed for an ‘A’ is the same today as it was several years ago, although the comparison is becoming increasingly difficult as A levels have become modular – can you compare the performance of a student in six discrete modules (2009) with one forced to take all the exams at the end of the two year course (1984)?
Much has changed over the last 25 years – syllabuses used to be just lists of books, whereas today they are detailed descriptions of what is and is not required, making it far easier for students to hit the required standard. As soon as the system changed from one of quotas to one where there was a defined standard, it was inevitable that more students would achieve it. If we set a given standard for a gold medal at the Olympics for the long jump, then as training techniques, equipment, diet and so on have improved, so would the number of gold medals. In 1936 Jesse Owens won the Munich Olympics with a jump of 8.06m. This would have been beaten by 7 of the finalists in the Beijing Olympics of last year. Does this mean that the long-jump is getting easier, or simply that athletes are getting better? The same analysis must surely apply to A levels.
In the end, the problem is that the A level system is being used for two mutually incompatible purposes. On the one hand we want to measure improvements in performance over time - part of the reason for the switch to setting a given standard for an ‘A’ grade. On the other hand, there is the need to use it to distinguish between students (for university application), at which point having lots of ‘A’ grades becomes a problem. It is difficult to use the system for both – either you have one gold medallist, at which point it is difficult to recognise improvements over time, or you have lots of gold medallists, but then there’s no winner.
Perhaps the solution suggests itself from the Olympic analogy – we can have lots of gold medallists, as long as we know who jumped the furthest. To do this we need simply to publish candidates’ raw scores. Then we can have the best of both worlds – we can see how many students hit the required standard for an ‘A’, but universities can see how much they beat it by. At this point, the annual hand-wringing over whether A levels have become too easy should simply go away.
In 1987, the world changed. Students needed only to gain a certain mark to achieve an ‘A’ grade or a pass. A standard was defined for an ‘A’ grade, and so long as you hit that standard, the ‘A’ grade was yours. Fast forwarding to 2008 97.2% of candidates passed, and 25.9% scored ‘A’ grades. At the same time, the numbers of students taking A levels have doubled, so that today 5 times as many students get ‘A’ grades as was the case in the 1980s.
Does this mean that A levels are easier? Ofqual, the Government’s exams watchdog, would say not – their analysis of standards over time suggests that the quality needed for an ‘A’ is the same today as it was several years ago, although the comparison is becoming increasingly difficult as A levels have become modular – can you compare the performance of a student in six discrete modules (2009) with one forced to take all the exams at the end of the two year course (1984)?
Much has changed over the last 25 years – syllabuses used to be just lists of books, whereas today they are detailed descriptions of what is and is not required, making it far easier for students to hit the required standard. As soon as the system changed from one of quotas to one where there was a defined standard, it was inevitable that more students would achieve it. If we set a given standard for a gold medal at the Olympics for the long jump, then as training techniques, equipment, diet and so on have improved, so would the number of gold medals. In 1936 Jesse Owens won the Munich Olympics with a jump of 8.06m. This would have been beaten by 7 of the finalists in the Beijing Olympics of last year. Does this mean that the long-jump is getting easier, or simply that athletes are getting better? The same analysis must surely apply to A levels.
In the end, the problem is that the A level system is being used for two mutually incompatible purposes. On the one hand we want to measure improvements in performance over time - part of the reason for the switch to setting a given standard for an ‘A’ grade. On the other hand, there is the need to use it to distinguish between students (for university application), at which point having lots of ‘A’ grades becomes a problem. It is difficult to use the system for both – either you have one gold medallist, at which point it is difficult to recognise improvements over time, or you have lots of gold medallists, but then there’s no winner.
Perhaps the solution suggests itself from the Olympic analogy – we can have lots of gold medallists, as long as we know who jumped the furthest. To do this we need simply to publish candidates’ raw scores. Then we can have the best of both worlds – we can see how many students hit the required standard for an ‘A’, but universities can see how much they beat it by. At this point, the annual hand-wringing over whether A levels have become too easy should simply go away.
Subscribe to:
Posts (Atom)
Posts
