Instead, it used the rank order that emerged from the 2019 exam results. Which is like showing you can predict the results of a horse race by including data about the order in which the horses crossed the finish line in that same race. “If a test uses aspects of the same data that it is trying to predict, then it results in a false sense of security,” says Nason.
Even by including some of the data they were trying to predict, Ofqual found their accuracy in predicting exact grades ranged from two thirds for History to one in four for Italian. For most non-language subjects, over nine in 10 students would be within one grade of the true result, but 3% of Maths students (for example) missing a fair result by two grades or more adds up to a lot of teenagers. Over 10% of Further Maths students, ironically the only ones who can understand the tortuous workings of the algorithm that betrayed them, would be over a grade away from a fair result.
“Their algorithm’s predictability is, especially for some subjects, not that good anyway, but if you then realise that they are over-optimistic and cannot be trusted, then one has to really question whether the algorithm is fit for purpose,” says Guy.
This isn’t just hindsight talking. The RSS offered to nominate two distinguished experts to the Ofqual technical advisory group in March. Guy was one of them. But Ofqual wanted to impose a Non-Disclosure Agreement that would bar them from public comment on the model for five years, in direct contradiction of the Society’s commitment to transparency and public trust.
Ofqual also ignored the House of Commons Education Select Committee’s call to publish details of their methods before releasing the results. They might have been spared some of the post-hoc dissection of their work, before public outcry and political pain caused Monday’s abandonment of the algorithm.
Because the grading algorithm has been withdrawn for political, not statistical reasons.
Its workings seem to have hit harder the very students who already felt the cards were stacked against them, and the communities to whom this government promised “levelling up”. Students in state schools and FE colleges, especially, and more deprived students, saw their awarded grades fall well short of their teacher-predicted grades.
It’s no surprise that small teaching groups, and less popular A-levels like Law, Ancient Greek and Music, are more common at private schools, which insulated those students from being marked down. Nor that previous results in selective and private schools would have been higher, bequeathing a higher range of grades to this year’s cohort.
But the picture is messier than that. Historically, teachers in large state schools, and of more deprived students, have been more likely to over-predict. There may be good reasons for this. Teachers may consciously give a student the benefit of the doubt, figuring that it at least gives them a shot at a good university place. If they fall short, they can haggle later. If they exceed more honest expectations, it might be too late for them to raise their sights.
And in a high-attaining school where students routinely get A and A* exam results, there is not much headroom for over-optimism, unlike schools whose students walk away with the full range from A* to U.
Whatever the causes, that over-prediction means that every year state school students are more likely to find their exam results lower than their predicted grades. Universities are often flexible, recognising that it’s easier to get good grades in a good school, and that students who fought harder for OK results often do better at university than the ones who got good results in easier circumstances.
It’s bitter to be disappointed with your exam results. Perhaps, like me in those Stats exams, you turned over the paper and finally acknowledged, too late, how poorly your work matched the standards expected for the subject. But even if you were unlucky on the day with what was on the paper, or with your own state of mind, you still had your chance to do the best you could.
To find that a faceless system has allocated you to a lower grade, simply because your school hasn’t previously achieved much in this subject, looks like the epitome of systematic unfairness. Why did you bother to put in all that work, only to be pre-judged on the assumption that you’re homogenous with your older schoolmates?
That’s the embarrassing truth about algorithms. They are prejudice engines. Whenever an algorithm turns data from the past into a model, and projects that model into the future to be used for prediction, it is working on a number of assumptions. One of the more basic assumptions is that the future will look like the past and the present, in significant ways.
You may think you can beat the odds stacked against you by your low-attaining school, and your lack of extra-curricular extras, and your having to do homework perched on your bed in a shared bedroom, but the algorithm thinks otherwise. Isn’t it strange that we are repelled by prejudice in other contexts, but accept it when it’s automated?
Until now. Now school students shouting “Fuck The Algorithm” have forced a Government U-turn. Some of them seem to think the whole business was an elaborate ploy to punish the poor, instead of a clumsy attempt at automated fairness on a population scale. But some of them must be wondering what other algorithms are ignoring their human agency and excluding them from options in life because of what others did before them: Car insurance? Job adverts? Dating apps? Mortgage offers?
Some disgruntled Further Maths students will no doubt go on to write better algorithms, but that won’t solve the problem. As the RSS wrote to the Office of Statistical Regulation, “‘Fairness’ is not a statistical concept. Different and reasonable people will have different judgments about what is ‘fair’, both in general and about this particular issue.”
You don’t need Maths, or Further Maths, or even a 2:2 in Maths and Statistics, to question what assumptions are being designed into mathematical models that will affect your chances in life. Anyone can argue for their idea of what fairness means. Algorithms, and what we let them decide, are too important to be left to statisticians.
Join the discussion
Join like minded readers that support our journalism by becoming a paid subscriber
To join the discussion in the comments, become a paid subscriber.
Join like minded readers that support our journalism, read unlimited articles and enjoy other subscriber-only benefits.
SubscribeWhat is missing from nearly all discussion of this subject is that, due to the (unnecessary) lockdown, there were no good options. Using an algorithm was a bad option. Relying on hopelessly unreliable predicted grades was a bad option. Deferring a cohort’s progress for a year was a bad option. Trying to run socially distanced exams in the face of the unions’ refusal to work was a bad option.
In this no-win scenario, OfQual picked the least-worst option, and did their best to implement it. Their thanks has, wearily predictably, been to be pilloried for it by the media, who thrive on chaos, and by the masses whipped into yet another frenzy by this media hysteria. Media hysteria caused this problem by demanding a lockdown that was not proportionate to the threat of coronavirus, and media hysteria has now exacerbated this problem. At what point do we admit that the real problem underlying most others is the media itself?
Yes, and I think more and more people are waking up to the hysterical awfulness of the legacy media. I certainly hear more people talking about it – even the sort of people who are not normally paying attention to anything. I, for one, have gradually been carrying out my own ‘walk away from the media’ campaign over the last 20 years.
Maybe the government should start ignoring the media and actually govern for a change instead of performing cartwheels every time the Daily Mail says boo! However that would involve carefully considered policies and a level of competence they clearly don’t possess. They had all the media apart from The Guardian and the Daily Mirror on their side for the election, so how useless do you have to be to lose so much support.
The only sane choice was to run the exams. If we had a strong and decisive government this would have happened
The Teachers’ Union is too powerful. They were completely obstructive. Their desire to bring down a Tory government is so strong they were, and will continue to be, prepared to sacrifice our children’s education.
A good point. And has been pointed out in the article itself, even examinations (in normal circumstances) are far from perfect. News reports seem to be saying that about 70% of applicants secured a place at their first choice of university – that wouldn’t sound to me like a bad overall outcome, even in a year of normality. But, the news media cannot, just cannot, be happy without generating their endless supply of conflict and victims of conflict…
We are cursed with a yellow media that, like Kane, tells its staff: you provide the pictures, I’ll provide the scandal.
The methodology is as follows. Set an impossible target for the government to achieve – in this case that every single student gets the grades s/he deserves. Then hammer the government for whatever solution they offer.
Few in the media know that at least for A-Levels there is evidence every year of systematic overestimation of the grades students will achieve, especially by state schools and FE colleges. Even fewer understand that the higher the grade of a student the less room there is for over-estimation.
Agree there was no good option – but not that they did their best implementing the least worst option. They did a very bad job implementing the option, having rejected the input of experts (rss) in order to keep the process secret.
Additionally I don’t think there is outrage that not all students got the grade they wanted, but because of the obvious inequities – native speakers being graded “C”, take a subject with < 5 students at your school get grade inflation, take a subject with >15 students don’t. This implication was noted in Ofqual’s own paper, but simply accepted.
A lot of people are starting to see it, even those who I normally would be surprised to see not accepting what they read at face-value.
The legacy media has become another prong in the system that is seeking to demoralize Western nations. Maybe journalism needs some kind of ‘Hippocratic’ oath to uphold objectivity and integrity. Less opinion, more facts, and let readers draw conclusions.
Which coronavirus? If you mean SARS-Cov-2, please say so. We thought you meant one of the other ones which cause normal flu and the common cold.
Your sarcasm would be funnier if you knew that the flu is caused by influenza viruses rather than coronaviruses.
The teaching unions must be hugging themselves with glee. This mess is a direct result of their refusal to reopen the schools. Not only have they managed to punish the children of those working-class people who dared to vote for the Tories, they’ve caused huge damage to a Tory government. Mission accomplished! Funny, I thought their mission was to teach. Then again, I never went to university so what do I know?
‘Funny, I thought their mission was to teach’
The teaching unions are to teaching what Adolf Hitler was to world peace.
On the positive, a record year for results without the need for teachers. So we can reduce the education bill!
Utter balls. I have a lot of friends who are teachers, who were teaching in lockdown, either remotely, or in situ because of the need to teach children of key workers, remember. And several who were involved in the assessment of kids for GCSE and A Level. This year’s workload to prepare those assessment predictions was higher than normal, and in end that work was largely ignored. But that’s ok, you fulminate away. Perhaps if our assessment system hadn’t been yanked back to being so single exam centric it would have been less of a problem, because moderated CA results would have had more weight. Whose bright idea was that? Do you remember?
(Disclaimer: I am not a school teacher, but did work in academia for well over a decade, so am reasonably aware of assessment modes, examination processes, and admissions)
Well said. There seems to be an unpleasant tendency in the comments section of UnHerd to traduce the teaching profession for no good reason.
I think it is the unions not the teachers themsleves, but I understand your point
And then try having a conversation with a teacher about how the union they blindly support is the problem and they should perhaps stop contributing to the problem.
I’m well aware that many teachers were desperate to get back into the classrooms. I was talking about the unions – try reading the first three words again if that’s not too difficult for you.
The problem is, as always, to lump everyone together. I am certain that most teachers did their best to teach over the last few months. Many taking classes, as you say.
But the unions are another matter, I understand they were consulted over the algorithm and agreed with it, and have been obstructive all the way through, preventing a return to school. Definitely not putting the needs of the children first.
You may be aware of assessment modes, examination processes and admissions but you cannot differentiate between teaching unions and teachers. Andrew S is quite correct that the unions were to blame NOT the teachers. The unions clearly explained to the teachers that they were not to cooperate with the Conservative Government and this cowardly inaction got them what they wanted – prizes for everyone with massive grade inflation and bad press for the tories. The teachers were just the cannon fodder to be used against the Government.
Two words, Public sector … failure.
OK, it’s really three words but you get the meaning. Another story of the public sector failing the public. Every week there is another. We get so many they become white noise in the background. Yet at a national level the scale of such failures is truly staggering.
If it’s not the failure of PHE to provide safety equipment during this crisis, it’s a local council (Braintee) spending ten years on a Local Plan to have the entire thing rejected as not fit for purpose… Yet the people who delivered ths failure are still being employed.
Exactly, and this chronic, disastrous failure of the state can be witnessed across most of the western world. It’s been going on for at least 50 years and it is the main reason why the West is bankrupt. Trillions upon trillions are thrown at the state and its pernicious incompetence, all of which is wasted. And nobody is punished. Indeed, those responsible are untouchable and handsomely rewarded.
This is one of the reasons for the emerging ‘civilizational states’ written about by people like John Gray here on Unherd They look at the West with its perverse incentives and disastrous outcomes and, quite sensibly, decide that it is not a model to follow.
Those “civilisational states” – Russia, Turkey, China, increasingly India – are one and all nations in which the state has powers which governments in Europe and the Anglosphere can, for the most part, only dream of. There are no checks and balances in China, the rule of law has been progressively undermined in Russia and Turkey, and the same process is now under way in India. If what’s happening in the West represents a disastrous failure of the state, then why are the alternatives to the Western model so assertively statist?
That’s a fair point. And I often say that in many cases the only thing worse than the public sector is the private sector. However, this does not alter the fact that the public sector is, for the most part, chronically devoid of all accountability or competence.
So Serco and the other private sector did so well?? I’ll keep with the local authority public sector thanks.
What normally happens is that these private companies are set up by ex public sector senior management who have the contacts to get the public sector contracts. Remember when Railtrack was privatised the senior management won the contract because they had the inside information to value the business. They made millions overnight.
Yes, I noticed this in Higher Education. Highly-placed bureaucrats receiving big pay-checks while ensuring that those under them work harder for less pay.
Well. The third one is redundant really. Could have left it out
“Exams are a blunt instrument. They assess performance on the day”
Life is a blunt instrument. It assesses performance on the day. You get one chance. Get over it. You can’t spend all your time in a safe space
Life actually assesses performance continuously over three score years and ten (or whatever). Similarly, achievement in education should be continuously assessed over the years from five to 18 (or five to 21, for those who go to university).
I don’t know what world you’re living in. Must be the public sector. My world, it’s a continuing series of examinations. You don’t cut it, you fail. Nobody’s interested in your course work or making allowances for bad days.
“Coursework”, in my analogy, is what you or I or anyone else does in his or her job every day. The model of education we have now, dependent on exams when you’re 16 or 18, is akin to starting a job and being told – “Don’t worry about your performance now. We’ll evaluate it in five or seven years.” Is that what it’s like in “your world”? I suspect not.
Surely it’s better to be judged on what you’ve learnt by the end of the process (as reflected, however imperfectly, in exam results) than on what you knew at the beginning or half way through?
Well, yes, in a sense, but the exam system as we have it now does advantage pupils who think and work in a certain way, and the skills needed to pass exams may not be the same skills that are useful in many of the jobs they will ultimately apply for. I don’t really think it would be useful to evaluate the performance of children of 12 or 13, but the system we had was I was a boy seemed to work pretty well. I was assessed by a mixture of exams and coursework over the last four years of my school education; I learned different skills and ways of approaching material. I don’t know if it was the ideal system, but I think there are worse options…
Don’t disagree – it’s just a question of getting the mixture right. I think a little rebalancing in favour of final exams is probably in order. And I’m sure you’d agree that education isn’t simply about providing skills for employment, how about expanding the mind?
Oh absolutely – but I think the expanding the mind part is certainly better done through coursework that exams.
“Better far than praise of men, ’tis to sit with book and pen”
For God’s sake Derrick, the only answer is send the Brat(s) to Eton.
You are ‘guaranteed’ a first class education, almost immediate access to Oxbridge, and membership of the worlds most elite club. “Dives in omnia”.
What more could you want?
Ah, Dives in Omnia. Puts me in mind of the TV classic Porterhouse Blue, now no doubt added to the Index Prohibitorum. No, if given the chance again it’d be home schooling for my brats
Every grandchild should be taught to sing the Porterhouse Anthem!
“Sed choro sonoro”
You are probably correct that Tom Sharpe has now been condemned to the ‘sin bin’ of political correctness. That was a classic production, so beautifully caste and a joy to watch!
When you are driving and a truck is coming towards you on the wrong side of the road you have only one chance to make your decision. You are definitely assessed by your performance on that day, at that second. An extreme example, maybe. But life is full of ‘assessing performance on the day’. Some are even life threatening.
True enough, but again (see my reply to Perdu en France below) that’s kind of what I was saying. In real life we encounter difficult and dangerous challenges all the time – not just on a few designated afternoons in June for which we’ve been preparing for five or seven years.
In which case the exams are, in fact, easier than real life. Wouldn’t you say?
Yes. That’s part of what I was complaining about.
Across the board, the real government is the narrow pool of self-satisfied career “experts” who advise ministers.
Irrespective of the party in power, the elected politicians will get the blame for what the real government does.
The “establishment” needs wholesale reform, both in structure and in personnel. But from where will one draw reformed personnel when the entire educational system has been narrowed to produce these self-satisfied problem minds?
I agree. Politicians all come from the same kind of background. We need a system that represents all kinds of socio-economic groups. Politics should also be a part-time gig, not a career. Many politicians are divorced from what is actually happening at the grass-roots of society.
Teachers have let us down. The systemic overgrading institutionalised by the performance assessment of schools reflects badly on them, so unprofessional . And the inevitable consequence is universities will not have the places required. But who wants to go to university when the tutors seem obsessed by finding ways of ‘teaching’ remotely.
I admire our bin men who have just got on with it during the lockdown – just compare their attitude to the teaching/academic community. Never thought I would write this…
Bin men and postmen have done a superb job, but they are working outside, where virus transmission risk is low, and mostly alone. Working in the classroom, indoors with a large number of closely gathered people, is by definition a much more high-risk activity.
Also, while most schoolchildren and students are by virtue of their age in a low-risk category, this isn’t necessarily true of the people they may come into contact with when they go home or go out. Classrooms and lecture halls are tailor made to spread the virus, and to spread it to people who by virtue of the age are likely to end up being symptomless carriers. Then an apparently healthy schoolboy carries COVID-19 back to his elderly grandmother, or a student passes the virus to the 60-year-old manager of the local pub…
I admire supermarket workers who just ‘got on with it’ and they don’t work outside. As for your bin men – mine travel 3 to a cab. Close proximity. No masks.
Sure – I admire them too… but since a very early stage, supermarkets have been restricting the number of people who enter at any one time, precisely for the sake of social distancing. More schools and universities are full to capacity most of the time. They’d need immediately to build twice as many classrooms to follow suit. Otherwise, the point about classrooms being likely vectors of infection stands.
Or, they could have done what some other countries did and have half the classes in on alternate days.
Where there is a will there is a way, but it would only have worked if the unions had worked with the government rather than against it.
Sure, good option. I’d have endorsed that one. Having said that, most other countries (apart of course from the US) aren’t run by governments that are constantly attacking teachers.
In the good old days of Ancient Athenian Democracy, those public officials who were judged to have produced “inadequate performance”, were in many cases put to death.
Perhaps we should reintroduce such a robust system of public accountability? Any suggestions of for who should be first?
Too many to mention. But I wouldn’t put them to death, I would put them to work at something useful like sweeping the streets etc.
You are too generous sir!
Once again we have a system that is unequal for Education. Perhaps this mess will accelerate the long over due move to post A level university applications for University. Yes Centre assessed Grades are better than the alogorithm. At least they were based on almost a years work and hopefully some mocks. For university applications normally teachers predict grades based on only part of the course and without pupils having taken exams based on those taken in the Alevel. The predictions are often wrong especially in Schools that have smaller cohorts applying for Russell group universities. There is also evidence that Schools over predict as Universities often accept lower grades in the end. I would also suggest that some independent Schools might over predict due to parental pressure.
Strange that universities report they are struggling to find places for everyone, now that every student has achieved their predicted grades. How can they be short of space when they are planning to deliver lectures and tutorials via video link?
Consider lab space. Some subjects, e.g. Chemistry, require students to undertake labwork. Now this has to be socially distanced, and there will be more students, so there will have to be more lab sessions, increasing staff costs etc. Not everyone, fortunately, does Eng Lit.
There is another reason why teachers massively overestimate the grades of poorly performing students. If a teacher predicts a poor grade for a student, the teacher will be expected to introduce a range of measures to improve this student’s performance, all involving extra work including the provision of ‘intervention’ classes. Before citing this as yet further evidence of the laziness of teachers, remember that this would apply even if the reason for the poor performance is the pupils’ absence from class. In poorly performing sixth forms, many students attend as little as 50% of their lessons. Others stop coming in at all but remain on the school roll. Many parents, who have dutifully sent their children into school up to GCSEs, no longer see themselves responsible for ensuring that their children attend A-level or BTEC classes.
This is likely true. Although I would argue that by luck or design it works well because it tests not only knowledge/ability, but the capacity to deliver under a certain amount of pressure (both time and mental pressure).
That is surely more valuable not only to future employers, but society as a whole and the very people themselves to realise the importance of being able to apply what you know.
If the following happened:
It would arguably be less than optimal, as it would overevaluate bright people who struggle to apply what they know, or at least who crumble under any sort of pressure.
Given that all this could be predicted, the question that is not addressed above is why Gavin Williamson adhered so stringently to this disastrous algorithm. There are only two possibilities: either he is lazy, and chose not to investigate the flowchart that was the algorithm in any detail, or he and his political puppeteers liked the sound of what the algorithm was doing.
More likely he liked the sound of what he was told the algorithm was doing, i.e. maintaining the credibility of the grading system from year to year, while being fair to disadvantaged students.
Or maybe he listened to the ‘independent experts’ (OfQual) and believed them. Foolish, I know, given past performance.
Actually, from what I’ve seen, that’s not quite how teachers were asked to rank. They weren’t asked to do it in class order, but according to to grade.
So, for example, they’d take all the students who they’d expect to get a B and rank them from
strongest to weakest. The problem with this is that could map students down the the b/c boundary, even if the weakest in that group would be expected to get a middling b – they’d be marked down when they shouldn’t be. This is not necessarily the same as ranking only on class ranking order, and seemingly had implications when mapping grade boundaries during moderation processes.
This year was anomalous. So worries about grade inflation could have been laid aside earlier, simply because assessment data was not like for like compared to previous samples.
And then there’s the algorithm itself. Why is this algorithm not in the public domain? Given its targets and its impact and public interest implications it’s difficult to understand why the algorithm is not subject to inspection and review by the very people who would have been able to point out obvious biases in methodology. If anyone says “commercial sensitivity” ask yourself why the examination of academic attainment should be commercially sensitive.
Perhaps this illustrates the need for more openness and transparency around this mass data and the way it is processed
Moral of the story. Don’t pander to the liberal left and close schools. Or if you do, make sure you keep years 13,11 and 6 at school full time and make sure they sit their exams. There was no positive outcome for this blunder.
Liberal Left, now there is a good oxymoron! When were you last at school?
I think some good might come out of the result of this fiasco. Obviously Algorithms will be less trusted, perhaps even ignored.
Think of the benefits, no more “R” numbers, no more lock down!
And best of all, no need to worry about climate change – all the predictions are based on Algorithms.
Trebles all round.
Computer models, economic forecasting and climate modelling are just astrology in a spreadsheet. I have made my living from computer programming, sort of, for 24 years and it always amazes me how people will trust any figures as long as they are shown on a computer screen. That includes the business people assigned to check the results of the programs! Most people find lots of figures mind-numbing and prefer just to let them pass without applying proper scrutiny. One day I won’t bother doing all that hard programming stuff and just present a page of random numbers in a pretty graph and see if it gets accepted (not really).
The grades for the majority, particularly of state school pupils would be expected to be poorer than previous years due to the fact they missed the final 2 months recap and revision classes that would be expected to significantly help prepare them for the exams. The private schools were seen to have provided the most pupil support through the closed schools and logically they have been graded higher. The media and education staff cannot seem to face the consequences of their actions through the media, in pushing for the lock down of schools and misinformation about pupil and staff risk; and the education staff response to having a ‘can do’ attitude being generally lacking by the majority. The response also illustrates the modern inability to accept that not all young people can get what they want and that shouting and moaning continuously as the young people, parents, education staff and media have done will work , as continues to be evidenced through government U-turns. This is mainly a middle class trait I would guess.
I think we should simply ‘trust our teachers’, as some of the WSP-printed placards said. In exchange, of course, we should deduct 12.5% pay from any teacher whose results next year fail to meet the proportion of A* and A grades this year, which are 12.5% higher than usual. Can we look forward to a 12.%% rise in First degrees in three/four years time? Maybe I’m being unfair: perhaps this is just an unusually brilliant tranche of students.
In terms of the overall grades awarded, it will be interesting to see the drop-out rate from universities in the next few years.
Williamson must be faulted for buying into this hokum. In the extraordinary and quite unprecedented circumstances it would have better to go for the line of least resistance – teacher prediction. Surely this debacle was not unpredictable?
But the fault lay with the teachers who tried, and now have succeeded, in gaming the system. OfQual were was right to ignore them – they lied! When these teachers should have the common sense to provide a distribution of results that were in line with previous years. Although the majority were honest, and their complaints beforehand ignored. Now we have 2020 as the year that “everyone got an A” and will be disregarded by employers. As well as uni’s not being able to differentiate, and those with increased grades now at an equal level to those around them – it doesn’t change anything!
A point I have not heard mentioned anywhere. What happens if the lockdown is repeated due to another wave of this or another virus? Next time teachers will know that they can write in their students’ grades. Parents will know that teachers can do this and many will put immense pressure on teachers to overestimate their children’s grades.
We can’t go on having lockdowns for this virus, or the next one. Our economy will soon be on its knees, and novel viruses are an unpredictable fact of life.
We will have to keep working next time, because ‘furlough and borrow’ won’t be an option.
Don’t rule out future lockdowns. Firstly, this year’s lockdown has been used by political opponents of the government especially in the US and also in the UK to bring the government down. This may be repeated even if a lockdown is not the best course of action. Secondly, a future virus might have a higher fatality rate with fatalities spread more evenly through the population, i.e. including healthy working age people. Thirdly, there is the possibility of biological warfare.
Your first point; yes, but why would a government order a lockdown that will be weaponised by the opposition and used against them?
Second point; the next virus may well be more dangerous but we can’t afford to do this again. We’re broke.
Third point; it may well be a weapon next time, but if you wreck your own economy you hand victory to the attacker. No shooting required; war over.
bull-schiff. so-called “education” by lib-t**d DemonRATs gone ridiculous. anybody spending $ on “higher” edu other than STEM is STUPID!
Great article, thank you for sharing! I’ve been trying to find the ‘golden source’ of the algorithm process via Ofqual and anywhere else – would you or anyone in this conversation be able to point to the specifics of the algorithm flow?
Reason for asking is I’ve seen some statements of the algorithm inputs (all ‘friend of a cousin of a teacher’) stating absolutely the algorithm flow, and I personally would love to have the actual stated flow to understand it’s principles. thanks in advance.
I certainly didn’t expect to see teenagers with placards taking to the streets to shout “f**k The Algorithm!”
Really? You obviously don’t spend that much time on the internet.
Algorithms were never “left to statisticians”. This, like many others, was evolved as a social management tool for what was (wrongly) understood to be a routine job to be completed under difficult circumstances and designed accordingly. Any surviving Scottish educational pyschologist from the time will tell you that in the days of the (in Scotland) 12+ they were instructed to bend the marks in favour of boys, who were believed to have a different rate of intellectual development from that of girls, to enable roughly equal numbers of both to be admitted to “senior secondary schools”. In those pre-algorithm days this activity was known as “moderation”, carried out, nonetheless to nationally prescribed criteria. How many borderline female careers were wrecked in a country not famed for fee-paying girls only secondaries outside Jean Brodie’s Edinburgh?
Perhaps they should be? After all, statisticians have a professional body with a code of conduct that requires them to act in the public interest with professional competence and integrity.
So do most professions, even teachers. But supposing that any of them are bound to “act in the public interest” depends on whatever the “public interest” is agreed to be. And by whom. Most of the headlines in this discussion are about prejudice. How can any form of social management be insulated from it?
Of course. I would hope, though, that professionals would explicitly ask themselves the question as to what is in the public interest in any given situation rather than simply assuming that whatever they would think or do is ipso facto the right thing to do. Someone reasonably competent when asked to design an algorithm based on historic data should be aware of the possibility of algorithmic bias and take it into account in their work. At the very least, if not given the time to assess whether or not there actually is bias or devise means of correcting for it, should explicitly report back to their customer that there is the possibility of bias which they have had had the time or resources to test.
At the risk of pointing out the bleeding obvious, which no one seems to have done, you can’t rely on teacher awarded grades because the teachers have a dog in the race. Good pupil grades reflect well on the teachers and likewise bad grades reflect badly. Teachers are almost in competition with each other to overgrade their students.
Indeed, why stop with the teachers, just let the pupils choose their own grades?
There must be independent moderation if the grading system is to be relied upon.
Likewise it’s equally obvious that any moderating process is going to be harder on poorly performing schools than on well performing ones. Teachers don’t need to overgrade their pupils when the school gets good exam grades anyway and the best grades are most likely to be seen in schools where parents are paying tens of thousands on pounds for their children’s schooling than those where they are not.
Algorithm was used by Brixiteers, and Tory electioneers to win and now they have failed. Did not Cambridge analectica also use algorithm for their own ends. Where does Cummins fit in all this. Public Health has gone next OfQual. Find a scapegoate.
“Where does Cummins fit in all this” – amazed it has taken all this time for his name to appear. Probably not at all and six words later you are accusing people of ” find a scapegoate (sic)” – oh the irony.