Are you a scout or a soldier? Are you trying to find things out or defend a position? It’s a metaphor that the author and rationalist Julia Galef uses. If you’re in a soldier mindset, you’ll treat your opponents’ arguments as enemy soldiers. Some pieces of information are useful to your enemy’s argument, and need to be shot down. Some of them are useful to yours, and need to be bolstered.
But if you’re a scout, you’re trying to find out what’s really there. You might hope that there is a conveniently placed bridge across a river, or that your enemy’s supply lines are vulnerable; but your job is to establish whether that’s true or not. When you look at purported facts, you want to determine how true they are, not simply how useful.
I was thinking about this when I read a story in the Guardian last week saying that British schools are “institutionally racist”, based on their own examination of the data around exclusions. The story said that the fixed-term exclusion rate for black Caribbean pupils — that is, the rate at which they are kept off school for a given period — was five times higher than that of white children in some areas. Roma Gypsy and Irish Traveller pupils were excluded at an even higher rate.
An accompanying analysis linked this to “racial micro-aggressions” and “racial micro-invalidation”, such as young black people being censured for wearing “inappropriate” haircuts; the implication being that the problem was that teachers were racist, intentionally or otherwise, towards their pupils.
This is a fraught topic. It is obviously deeply worrying if black Caribbean pupils are being excluded at high rates. But I am extremely hesitant about assuming this is an uncomplicated story of racist teachers (or institutionally racist schools), because I think there has to be a lot more going on here. I think this is a case of looking for arguments that support your position, not one of going out and trying to find out the actual facts as objectively as possible. I can understand why the Guardian might do this, because racism really is terrible, and calling attention to it feels sensible. But they’ve flattened a complex problem into a simplistic narrative.
Here’s what I mean. First: the Guardian has been really shoddy in its reporting of the data.
For instance, they say that the exclusion rates for black Caribbean boys “are up to six times higher than those of their white peers in some local authorities”. Which is true. But there are 343 local authorities in England. They will have very different ethnic breakdowns and will vary in all sorts of ways just by chance. The Guardian story sort of acknowledges this – Cambridgeshire, the worst-affected area, has “a relatively small number” of Caribbean students, they say, so the data isn’t trustworthy.
But even if black and white children were excluded at exactly the same rate nationally, you might expect to see some quite large differentials at local level, just by chance. Cherry-picking the worst areas, unless you have some underlying reason to think that they really do have a worse-than-average problem with racism, is bad practice.
If you look at England as a whole (the data is here), the problem doesn’t go away, but it does become much smaller – there are about 10 exclusions per 100 black Caribbean children per year, as opposed to about six for white British children. (Note that that’s not quite the same as saying “About 10% of black Caribbean children are excluded”, because some children are excluded more than once.) Still, you might say, that’s hardly a negligible problem, if black Caribbean kids are excluded 60% more often than white ones. But the data is not the only issue with the reporting.
The story makes, as far as I can see, absolutely no effort to check whether there might be any confounding variables. It sees that black Caribbean children are excluded at higher rates, and assumes that the one causes the other directly. By analogy: famously, ice-cream sales and drownings tend to go up and down on the same days. Do ice creams make you drown? Probably not — more likely, it’s that hot weather makes people buy ice creams, and also makes them go swimming (and then drown). The weather is a confounding variable which influences both ice-cream sales and drownings, and makes them look as though they are connected. But when you control for the weather, they’re not.
Is there any possible factor that might be connected to both school exclusions and ethnicity, and which might make them look as though they are more correlated than they are? An obvious one might be poverty: it could easily be that poorer children are more likely to be excluded, for any number of reasons, and that black and ethnic minority children are more likely to be poor.
You could, for instance, look at how many children in each ethnic group are eligible for free school meals (FSMs), and see whether, when you control for that, you still see the link between ethnicity and exclusions.
Annoyingly, the Department for Education’s website doesn’t let you break the data down by ethnic group and FSM eligibility1. But the sainted Kevin McConway, emeritus professor of statistics at the Open University, helped me find a rough-and-ready workaround. We found an FOI release of data showing free school meals by ethnicity at each educational key stage — from 2017, which isn’t ideal since the DfE data is 2018/19, but hopefully it’s close enough. We took a crude average across the stages for each ethnic group.
Then we looked at exclusions by FSM eligibility for English pupils as a whole. The exclusion rate for children on FSMs is a little under 14%, and for those not on FSMs it’s a little under 4%.
This is a non-trivial piece of work which we shouldn’t have had to do. The Guardian should have. It is the most basic and obvious piece of data-checking imaginable, and the fact that they didn’t do it makes me think that they settled on their chosen explanation long before the facts came in.
As it happens when you control for FSMs, it doesn’t make all that much difference. But they were only right by accident.
On this chart, the rate of fixed-term exclusions is on the X (horizontal) axis, and the rate of exclusions after controlling for FSM eligibility is on the Y (vertical) axis. If a point falls below the diagonal line, it means that poverty is “causing” some of the link between that ethnicity and exclusions, and controlling for it makes it go away somewhat. If a point falls above the line, then that suggests that the ethnic group is actually doing worse, by comparison to its poverty level, than its headline numbers.
Most of the dots are pretty close to the line, suggesting that the overall picture is not all that different from the one painted by the Guardian’s data:
But this is important. I read that Guardian story and thought “hang on: they haven’t controlled for poverty. Might that be what’s going on here?” And then I checked, and it doesn’t look like it is, so I’ve included it. I tested my own theory and showed the results. If you’re actually trying to work out why something is happening, that’s what you need to do, not just say “Look, A and B are correlated! Therefore A causes B, no further questions. Ice cream definitely causes drownings.”
It’s also worth having a closer look at that graph. As the Guardian said, black Caribbean children are excluded at a higher rate than white children, and Roma and Irish Traveller children at an even higher one. (In the case of Irish Travellers, that seems to be related to poverty, but the numbers of Traveller children are tiny, so we can’t have much confidence in it.)
There’s something worth noting, though. The hard blue lines are the national average. Black Caribbean children are well above it. But black African children are somewhat below, and Indian, Chinese, Bangladeshi, Pakistani children are hugely below. If British schools are institutionally racist, then it’s a really laser-targeted racism, which discriminates against the kids of Trinidadian descent but actively encourages the ones of Ghanaian heritage.
Also, white children are excluded at above the national average – a white child is about 50% more likely to be excluded than a child of black African heritage, and about three times as likely as one of Indian heritage – and they do worse than their poverty levels would predict. When you look at overall educational performance, you see a similar pattern: white and black Caribbean children do badly, when you control for FSM eligibility. Black African children, and Asian children, consistently do better than average.
No doubt there are many problems with racism in British schools. But I don’t buy the idea that the differences in exclusion are driven by it; either that racism is bizarrely specific, or whatever is behind this problem is upstream of schooling.
You could easily come up with some alternative hypotheses. For instance, someone pointed out to me by email that Roma and Irish Traveller children have by far the highest rate of absenteeism, more than double the next highest group, perhaps because their tight-knit communities do not place very high value on school-learning provided by the state. You could also tell a plausible story about black Caribbeans feeling that the British state has not traditionally been on their side, and that making them place less value on state institutions such as schools.
I don’t know how it fits with the problem with white British children, or why Asians and black Africans have such a different response, but each group has very different histories and therefore different cultures and attitudes.
But if you want to say that schools are institutionally racist – that there is something, whether explicit racial attitudes among teachers or some structural problem with the education system that harms black Caribbean children – then you have to do a better job than simply pointing to some data that on the surface tells a convenient story, not interrogating it at all, and moving on. It’s not as if there aren’t much more clear-cut examples of racism in the world; you shouldn’t need to point to ambiguous-at-best situations like this and call them racism to bolster your argument.
This isn’t the only example in recent days. Another Guardian story found that 60,000 “racist incidents” were recorded in UK schools in the last five years, but made no effort to help us understand if that was a big number or not; how many schools are there? How many pupils? (There are about 24,000 schools in the UK, teaching about nine million pupils; that doesn’t make the numbers OK, but it’s vital context.) The New Statesman ran a piece about excess deaths going up in the UK even before the pandemic, blaming it on austerity; but forgot to control for the growing and ageing population, which muddied the story somewhat.
This all makes sense if you see the public sphere as a battleground, if you see debate as war and arguments as soldiers: and perhaps that is a sensible attitude to have. Perhaps if you want to get things done, to defeat racism or win elections or any such goal, perhaps you need to have the mindset that sees inconvenient complexity as something to be destroyed.
But I would rather people tried to think a bit more like scouts, to ask where data comes from and whether it means what you think it means. And if you do find out what’s really true — where that bridge is, how the enemy’s troops are arrayed — you can fight against it all the more effectively.