Don’t fight dirty with statistics

Last week, as pointed out by Peter Franklin, The Living Planet Report was released. It studied at the numbers of vertebrate animals from 3,000 species in 14,000 different local populations, and found that, on average, the populations had declined by 60% since 1970.

That sounds, and is, catastrophic. But what it doesn’t mean, despite how many media outlets reported it, is that there are 60% fewer animals than there were in 1970.

Imagine if, instead of examining 14,000 populations, they’d only looked at two: rats, of which there were 1,000 in 1970, and white rhinos, of which there were 10. And let’s imagine that between 1970 and 2017, both populations had lost five members, so there are now 995 rats and five white rhinos. The average population has declined by more than 25%. But the total number of actual animals has declined by less than 1%.

This is a silly example, but it demonstrates that the “decline in average population” figure and the “decline in actual number of animals” figure don’t have to be related at all. I’m not saying that everything is OK and we don’t need to worry. There really is a human-caused mass extinction going on, and besides, there are good reasons to care more about the death of five white rhinos than of five rats. But it does mean the headlines were wrong.

Still, that didn’t stop several people from telling me off for my pedantry. Sure, the headlines might not be exactly true, they said. But they grab public attention and get the statistic reported. If we point out that they’re false, then people will think there’s no problem. If we – people who care about the environment and wildlife – let the stat stand, it will convince people to take action. We can win.

I know this attitude. I’ve bumped into it before. A statistic went around in 2014 that 10,000 people had died within six weeks of having their disability benefit stopped by ATOS, the company doing the DWP’s disability assessments. It sounded pretty awful; as though ATOS was declaring people fit for work while they were on their deathbed. But it turned out that the stat actually included people who had died up to six weeks before their benefit stopped.

That is, people died, because people sometimes die, and then the company successfully stopped paying benefits to them, because they were dead. It wasn’t that they were dying just after being told that they didn’t need their benefits any more.

Again, this doesn’t mean that what the DWP and ATOS were doing was OK. They were hiding statistics that could have shown whether people really were dying early; the coroner blamed their assessments for at least one person’s death, and by ATOS’s own admission the arrangement wasn’t working. But 10,000 people did not die after being declared fit to work.

Again, I was told that this is quibbling with statistics. Who cares about the exact number. The point is, people are dying. Nitpicking the figure just gives the enemy ammunition.

This is a pretty standard position. A few years ago, a claim went around that men are 32 times more likely to be struck by lightning, and 11 times more likely to be killed by a meteorite, than they are to be falsely accused of rape. The claim was spectacularly false, of course. (I mean, come on. Literally no one has ever been confirmed killed by a meteorite. It doesn’t have to be very likely to be more likely than that.)

While it is true that false rape accusations are uncommon, those figures were off by a factor of several thousand, because the author had compared the risk of a man being accused of rape each time he has sex with the risk of dying in a meteorite strike in your entire life. But it went enormously viral, because it fit a popular narrative. And when its falseness was pointed out, its fans defended it as the sort of weapon – “war and fire”, they called it – that we, the good guys, need to use to win in the war against the bad guys.

It’s not always that people are consciously thinking “yes, that is false, but I will say it anyway”. Part of what’s going on here is simple motivated reasoning: you don’t bother to fact-check stats that support your side. When we see very obviously false stats like the ones above, a little alarm in our head should go off, but if it’s a ‘helpful’ stat sometimes we ignore it.

There is, also, a mindset that – even once the error has been pointed out – wants to cling to it, because it is a useful weapon. That is counterproductive. We’re living in a world of increasing political polarisation. One of the problems behind it is that political opponents increasingly fail to share not only opinions, but facts: we don’t agree whether the world is warming, or whether immigrants are hurting wages, or whatever. That happens even when we don’t want it to. If we start knowingly making stuff up because it’s convenient, it will only make that process worse.

Aside from driving yet another wedge into the ever-widening gap in our society, it undermines our ability to trust things in general. If I start to think I can’t trust stats and evidence because most of them have been made up or gamed by partisans, then I won’t be able to make decisions off the back of them. Public policy suffers if the electorate can’t get access to good information.

In any case, if something is really bad, then you shouldn’t need to exaggerate your statistics to make your point. The holocaust of vertebrate species (and invertebrate species) is truly awful. Species have been devastated; some by 60%, some by less, many by more. We are living in a world with far fewer incredible creatures in it than the world our ancestors lived in. That is terrifying and tragic enough without making stuff up. False rape accusations really are rare; you don’t have to say they’re 30 times rarer than lightning strikes.

In fact, if you do make stuff up, you give your enemy ammunition. The DWP could point to obviously false claims about 10,000 deaths and say that it was all stupid hype. Anti-feminists can say, look, the feminists lie about false rape accusations: why should we ever believe them?

But this is neither a game, nor a war. The information around us is a common resource. We all take from it, like drawing water from a well. But we contribute to it as well, filling it up with facts and stats that we find. We can try to replenish it with useful, relevant, true things, based as best we can in reality.

Or we can poison it with falsehoods, as Donald Trump does, in the hope that it kills our enemies before it kills us. Poisoning the well can feel good – it can even feel like we’re doing the right thing – but in the end we all lose. We need to be right, because there isn’t anything to win.

---

Tom Chivers is a science writer. His second book, How to Read Numbers, is out now.

TomChivers