by Tom Chivers
Wednesday, 6
May 2020
Reaction
16:37

I told you so! 100,000 tests was a foolish target

When people say “I hate to say I told you so,” they never mean it. I absolutely love to say I told you so. So, without further ado: I told you so.

I said last week, when we were all on tenterhooks waiting to find out whether Matt Hancock had personally stuck swabs up the noses of 100,000 people (subs, check this please), that it was going to be a “hotbed for Goodhart’s law”.

Goodhart’s law: as soon as a measure becomes a target, it ceases to be a good measure. You want schools to give children a good education, in some complex but real way; you notice that the schools that give children good educations tend to have lots of children getting high grades; so you tell schools that they all need to have 50% of kids get five A* to C grades at GCSE. And then you notice that suddenly lots of schools are getting the grades without the actual quality of education going up, because they’re gaming the system.

Or, you tell people that you absolutely must reach 100,000 tests by the end of the month, because you hope that the number of tests performed will be a good proxy for the quality of the testing regime and its effectiveness in fighting Covid-19. But then you notice that actually more than 40,000 of the tests you say you’ve performed have in fact not been performed at all but have only been mailed out, that the numbers tested dropped immediately back down, and that only 7% of those mailed to care homes have so far been carried out.

A couple of people suggested to me that the 100,000 target pushed the huge increase in testing; even if corners were cut and goalposts shifted, the overall regime has been boosted enormously, and it wouldn’t have been possible without that super-ambitious and very public target. They may be right; it’s a reasonable case to make.

But it strikes me that the goalpost-shifting and stats-juking was entirely predictable given the very public and simplistic measure, and that some relatively simple steps could have been taken to make a more nuanced metric or set of metrics that drive the improvement nearly as well but that capture more closely the thing you really care about.

(You can listen to me blethering on about all this on Radio 4’s More or Less, if such a thing appeals.)

Comment


  • May 8, 2020
    I was astounded when he came out with this. Hasn't he taken Politics 101? I can only guess it was a most unpolitician-like attempt to motivate people: more befitting of a football manager, but showed a lot of b****, for which many people gave him credit. It was very funny when he announced 122k.... Read more

  • May 7, 2020
    What is most sad about it is how gutted MSM was when the figure exceeded the target on 30 Apr and 1 May and how obsessed they are now with pointing out it is below target. What matters is whether we have the testing capacity and associated logistics to support the testing strategy for the next... Read more

  • May 7, 2020
    In any numbers game it's always going to give someone a stick to beat you with, and with our venomous hateful press even achieving it won't help. Read more

To get involved in the discussion and stay up to date, join UnHerd.

It's simple, quick and free.

Sign me up