A classic case of Goodhart's Law...
When people say “I hate to say I told you so,” they never mean it. I absolutely love to say I told you so. So, without further ado: I told you so.
I said last week, when we were all on tenterhooks waiting to find out whether Matt Hancock had personally stuck swabs up the noses of 100,000 people (subs, check this please), that it was going to be a “hotbed for Goodhart’s law”.
Goodhart’s law: as soon as a measure becomes a target, it ceases to be a good measure. You want schools to give children a good education, in some complex but real way; you notice that the schools that give children good educations tend to have lots of children getting high grades; so you tell schools that they all need to have 50% of kids get five A* to C grades at GCSE. And then you notice that suddenly lots of schools are getting the grades without the actual quality of education going up, because they’re gaming the system.
Or, you tell people that you absolutely must reach 100,000 tests by the end of the month, because you hope that the number of tests performed will be a good proxy for the quality of the testing regime and its effectiveness in fighting Covid-19. But then you notice that actually more than 40,000 of the tests you say you’ve performed have in fact not been performed at all but have only been mailed out, that the numbers tested dropped immediately back down, and that only 7% of those mailed to care homes have so far been carried out.
A couple of people suggested to me that the 100,000 target pushed the huge increase in testing; even if corners were cut and goalposts shifted, the overall regime has been boosted enormously, and it wouldn’t have been possible without that super-ambitious and very public target. They may be right; it’s a reasonable case to make.
But it strikes me that the goalpost-shifting and stats-juking was entirely predictable given the very public and simplistic measure, and that some relatively simple steps could have been taken to make a more nuanced metric or set of metrics that drive the improvement nearly as well but that capture more closely the thing you really care about.
(You can listen to me blethering on about all this on Radio 4’s More or Less, if such a thing appeals.)