£22 billion was made available — so where are the results?
You may have forgotten that we have a test-trace-and-isolate (TTI) system. I almost had. But it was quite an expensive thing; £22 billion has been made available for it, although I don’t know exactly how much has actually been spent. It’s also widely said that it didn’t work very well.
Interestingly, a Government report came out last week which seemed to confirm that point. But it’s been (as far as I can see) largely ignored; the estimable Full Fact has just published a piece on it as I’ve been writing this, and the Guardian mentioned it a few days ago, but other than that I don’t think it’s got much attention.
Partly, that’s because the report was not press-released; partly it was because Dido Harding, the chair of NHS Test and Trace, somewhat overstated how positive its findings were, in front of a Commons select committee.
The report was based on a mathematical model, rather than an observational study. It found that the entire TTI system reduced R by between 18% and 33%. But the actual tracing bit, the phoning-people-up-and-tracing-their-contacts bit, made a very small contribution — reducing R by between 1.7% and 4.6%.
All the rest of the impact, the model said, was down to people self-isolating when they got symptoms, which you don’t need contact tracers for. Harding told the Commons science and technology select committee that the model suggested TTI brought down R by “between 0.3 and 0.6”, but that’s using the 18-33% figure. Using the 1.7-4.6% estimate, it would be more like a reduction of between 0.02 and 0.05.
Of course, the model may be wrong. But it’s the government’s own report, so it’s a bit embarrassing, and probably ought to be more widely known. I should have said that the phoning-people-up contact tracing only accounts for a part of the £22 billion cost, probably a billion or two; the rest was on testing. But a billion here, a billion there, and pretty soon you’re talking about real money.
I think there’s a bigger issue here, too. A statistician I spoke to grumbled that TTI was assessed entirely on “process metrics”: how many people were contacted, things like that. But there were no “outcome metrics”: as in, how many people actually didn’t get ill as a result. That’s hard to measure, but it might have been possible if it were carried out as a randomised control trial (RCT), with contact tracing randomly assigned to some areas and not others.
That would, in fact, be a good general policy. We don’t know the impact of schools on R. We would like to. If we were to randomly select schools and open them for two weeks, we might be able to see differences in R, and as a result do policy better. Policy RCTs are weirdly unpopular, but they would be extremely powerful now.
“They love to decide, announce, defend,” says my grumbling statistician, “instead of experiment, adapt, improve.” The TTI system has, per the government’s own report, been an expensive failure, even if no one is talking about it.
There’s no shame in things not working out, but there’s quite a lot of shame in not admitting when they don’t work out, and not trying to learn from the failures.