We’ve been in this third national lockdown for nearly three weeks now. Covid-19 deaths are at an all-time high: 1,800 were reported on Wednesday. Schools are closed, bars are closed, we can’t see our friends or family.
It is pretty disheartening, therefore, to read that there is “no evidence of a decline” in infections since the lockdown began. The headlines are based on Imperial College’s excellent REACT-1 study, a survey that randomly samples the population and estimates the total prevalence, just as an opinion poll estimates voting intentions.
The headlines are, on the face of it, correct. The most recent REACT-1 data, round 8a, covering the period 6 January to 15 January, found about 1.58% of its respondents to be positive. Round 7b, the one before, found only 0.91%. That’s an increase of well over 50%, and all ages and subgroups showed a rise.
But there’s a wrinkle. Round 7b took place from 25 November to 3 December, well over a month earlier. Normally REACT-1 does its work every couple of weeks, but they seem to have taken Christmas off, understandably enough.
That means that we’re not looking at how the infections have changed since lockdown; we’re looking at how they’ve changed since the end of the last lockdown, which ended on 2 December. Presumably cases were pretty low then. In the meantime, we’ve had an easing of restrictions, a one-day Christmas free-for-all, and the rise of a new and more transmissible variant. The big gap towards the right of the graph below is the area of missing data; it could be that cases were steadily rising, or it could be that they shot up in December and have slowly declined again since lockdown.
REACT-1 takes its samples over several days, and tries to make some estimate of how things are changing from the data collected on those days — the little red line in the rightmost collection of dots is their best guess at the trend. But Kevin McConway, a professor of statistics at the Open University, tells me that the uncertainty intervals around them are huge — the red-shaded area could fit a line going steeply up or a line going steeply down.
It’s certainly safer to treat the whole lot as one data point.
There’s also some rather more reassuring data from the government’s Pillar 1 and 2 testing, which has found the number of cases and the positivity rate declining somewhat; using that, estimates are that the R value is below 1 in England. But we ought to be quite cautious of that too, because as McConway says, they are non-random, so they may be changing for other reasons; the number of tests goes up and down, and recently lateral flow tests have been added to the mix, so it’s hard to compare results across time periods.
In short, “no evidence of a decline [from the REACT-1 study]” is sort of true, but incomplete: “no evidence of anything” might be fairer. It’s consistent with lockdown working brilliantly, lockdown actively making things worse, and anything in between. Evidence from elsewhere is a bit more reassuring but we should be cautious of it.
Incidentally, the breathless headlines aren’t really the media’s fault. The study itself recognised its limitations: “Since there was a gap of over one month from the end of round 7b on 3rd December 2020 to beginning of round 8a on 6th January 2021 we may have missed a peak in prevalence during the intervening period.” But the Imperial press release was headlined “Coronavirus infections are not falling in England, latest REACT findings show”, which I think overstates the reality. Coronavirus infections were higher in early January than in early December, almost definitely. But we can’t, with confidence, say much more than that.