March 10, 2021

I desperately want to have a drink in a pub garden — something which, in theory, I might be able to do quite soon. But when? We’re told that our exit from lockdown will be guided by “data, not dates”. But what does that actually mean?

Because, although it’s supposed to be about “data”, we don’t actually know precisely what the relevant data is. The Government has given four “tests” for easing restrictions, but we don’t know what “passing” those tests actually involves.

According to the “roadmap” announced towards the end of February, lockdown restrictions would be eased in four main stages. Schools would reopen on 8 March (with a few small extra changes, such as meeting up to six people outdoors, and the reopening of outdoor sports, on 29 March, as children broke up for Easter).

On or after 12 April, non-essential shops would reopen and pubs and restaurants would be able to serve people outdoors. On or after 17 May, you’ll be able to meet up to six people or two households indoors, and get a drink or a meal inside a pub or restaurant. And on or after 21 June, all restrictions will end.

It’s the “on or after” which is key. Each new easing will come only if we meet certain criteria and if we don’t meet them, then the dates will be pushed back. So it’s interesting to note that whether we pass the criteria — as set out in the four tests — is entirely subjective: there is no clear quantitative answer to whether or not we achieve the goal.

 

 

These four tests need to be passed at each stage of our liberation and you’ll notice a distinct lack of numbers:

1) the vaccine deployment programme continues successfully; 2) evidence shows vaccines are sufficiently effective in reducing hospitalisations and deaths in those vaccinated; 3) infection rates do not risk a surge in hospitalisations which would put unsustainable pressure on the NHS; 4) our assessment of the risks is not fundamentally changed by new Variants of Concern.

So what, then, counts as “successfully”? What counts as “sufficiently effective”? What is “unsustainable pressure”? What counts as “fundamentally changed”? (Also, it’s noteworthy that there’s no mention of R at all.)

Given my desperation for a pint, I need to understand what it all means. So I asked Dr Duncan Robertson, a policy and strategy researcher at Loughborough University, to have a go at explaining  And I have had a go at trying to work out how likely we are to pass the tests. 1

 

First test: vaccine deployment

This is going well. According to NHS England statistics, 95% of 70- to 74-year-olds, nearly 101% of 75- to 79-year-olds, and nearly 95% of the over-80s had been given a first dose of the vaccine by 28 February. (Yes, 101%. That’s as a proportion of the ONS population estimates, suggesting that the ONS population estimate may have been a little low.) The Scottish data is similar and much more conveniently presented.

There are some wrinkles. For one thing, while care home residents have been very effectively covered – nearly 94% – care home staff seem less so, at below 73%. Since care home staff were on the highest priority list, that seems suboptimal.

Robertson points out, though, that things may not be as bad as they seem, or at least they are bad in a different way. We have good data on how many, say, 75- to 79-year-olds there are in the country, through NHS databases. (Not perfect, obviously, given that we seem to have vaccinated 101% of them. But good.)

We do not, however, have a good central database for care home staff. Robertson suspects that a lot of them will be registered at two different homes. “It would not surprise me if there were some double counting of care workers,” he says. “I don’t think we have very clear numbers of them, so getting that percentage is fairly difficult.” He suspects the true figure will be higher; certainly in Scotland the figure is almost 85%.

It is a real problem that we don’t have this data, and workers moving between different care homes may, he thinks, have been a driver of some of the many outbreaks in those homes last year: “It would be nice if there was more reliable data,” he says. (It’s another reminder that, as Robert Colvile says, a lot of modern government is database management, and if you don’t have a good database, you’ll struggle to implement policies.) But it may not be a disaster in terms of the rollout.

A more interesting problem is whether the rollout is evenly distributed. We can see discrepancies, to some degree, in the NHS England data at a regional level: London is lagging behind all the other regions at every age category, although that could be because the ONS population estimates are wrong (London population is very transient, and there may have been, for instance, a significant net outflow of foreign nationals in the last few years, with Brexit and Covid). But while there is much more granular data – local authority level and below – it’s not broken down by risk categories, so we can’t see how many, for instance, over-70s in Blackburn with Darwen have been vaccinated. If we reach an average of 90% coverage, but some local communities are badly covered, that could leave pockets for the virus to keep transmitting.

So how do we “pass” this test sufficiently for us to carry on coming out of restrictions? Away from the roadmap, the UK has set itself milestones – vaccinating all the over-70s, care home residents and extremely vulnerable by 15 February; all the over-50s and other vulnerable people by 15 April, all adults by 31 July. We’ve passed the first of those milestones, and even though we haven’t quite managed “all”, getting well past 90% for most of the categories looks pretty good, and we seem to be well on course to meet the rest.

If I crudely stick the figures on a graph and extend the line, I expect we’d have given first doses to the 32 million people in all the priority cohorts around the end of this month, and the whole UK adult population of 53 million or so by late May. Obviously that doesn’t take into account any of the nuances above, and it’s unlikely we’ll actually keep on at the same rate once second doses start in earnest. But still, I’d be very confident that this test will be met — that, by any reasonable definition, the vaccine rollout will be “continuing successfully” — for Stage 2 in April, and almost as confident that the May and June reopenings will be met too.

Crude probability estimates of meeting this criterion at each stage: Stage 2, 90%; Stage 3, 85%; Stage 4, 80%.

 

Second test: vaccine efficacy

Early data from nearly five million Scottish people and from a smaller group of healthcare workers across the UK found that they reduce infections, hospitalisations and deaths by large percentages; over 90% in the case of hospitalisations and deaths. That’s true of both the Pfizer and the Oxford-AstraZeneca vaccines. Hospitalisations are falling faster in over-75s than other groups, suggesting a vaccine effect. It’s not unreasonable to assume that they’ve saved about 3,000 lives already.

I think it’s pretty clear, then, that the vaccines are “sufficiently effective”.

The only thing that is likely to reduce this effectiveness would be some vaccine-escape mutant. But assuming that’s not the case (more on that in a bit) then we can probably be confident that the vaccines will continue to be seen to work, and that they won’t prevent us from opening up in April, May and June.

Crude probability estimates of meeting this criterion at each stage, assuming that we don’t get a significant new variant: Stage 2, 95%; Stage 3, 95%; Stage 4, 95%.

 

Third test: pressure on the NHS

Now it starts to get more complicated. Can you define “unsustainable pressure on the NHS”? Matt Hancock, the health secretary, said in November that the NHS was never overwhelmed during the first wave. And in terms of never quite running out of ICU beds, that may be true. But certainly in terms of its ability to carry out elective procedures, manage chronic care, generally look after people who didn’t have Covid, it was severely struggling. Is that “unsustainable pressure”?

“There’s a whole backlog of patients who need to go into hospital for operations and elective care,” says Robertson. “If you keep all of them out of hospital, you reduce the demands on the NHS, but the pressure doesn’t go away, it’s just off the balance sheets.” Somewhere between “higher-than-normal demand for a normal July day” and “the peak of surge capacity at the height of the Covid waves”, there is an invisible line marked “unsustainable pressure”, but exactly where it is we don’t know.

We won’t find out, either, at least for a few weeks. Schools have now been open for two days as I write this. If that leads to new cases, we won’t see any of them for about a week, and there won’t be enough to reliably say that there’s an effect for at least three. Hospitalisations are around a week behind that. If the first stage of the roadmap causes significant pressure on the NHS, we won’t know until early April at the very earliest. That’s why the roadmap includes these long gaps between stages: so we can see the effect.

Chris Whitty expects to see a surge in cases, and then hospitalisations and deaths, as we open up — much reduced, though, because the most vulnerable people will be protected by the vaccine, so the ratio of cases to hospitalisations (and deaths) will be far higher. But it will not be zero. Whether the surge is enough to cause “unsustainable pressure” will be a question both of how big it is and what is meant by that phrase.

At the moment, there are about 600 admissions a day for Covid in England, which is roughly where we were in October. For context, we got up to almost 4,000 a day in January, but were way down at around 60 a day in August last year. And at the moment there are around 1,500 people in ICUs needing mechanical ventilation, down from about 3,000 in January, but that number drops slowly because people stay in ICU a long time.

I’m much less confident about this than I was about the vaccine rollout. I am less confident about what is defined as “unsustainable pressure”, but, also, I worry that by 12 April, with the schools having been up and running for three weeks and other things opening up, we could be starting to see the beginning of a rise in hospital admissions. The vaccines should help keep the numbers down, so I still think it’s pretty likely that we’ll go ahead with stage 2, but I’m a bit less confident about stages 3 and 4, in May and June.

Crude probability estimates of meeting this criterion at each stage: Stage 2, 80%; Stage 3, 70%; Stage 4, 70%.

 

Fourth test: variants of concern

The B.1.1.7 variant arose, or at least was detected, in the UK in September. It turned out to be much more transmissible than Covid Classic and, lately, death data suggests that it’s between 33% and 100% more deadly than the vanilla version, as well. Does that “fundamentally change” the assessment of risks?

Well, probably not, since the variant was known about before the roadmap was released. But back in January, there was serious concern that the inherent transmissibility of the new variant would make it impossible to control using lockdowns alone: that its R would remain above 1 and the virus would spread regardless. It hasn’t turned out that way, but I think that would have counted as “fundamentally changing” the risk.

Now, there’s real ambiguity over how well the various vaccines perform against the more recent variants, the Brazilian, South African and other mutant strains with the E484K mutation. This mutation changes the shape of the spike protein and makes the vaccines less effective — vaccine escape — but there seems to be evidence that they still work pretty well.

If a real vaccine-escape variant were to come along, says Robertson, that’s a “worst-case scenario”. Then all those vaccinated people in the at-risk groups are vulnerable again and we’re perhaps not back where we started, but certainly nowhere good. Another possibility would be if the virus mutated to be more dangerous for young, healthy people.

So far that hasn’t happened, and so far the more worrying variants that do exist seem to be relatively well-controlled. If, as we open up, they start spreading around more, or more mutations start happening, that will be worrying. This, incidentally, is where I get a bit worried about the idea that the easing of lockdown is “irreversible”: I hope it’s pretty bloody reversible if some serious vaccine escape variants start spreading around.

Whether we pass this test is a matter of pure luck, although the more cases there are in the community, the more chances there are for the virus to mutate. There are reasons to think that it might be quite hard for the virus to become yet more virulent or to properly escape the vaccine, but they’re very theoretical. The UK does have excellent genetic sequencing, though, so it should get very early warning of any variants that do arise. What exactly counts as “fundamentally changing” the situation is, again, unclear.

Crude probability estimates of meeting this criterion at each stage: Stage 2, 85%; Stage 3, 80%; Stage 4, 75%.

 

When will I be having that pint, then?

I had imagined writing this piece with lots of graphs, showing where we are on a route towards opening up – these numbers counting down, these ones going up, a neat projection for where we’ll be on 12 April and 17 May if current trends continue. But unlike previous lockdowns, there isn’t a clear quantification of what it means to meet the different tests at any of the three stages to come.

That may be sensible. The last lockdown had apparently objective measures – local areas would have restrictions imposed when they reached a certain threshold of cases per 100,000. But those were quietly changed, then ignored. Also, if you start imposing lockdowns on any simple measure, you become subject to Goodhart’s Law pretty quickly.

But on the other hand, if it is going to be a judgment call, then whose judgment it is becomes extremely important. The roadmap document only says “the Government” will examine the data, not SAGE or the Joint Biosecurity Centre. “The critical thing is assessing the data after four or five weeks,” says Robertson, and the point is assessing the data, not the political pressure.

I think the vaccine rollout and vaccine efficacy tests should be met with ease. I’m much less clear about what “unsustainable pressure” on the NHS looks like, and whether we’re currently experiencing it or not; and given that we’re yet to see what the effects of opening schools will be on transmission, that may end up slowing things down. And as for variants of concern, it’s a crapshoot – literally random, although the UK’s sequencing capability is helpful. But I think the odds are reasonably good.

One complicating factor: there are, inevitably, calls for the Government to open up faster. But Whitty and Patrick Vallance have told the Science and Technology Committee that this would be unwise; I suspect people wouldn’t like it, given that lockdown measures have been pretty popular so far. Either way, it seems absolutely crazy to speed things up, when the costs of coming out of lockdown too early could be enormous, and the costs of waiting a few more weeks as the vaccines get rolled out are large but limited. If, however, it’s a decision made by government, rather than scientific advisers, then the political pressure might tell.

So what about my beer? Of the four tests, two — the vaccine-related ones — are clearly being met. With the third, hospital pressure, we’ll have to wait and see, but the success of the vaccine rollout should give us plenty of room to play with. And, so far, we’ve been reasonably lucky with new variants. So any given test will probably be met at any given stage.

All taken together, though, it’s not quite so obvious. If you take all my probability estimates above and simply multiply them out, then you get about a 60% chance that stage 2 will happen on April 12, about a 45% chance that stage 3 will happen on May 17, and about a 40% chance that stage 4 will happen on 21 June. But I’m going to bump them all up a bit, because I think if it’s borderline they could be tipped upwards by political pressure, so I’ll say 70%, 55%, and 50%, respectively. So, if I were a betting man, I’d wager at least one round on the fact that we’ll be drinking our first pint this year in a pub garden on 12 April. Even if it’s raining.

 

FOOTNOTES
  1.  Almost a year ago, I said that I was going to start making more falsifiable forecasts more regularly. I haven’t really honoured that, and (I should admit) most of the forecasts I have made have been wrong. But this seemed a good opportunity to get back in the habit.