The extent of the failure of British Covid scientists in the pre-Christmas Omicron panic is only now becoming clear. As brutally documented over at the Spectator, the models produced by both the London School of Hygiene and Tropical Medicine and Warwick University were wildly pessimistic compared to what actually happened. It was a failure that very nearly bounced the Government into further lockdown-style measures before Christmas.
What is additionally troubling is that other countries’ modelling groups seem to have been much more successful. I spoke to Dr Camilla Holten-Møller, the chair of the Expert Group for Mathematical Modelling at Denmark’s public health agency ‘Statens Serum Institut’. She led the team that produced the models in December that informed Danish policy, and her group’s updated advice in January led to today’s cancellation of all Danish Covid restrictions (even as case numbers continue to climb to all-time highs).
Take a look at their model from December 17th, showing how Omicron would rapidly take over Delta, and forecasting the upper and lower expectations of numbers of new hospitalisations each day. The dots are broadly within the cone — in other words reality roughly matched their predictions:
Compare that to Warwick University’s forecast of new daily hospitalisations, if we continued with ‘Plan B’ restrictions and didn’t impose any further lockdowns. The red line is what actually happened:
I did some back of the envelope calculations, and if we had had the Danish scientists in charge here in England, it looks like the estimates would have been a lot more accurate. Of course you can’t simply transpose these figures (and Denmark has had a higher Omicron surge than the UK) but it is a useful high-level exercise. The population of Denmark is 5.8m, while the population of England is 56m — almost exactly ten times the size. If you multiply the Danish forecasts by ten, you get a lot closer to what happened in England than the numbers put forward by the SAGE committee to our Government:
Of course, the Government (only thanks to cabinet and backbench pressure, no thanks to Chris Whitty or Patrick Vallance) rejected the advice of SAGE to introduce more restrictions, which turned out to be the right call.
But what is the explanation for this huge difference with the Danish modellers?
One idea might be that the Danes paid better attention to the real-world data coming out of South Africa at the time that Omicron was intrinsically much milder than Delta. On 17th December Neil Ferguson’s group at Imperial produced a meta-study that concluded that there was “no evidence of Omicron having lower severity than Delta.” Even at the time this felt like a bizarre finding, and evidence now seems to suggest something closer to 10%-20% of the severity of Delta. So that was overly cautious, we now can say for sure, bad information.
However, both the UK modelling groups and the Danish group produced a range of scenarios with different severities, and both used the 50% mid-point as most likely. So that doesn’t explain the difference.
Dr Holton-Møller suggested two other variables that might explain it.
The first was the attention the Danish groups paid to behavioural changes that weren’t mandated. In other words, from their observations over the course of the pandemic, people moderate their behaviour at times of high case numbers even if they are not forced to by the Government.
The British scientists, even a year and a half after the pandemic began, seem unwilling to consider this crucial factor. The discussion paper for the Warwick model admits that, while unmandated behaviour change is “highly likely… such dynamic changes are beyond the current capacity of this model.”
The other variable that Dr Holten Møller suggests might account for the difference is what she calls “susceptible depletion” leading to “peak shaving”. This is also crucial: failure to properly consider this means that early exponential growth rates are incorrectly forecast to continue much longer than they do in reality:
Dr Holten-Møller was careful not to criticise academic colleagues from British universities, but offered her input into a future enquiry into what went right and wrong with this process. “The modelling community needs to get together and look into each other’s successes and failures,” she said.
She did observe that the Danish modellers are “detached from the political situation” — which might explain why all the political parties support Denmark’s bonfire of Covid regulations, a scenario that would be unrecognisable in the UK or the US.
If there is to be a shake-up of British scientific advice, and a search for superior talent in the near-future, Camilla Holten-Møller might be one name to keen in the frame…