I want to talk about the models, and what they tell us, because the outputs of these models drive the government’s response — and thousands of lives could turn on them. It’s important, therefore, that we understand them, and why the numbers they give us are so different. These figures led Peter Hitchens, the Mail on Sunday columnist, to complain that the number of deaths has jumped around from 500,000 to 20,000, to 5,000. I can see why people are confused, if they just think “the models” are taking the same numbers and spitting out these weirdly different results.

But first, I want to talk about something much simpler. It’s the question that many of us, I’d say, most want to know, when we’re anxiously thinking about Covid-19 and ourselves and our loved ones. That is: if someone gets the disease, how likely are they to die?

The splendid Our World in Data (OWID) — who, full disclosure, I’ve been doing some editing for during this crisis — have been working on answering that question. So has Oxford’s equally splendid Centre for Evidence-Based Medicine (CEBM).

It is probably the most basic question you can ask about a disease, and yet it’s bloody hard to answer. It changes between places and over time, and between different groups; it depends on a variety of hard-to-measure factors, and the answer you get itself directly affects our understanding of several other numbers, all of which are vital to any model of the disease.

So. You may have heard a term being used: the “case fatality rate”, or CFR. That is the number of deaths divided by the number of confirmed cases. When journalists talk about the “death rate”, that’s often what they are referring to. If a country has 10,000 confirmed cases and 100 deaths, then the CFR in that country is (100/10,000), or 1%.

That is not what we are looking for, and it is probably not even very close to what we are looking for.

Instead what we want is the “infection fatality rate”, or IFR. That is the number of deaths divided by the number of people who actually have the disease. The number of people who have tested positive for the disease is probably only a fraction of the total number who had it, because only a fraction of the population has actually been tested. 

Obviously, the IFR is much harder to determine accurately. The only people getting tested will be the people who are most ill, so your IFR is probably much lower than your CFR, because your denominator — the number you’re dividing by — is probably much bigger. 

So if your country has tested absolutely everyone and found all cases of the disease, then your IFR is the same as your CFR, or 1%. But if it has only found 10% of the people with the disease, then your 10,000 confirmed cases are just the tip of a 100,000-person iceberg. With those 100 deaths, your IFR would be (100/100,000) or 0.1%.

Sadly, all we know is the CFR, and it changes hugely from country to country. Professor Jason Oke, a statistician at Oxford University and one of the people behind the CEBM analysis, points out that the CFR in Italy is many times higher than that in Germany; 11% of confirmed cases have died in the former, compared to 0.79% in the latter. “That can’t be down to healthcare differences,” he says: “Italy isn’t a third-world country. And it can’t be demographics — Italy has an old population [and older people are at greater risk], but Germany isn’t far behind it.” 

The likely explanation is that the difference is down to testing. Italy has largely tested people with symptoms, in hospitals; Germany has tested many thousands of people who have no symptoms. That probably helps prevent some deaths, but the most direct impact will be that it hugely increases the denominator; again, if you’re dividing your 100 deaths by 100,000 instead of 10,000, your death rate will be much smaller. Germany’s will be closer to (but not the same as) the IFR, the number we really want.

So, a country’s CFR will vary depending on how many tests it’s done, because that changes the denominator. But at least the numerator — the number being divided — is probably pretty straightforward, right? A death is a death.

Sadly, that’s not the case either. Dr Hannah Ritchie, one of the data scientists at OWID, points out that the death statistics are complex too, for two reasons. One is prosaic: a lot of people who have the disease and will die of it have not yet died. “The period from onset to death is about a month,” she says. So your simple “divide the numerator by the denominator” rule — the number of deaths by the number of patients — doesn’t work, because your number of deaths is a product of how many people had the disease a month ago, not how many people have it now.

More from this author

Would you take a coronavirus risk?

By Tom Chivers

That’s bad enough. But there’s a more profound problem, which is that deaths themselves can be recorded very differently in different places. Professor Sir David Spiegelhalter, a statistician at the University of Cambridge, says that the UK simply counts people who have tested positive and then died. But in some other countries, people are recorded as having died of Covid-19 if they had the symptoms, even if they weren’t tested (“suspected” as opposed to “confirmed”); in others, people outside hospitals are not tested and so are not recorded.

“Even the number of deaths is not a perfect statistic at all,” Spiegelhalter says. El Pais did an interesting look at some of the international differences here; Britain’s Office for National Statistics explains why its numbers look different from the official Government ones here.

To some extent it doesn’t matter, says Spiegelhalter: as long as each individual country maintains the same regime, ”the number of deaths is still a good monitor for the shape of the epidemic”. 

But it’s not clear that they are fixed; countries may have good reasons to change the way they collect data as circumstances change, but it apparently happens often enough that the World Health Organisation feels that they have to ask countries to notify them when they do it. Famously, China did so earlier in the epidemic, but others do too: in complying with the WHO’s request, Australia has noted that it has changed its definition of a Covid-19 “case” (and therefore a Covid-19 “death”) at least 12 times since 23 January. 

So, in essence, to work out the IFR — the number we want — we need to know two other factors: the number of people infected with Covid-19, and the number of people who died of it; the denominator, and the numerator. And, sadly, both numbers are uncertain.

How much does any of this matter? Well: now, I’m going to try and build my very own, rather stupid model. I’m not going to try to predict the future; I’m just going to try to use some very simple numbers to “predict” the number of cases there are in the UK right now.

First, the number of reported Covid-19 deaths in the UK is 1,408. Second, the lag between infection and death is about three or four weeks; let’s say three. Third, the number of (confirmed) cases in the UK has been doubling about every three to five days; let’s say five (and assume it’s staying constant; forget social isolation for now).

With those numbers, we can plug in our guess at the IFR (the infection fatality rate, remember: the real number we want to know, the “if I get it, how likely am I to die” number) and use it to work out roughly how many cases we’d expect now.

So what number should we use as our IFR? I’m going to use three: one from Imperial College London’s MRC team, the one behind the famous model; and two from CEBM. Imperial’s latest work assumes an IFR of 1%; CEBM estimates it to be between 0.1% and 0.26%.

If we take the Imperial 1%, then that means that we can multiply the 1,408 deaths now by 100 to get the number of people who’d had the disease three weeks ago, because we think about one in 100 of them died. So about 140,000 people.

Then we can take our doubling time — we said five days — to get how many cases we’d see now. In three weeks, 21 days, you’d see four doublings; two to the power four is 16. So 16 times 140,000, which is 2,240,000. We could imagine that we’ve probably got about two million cases now.

But let’s use the CEBM numbers. First the highest one, 0.26%. If we take that, we can multiply the 1,408 deaths by 400, instead of 100, to give us the number of infections three weeks ago. That is 560,000. Then we can multiply that by 16 to give us an estimate of how many there are now: about 9,000,000.

And how about if we use their lowest estimate, 0.1%? Same routine: 1,408 multiplied by 1,000, multiplied by 16: more than 20,000,000.

So by changing a single number, the IFR, to one of three plausible values, in a very simple model, we get outputs that range from “3% of people have already had it” to “30% of people have already had it”. And that’s before we start messing around with the other assumptions; is doubling time three days, not five? Is infection to death four weeks, not three? Is “1,408 deaths” even correct?

More from this author

Self-isolation is feeding my Warhammer addiction

By Tom Chivers

I want to reiterate: this is a very simple, stupid model, put together by a journalist, not an epidemiologist. The actual models will be far more complex, and will take into account other things — the number of cases in hospital and so on — to try to ground them in objective fact. Don’t mistake this for some plausible estimate of infection numbers. And there are loads of other things to worry about: people suffering long-term health consequences, even if they live; people dying of other things because the healthcare system is overwhelmed.

But the problem that I’m illustrating is real. Small, plausible adjustments to your inputs make your model spit out very different things. The assumptions you make are vital. We haven’t even started to think about other crucial things — for instance, government interventions, and how effective they are.

“The different scenarios — isolation, closing schools, quarantines — they come with massive assumptions about how adherent people are,” says Ritchie. “You get massively varying outputs, depending on what you put in.” Plus, of course, it’s all circular: your model influences how you respond; your response changes what numbers get put back into the model.

What you need is better numbers, to plug into the models. And that’s what people are trying to get. The CEBM paper uses, among other things, numbers from Iceland, which — being tiny — managed to test a huge proportion of its population, nearly 3%. It found 963 cases and just two deaths; an CFR of 0.2%, from which the CEBM extrapolates an IFR of 0.05%. But that’s likely an underestimate, because quarantining has protected the elderly, the most at-risk group. Others have done something similar with passengers on the cruise ship Diamond Princess, finding a CFR of around 1% — likely an overestimate, since the passengers tended to be older.

Another way has been to screen small groups who are in quarantine, such as the Diamond Princess passengers or passengers on aircraft, to see how many people 1) test positive and 2) show symptoms. If you know how many people have the disease but are asymptomatic, then you can extrapolate to the wider population — if you’re testing all the people who are symptomatic, and you know that 50% of infected people don’t show symptoms, and you find 20,000 cases, you can estimate that the real number is more like 40,000.

But there’s a problem here, too, which is that “asymptomatic” is not a simple thing, according to George Davey Smith, an epidemiologist at the University of Bristol. If you’re sitting in quarantine and you cough, you might be recorded as symptomatic; but out in the real world, you’re not going to take yourself to hospital for a gentle cough, so you still won’t get tested.

Instead of there being a neat “symptomatic/asymptomatic” division, you have a third group: people with some symptoms but who think it’s just a cold in the chest. (As I write, I’m coughing a little; but I don’t think that’s Covid-19, I think it’s just because I went for a run this morning and the cold irritated my lungs. But I’d probably be recorded as “symptomatic” if I were in quarantine and being screened.)

And yet another way is to look at how many people die every year, and how many people are dying now, and seeing whether more people are dying than usual. That’s how we attribute deaths to flu each year, says Spiegelhalter; the European Monitoring of Excess Mortality group (EuroMOMO) uses this data to say that in an average year in the UK, 17,000 deaths are “associated” with influenza. But so far there is no excess death at all, except in Italy: the EuroMOMO charts are all around the seasonal average. That will likely change, but we forget that in a population of millions, you’d expect thousands of deaths every day anyway: even the Covid-19 pandemic is still being lost in the noise.

Suggested reading

Don’t trust the psychologists on coronavirus

By Stuart Ritchie

In the end, we need testing. And not just the sort of testing we have now — PCR testing, which shows who has the virus right now; we need serological testing, which shows who has had it in the past. That will come along relatively soon, and hopefully can be quite quickly used to test randomly selected people, like an opinion poll sampling a population; then we can see how many people have had it, and from there work out the IFR. But for the moment we don’t have that. 

I wanted to write this to give an impression of how appallingly difficult the modeller’s job is. I write, sometimes, pieces about statistics — I suggested that claims about the loneliness epidemic, teen suicides, and the media’s influence on Brexit were overstated, for instance. They involved very basic maths, done for very low stakes: if I messed up, if I failed to carry a 2 or whatever, I would look very stupid and would be very embarrassed, but no one would die. 

Whereas, if the Imperial College modellers get it wrong, with their far more complex maths and their far more uncertain inputs, they could sway government policy enormously. Whether we lock down society or carry on as normal depends heavily on the outputs of models like these. And it’s not that there’s an easy “better safe than sorry” option; if we crash the economy, it will (eventually) cause real health problems.

A February 2020 review found that 10 years of austerity may have caused the growth in life expectancy to stall, especially among the poorest; I’m sceptical of the “130,000 deaths caused by austerity” stat, but it’s pretty clear that it had a real negative impact. The post-Covid-19 world will almost certainly involve huge austerity to pay for the vast costs incurred during the virus.

Get it wrong one way, and thousands of people die unnecessarily from the virus; get it wrong the other, and you crash our public services and kill people that way. (I’ve only seen one attempt to model the health outcomes of that crash, and I have no way of judging it; for what it’s worth, though, it does say they will be extremely terrible; on the other hand, recessions don’t seem to shorten life expectancy, so who knows.)

So I’m very sympathetic to the modellers. But there are things which would help, and which they can do, but haven’t, so far at least. Ferguson’s team has not released the code his model is based on; he says he is working with software developers to do so, but proponents of open science, like Davey Smith and his colleague Marcus Munafò, say this isn’t happening fast enough.

“These models are so sensitive to their assumptions,” says Munafò. “And they’re black boxes.” The code is 13 years old; it’s vital that other scientists are allowed to look at it, check it for mistakes and stress-test its assumptions. There are other, open-source models available, but the Imperial one is still kept under wraps, and it shouldn’t be.

Because a lot rides on the outputs. If millions have already been infected and the disease is less deadly than we think, then our response should be very different to if millions are still to be infected and tens or hundreds of thousands more will die. 

Suggested reading

Nineties Russia prepared me for coronageddon

By Daniel Kalder

The latest from Ferguson’s team suggests that between 1% and 5% of the UK’s population has already been infected. Oke of CEBM thinks that that could be an underestimate — he thinks that the disease was circulating in China for a month or so before it was announced. “There were early reports of doctors saying they were seeing unusual respiratory symptoms, which were suppressed,” he says. That could have brought it all here much earlier.

When I mentioned the Oxford “study” — in fact a model showing what plausible inputs could produce what we’ve seen, one of which was a very low IFR and huge number of people already infected — he didn’t endorse it, but said “I think a lot of people have underestimated how far this has already spread, and how early.” Davey Smith also thinks that those Imperial figures could well be an underestimate. I have no idea if they’re right or wrong, but whether they are or not matters a great deal.

For the record, to take us back to the beginning, Peter Hitchens had flatly misunderstood what was going on. The 500,000 number was a worst-case scenario if we did nothing; the 20,000 was Ferguson’s team’s estimate of what we’d see now that social-distancing measures and so on are in place.

The 5,000 wasn’t from Ferguson’s team at all but from an electrical engineering group also at Imperial who just eyeballed the death curve on the China graph, fitted the UK numbers so far onto that, and extrapolated from there. “They explicitly say they’re not doing any epidemiological modelling at all,” says Spiegelhalter, “and they retracted it two days later on Twitter,” after it became obvious that it was wrong.

But we shouldn’t be complacent and assume that, just because people are misunderstanding what the modellers are doing, the models must be correct. Everything that comes out of them is the product of what goes in, and all of that is going to be wrong, to some degree. “The key point is that the numbers we have now are not correct,” says Ritchie. 

If you look back to previous outbreaks, such as the 2009 swine flu epidemic, the numbers people were using while it was still going on were wildly different from the ones scientists settled on afterward: early estimates in 2009 were between 0.1% and 5.1%; the eventual WHO estimate was just 0.02%, similar to seasonal flu. In a fast-moving situation, it is easy to make large mistakes. (In either direction! I am not suggesting that the Covid-19 situation will necessarily be similarly overstated.)

These models are the best information we have at the moment. But they are hugely uncertain, and likely to be wrong. All we can do is try to get better information for them, and make the best decisions we can under conditions of appalling uncertainty — and be forgiving of the modellers who are desperately trying to make life-changing, history-changing decisions at high speed and with bad data. 

Like Mildred, the stubborn old-girl character played by Frances McDormand in Three Billboards Outside Ebbing, Missouri seeking restitution for her murdered daughter, I sometimes have problems with compliance. And I do think it’s a generation thing. Mine is the generation which came to young adulthood in the 1960s when the contrarian reflex was to defy authority, rather than succumb to it. Authority can often turn authoritarian. Give me the information, and I’ll make up my own mind about my conduct, thanks very much.

Contemporaries — some a little younger, some slightly more senior — seem inclined to react to all the rules and regs with a more laid-back manner than younger folk. Madeleine Grant says she cannot get her 79-year-old father to take the warnings seriously. Peter Hitchens in the Mail on Sunday claims that the alarm about the spread of the virus is over-stated, and he doesn’t need to be told when to wash his hands, or whether there are germs on doorknobs. Here in coastal Kent, my friends and peers display a blithely stoical attitude. “If my number’s on the bullet, it’s on the bullet,” laughs Josephine, seventyish. “I feel sure I’m going to get it,” chuckles Colin, born, like me in 1944, and just off to join pals for a night-time pub session. “Face the music and carry on!”

Suggested reading

Will coronavirus change the world?

By Ed West

Many of the Irish — who are implementing much more draconian measures, closing down the pubs by diktat — have been complaining on Twitter that the Brits allowed Cheltenham to go ahead. Disgraceful! I asked a horsey friend (78) if he thought it should have been cancelled – 60,000 people huddled together for the legendary festival. “No!” he chortled. “I had three winners!”

No, I know I shouldn’t take a cavalier attitude to the coronavirus — and I promise my family that I won’t. I am diligently obeying all the rules. I am practicing “social distancing”. I am washing my hands on the hour every hour. I am monitoring my health for coughs or fevers — as it happens I do have a slight chest infection, picked up from an Easyjet flight from Belfast (airplanes are full of germs). Altogether, I am acting responsibly — to protect others, as we are told to do, as much as myself. And yet I am still not really aligned with some of the general attitudes around the Covid-19 alarm.

I know my dear family are worried since the “elderly” (please call us “old”, rather than use twee euphemisms) are more vulnerable; and because those of us prone to respiratory problems are particularly in the firing line. But whatever became of the virtues of fortitude and courage? In my convent schooldays, we were taught that in life it is really important to be brave. We were regaled with stories of heroic nuns who willingly embraced lepers — St Catherine of Siena drank a cupful of leper’s pus just to show that she didn’t entertain fear. If we were to be tortured to death for our faith, we should, like Joan of Arc, accept it with a soldier’s valour.

But safety, caution and aversion of all risk are now prized more greatly than these stoical values.

It need hardly be pointed out too indelicately that the graphs and statistics clearly indicate — in Italy, for example — that the greatest proportion of deaths occur among those over 80. The next greatest number are in my own demographic — in their seventies. I certainly think that oldies have just as much entitlement to medical care as anyone else, and speculation about leaving them to their fate is harsh. But facing death in your seventies and eighties is also accepting that nature eventually takes its course. No, I don’t want to pop my clogs quite yet, but I do want to have the fortitude to recognise that at our age, we are moving towards the departure lounge.

Suggested reading

Farewell free trade, and good riddance

By David Goodhart

I actually don’t mind being old: I can put up with the aches and pains: I don’t share the dislike that some women have of being “invisible”: and I thoroughly concur with the Greek philosopher who said that being freed from sexual desire is like being unchained from a maniac. The one thing I do dislike — even dread — is being patronised. I hate people asking me “Are you all right, dearie?” Or cautioning: “Watch the step, my lovely.” To the prudent counsel, “take care”, my instinctive reaction is to reach for Nietzsche’s “Live dangerously!”

And I know that all sorts of kind and good people are going to be asking me “Can you manage, dear?”, “Are you very lonely?”. I’m just going to have to be a well-behaved old gal, comply, and be grateful.

My philosophy of old age is to do as much as you can for as long as you can, and I fear that the Covid-19 house arrest will put a halt to my gallop, as the Irish say. I already look wistfully at our hi-speed trains, departing on the hour for St Pancras International. Will I ever see the inside of the London Library again? From the beach at Deal, we can glimpse the white cliffs of Calais and the twinkling lights of France: will I ever again board that ferry across the Channel? Will I never again walk by Galway Bay?

Food and loo rolls – who cares? It’s being deprived of the stimulus of life itself that’s so dismaying.