Tom Chivers is a science writer. His second book, How to Read Numbers, is out now.
April 7, 2020
TomChivers
Filed under:
Share:
We need antibody testing. Everyone agrees with that — the question is how it can be done.
There are two main kinds of test for Covid-19: PCR tests which tell us who has the disease right now; and antibody or serological testing, which can tell us who’s had it in the past. The Government wants to build testing programmes that can give a reliable answer from a pinprick test in under 20 minutes.
I’ve been banging on about antibody testing in recent pieces. It’s vital for understanding how far the disease has spread throughout the population; without knowing that, we can’t know how deadly it is, or exactly how alarmed we should be.
But other people have been making another point: that if someone has already had the disease, then — hypothetically, hopefully — they ought to be immune to getting it again. If they’re immune to getting it again, they can’t spread it, so they should be allowed back out into the world, to help get the economy started again.
This is the idea behind mooted “immunity passports” for people who’ve tested positive. Tony Blair seems to be talking about something similar when he writes, in the foreword to a report for his Institute for Global Change, that unless we can “track who has the disease and who has had it”, we will remain locked in a situation “in which everyone has to be isolated so that we don’t miss the small number of cases which need to be isolated”.
Immunity passports sound like a good idea. But I wanted to sound a note of extreme caution. I think antibody testing is absolutely vital — for understanding the spread of the disease, at a population level. But using it to establish individual immunity is very different and potentially much more dangerous.
Let’s assume that people who have had the disease really are immune. Now, imagine that you have a test that is 95% accurate — that is, if you use it on someone who has had the virus, it’s 95% likely to say they have had the virus; if you use it on someone who hasn’t, it’s 95% likely to say they haven’t”. The US FDA has just issued emergency approval for a test with reported numbers pretty much like that (to be precise, 94% and 96% respectively).
So you are given the test, and it comes back positive. How likely is it that you’ve had the disease?
The answer, for the record, is not 95%. The answer is: you don’t know. Unless you know what percentage of people in the population have had it, then you simply don’t have enough information to answer the question.
Let’s look at why. Imagine that 3% of people in this country have had Covid-19, which probably towards the upper end of most reasonable estimates at the moment. Now let’s imagine that you test a million people, completely at random.
Of those million, about 30,000 will actually have had the disease. Your test will correctly identify 28,500 of them. And 970,000 people will not have had the disease. Your test will correctly identify 921,500 of them. So there will be about 1,500 people who are wrongly told that they haven’t had it, and who are forced to remain in unnecessary isolation. That’s bad. But it’s not the real problem.
The real problem is that 48,500 will be told they have had the disease and are now immune. That outnumbers the true positives by almost two to one. If you issue immunity passports on this basis, barely a third of the people you give them to will actually be immune. “There’s nothing peculiar about this statistically,” Kevin McConway, an emeritus professor of statistics at the Open University, told me. “It’s just Bayes’ theorem.” The likelihood of you having had Covid-19, if you’ve had a positive test, depends not just on the accuracy of the test but on the prevalence in the population you’re looking at.
This is a huge problem in diagnostic screening for other diseases, such as cancer and dementia, and can mean that screening tests do more harm than good. It’s why there are follow-up tests. With Covid-19, the concern is that if you just test the same person twice with the same test, there’s a good chance it’ll get it wrong for the same reasons.
This is not to suggest that immunity passports are impossible. For instance, McConway points out, it would be possible to narrow the group you’re testing, so that it contains more people who’ve probably had the disease. It might be, let’s imagine, that people who’ve had the main symptoms of the disease — a persistent dry cough and a high fever — are 60% likely to have had it. So you ask people if they’ve had those symptoms, and you test a million people who have.
In that case, your million tests would send you back 570,000 true positives and only 20,000 false ones; you could be 97% confident that a positive test was correct.
Of course, some tests will eventually be better than 95% accurate — some of those in development claim 100% specificity (that is, they never give false positives). If that’s real, great, although I agree with McConway when he says that no test can be literally perfect.
And this blog post by Sir John Bell at the University of Oxford explains the difficulties of making an antibody test for a new disease: you need something that can be rapidly rolled out to millions of people; you need to validate it on people who you know tested positive on a PCR test, at least 28 days ago, to check for false negatives; and you need to test it on blood taken from people before the outbreak, so you can check for false positives.
The Oxford post says “Sadly, the tests we have looked at to date have not performed well. We see many false negatives (tests where no antibody is detected despite the fact we know it is there) and we also see false positives.” None of those tests have reached the standards that they and the Medicines and Healthcare products Regulatory Agency have agreed.
This does not mean that antibody tests are of no use! Far from it — antibody tests are essential. If you have a well-validated 95% accurate test, you can’t confidently say that any individual is or isn’t immune — but you can use it to get a sense of how many people have the disease. The more people you test, the more accurate an answer you can get; tens or hundreds of thousands of tests would give you quite a precise estimate with quite narrow confidence intervals.
That would, as I keep saying, give you much more information about the disease itself. But it would also give you a better chance of using antibody tests for immunity passport purposes. As we’ve seen, if you don’t know the prevalence in the population, you don’t know what a positive test means for the individual. But if you knew that some high percentage had already had it, and were able to narrow it down further by asking about symptoms, and then could use a highly accurate test, then you could start to get towards the sort of precision we need; where you might be able to let people out into the streets again.
In the end, that’s going to be a horribly cold-blooded calculation. If you let people out when they’re 90% likely to be immune, that means one person in 10 is going to be at risk of getting and spreading the disease. Is that risk a price worth paying for reducing the real costs (economic, social, physical, mental) of isolation? I don’t know and I’m glad I don’t have to work it out. But someone has to. And they’ll have to start by getting a reasonably effective test, and testing hundreds of thousands of people, to see how many of us have had it.
Join the discussion
To join the discussion in the comments, become a paid subscriber.
Join like minded readers that support our journalism, read unlimited articles and enjoy other subscriber-only benefits.
Subscribe