Now, in Rationality: What It Is, Why It Seems Scarce, Why It Matters, Pinker synthesises both aspects of his oeuvre. On the one hand, Rationality briskly introduces you to essential concepts like Bayesian reasoning and technical terms like modus ponens. But the technical, learned aspects of the book are accompanied by a polemical drive. Rationality matters — every day, everywhere. Especially at present, in a world that many think has gone mad, whipsawed irrationally by social media mobs and online conspiracy theories. Pinker argues that methods of rationality represent not an esoteric corner of cognitive science or philosophy, but an essential set of tools for individual human flourishing, maintaining our sanity and perpetuating civilisation as a collective whole. To reason and think clearly is to be modern, and even requisite in order to be moral.

In the wake of the financial crisis of 2008, the rational actor and “homo economicus” seemed to have failed us. Nature abhors a vacuum, so there was a vogue for the field of behavioural economics, a discipline that operates in the gray zone of economic frictions and psychological irrationalities powered by faulty intuition. A crop of authors harvested these ideas: Dan Ariely, Rom Brafman, Ori Brafman, Cass Sunstein, and Richard Thaler among them. More broadly, Daniel Kahneman and Amos Tversky’s heuristics and biases research program, to which all these authors owe a debt, entered the mainstream — Kahneman also contributed a bestseller, Thinking, Fast and Slow in 2011. Barack Obama’s administration tried to ‘nudge’ America through policy. David Cameron’s government, and the British civil service, embraced nudging too.

Rationality is part of a tacit backlash against these authors, their books, and the atmosphere they generated in public policy circles in the 2010s.

Today, Dan Ariely faces accusations of fraud at worst and scientific malpractice at best. More broadly, behavioural economics has been hit by the replication crisis. Humans respond to incentives, and for scientists, sexy and vivid results are rewarded, while negative findings can stall a career. The scientific culture of the aughts was defined at its peril by “publish or perish,” as well as sloppy experimental practices, a mechanically unthinking application of statistics, and cognitive biases in the very fields dedicated to understanding how rationality fails us. Kahneman himself began having doubts in 2012, justifiably, about some of the results reported in his 2011 book.

Pinker’s good news in Rationality is that with deliberate effort, humans can think clearly and move beyond their biases. Where much of cognitive psychology focuses on illusions and mental misfires, Rationality instead argues that our faculties are up to the task, and that this is all a natural consequence of our evolution. Evolutionary psychology leans heavily on the idea that humans are only equipped with “stone-age minds” in a technologically advanced present. Pinker seems a bit chagrined at this angle being so overplayed, and in Rationality he attempts to redress the balance, making the case that humans are naturally equipped to reason and that reason is not a dark art or the province of philosophers and mathematicians alone.

Suggested reading

Never trust a scientist

By Stuart Ritchie

If you are a consumer of anthropology and ethnography or even have simply watched The Gods Must Be Crazy, you are aware of the preternatural skills of the San hunters of the Kalahari desert. Pinker shows this is not due to inborn traits, but the ability of the San to take in the information around them, the time of the year, the season, the species of antelope, and make inferences in alignment with their overall goals. The San, like all humans, engage in a rough and ready form of syllogistic reasoning. They start with axioms, premises informed by their experiences, and deduce logical consequences from their assumptions. Things don’t just “happen” in the Kalahari, there is a rhythm and rhyme the hunters exploit, a pattern and sense of the world that allows them to survive and flourish. Steenbok antelope are hunted during the rainy season while eland are stalked during the dry season. You might think this is due to custom and tradition, but the San are aware that steenbok has stiff joints during the rainy season while eland hooves are ill-equipped to navigate sandy soil. They hunt animals in the season that loads the dice for a success hunt. The alternative, simply, is a higher risk of starvation. The San have goals, and they naturally use the tools of rationality to achieve them.

But what if the San did not rationally know the reasons for their actions?

More from this author

Do genes determine intelligence?

By Razib Khan

One of the essential points in Rationality is that reasoning and understanding the world are collective enterprises. We know that socially-generated knowledge and rationality can be embedded enduringly through rules and taboos. Yes, those executing a cultural script may not be aware of the ultimate reasons, but a deeper investigation by anthropologists can reveal that mindless traditions actually serve functional purposes.

Joe Henrich in 2015’s The Secret of Our Success: How Culture Is Driving Human Evolution, Domesticating Our Species, and Making Us Smarter reports how traditional societies across the world have a system of preparing cassava that prevents them from getting cyanide poisoning. Those following the methods do not know what cyanide is, nor do they even know that consuming unprepared cassava will lead to death, but their culture comes preloaded with the fruits of adaptive learning that are highly rational.

The deepest insight then is that our ability to reason through complex problems depends on our willingness to stand on the shoulders of giants. Though we can all reason, Pinker does not shy away from conventional pitfalls of human analysis such as the Monty Hall problem, the gambler’s fallacy and the base rate fallacy. He recounts that Paul Erdos, author of 1,500 mathematical papers, was as mystified by the Monty Hall problem as most people who encounter it. He refused to believe he was wrong, relying on his native intuitions, despite his prodigious ability to work through formal mathematics.

The point is that the most brilliant individual mind can fail at reasoning unless it approaches the enterprise with humility and the willingness to update beliefs and understanding. The scholarship reviewed in Rationality, from Aristotle to machine learning, is not just the product of singularly brilliant minds, but the collective efforts of whole scientific traditions. The replication crisis shows that groups of scholars can blunder into traps and dead ends en masse. But advances in fields like statistical decision theory would be impossible without the collective efforts of disparate research groups across numerous domains. Pinker’s treatment of deep learning neural networks also makes it clear that some of these tools for rationality are approaching the dark edge of Arthur C. Clarke’s formulation that “any sufficiently advanced technology is indistinguishable from magic.” And, like magic, they often fail without the guiding hand of human reasoning faculties.

More from this author

How the Irish became white

By Razib Khan

Rationality, Pinker says, was partly in response to students at Harvard finding the world puzzling, especially with much of psychology focused on how human rationality fails. If Pinker makes the case for thinking clearly, with the most powerful tools we have, whether it be 21st-century computational methods in machine learning or sharper formal analysis that draws upon mid-20th-century economic game theory, he still remains fully aware that he lives in an empirical world of irrationality.

Pinker was not one of the ‘Four Horsemen’ of New Atheism: Christopher Hitchens, Richard Dawkins, Daniel Dennett and Sam Harris. But he is, and long has been, a fellow traveller of the group (he is also personally close to Dawkins). In the early 2000’s he took equal-opportunity aim at the Right’s attachment to traditional religion in The Blank Slate while taking the Left to task for its denial of human nature. Twenty years later, he made an extended call for a revival of the cult of rationality in Enlightenment Now. Pinker’s enthusiasm for applying rationality and empiricism to the world reflects the intellectual culture of the mid-2000’s when Dawkins was riding high with his critiques of evangelical Christianity and public intellectuals were gathering to celebrate the Enlightenment in conferences like Beyond Belief. The familiar flame of this hopeful past burns bright throughout Rationality.

Suggested reading

The Left's science denial

By Tom Chivers

But ours today is a world of cancel culture and QAnon. Ideological polarisation that has turned the pandemic into the object of a culture war in the United States. Pinker does address the reality that the world today is not the world he might have imagined it would be a generation ago, as irrationalities and superstitions spread like ideological pandemics on social media.

The vision of the aughts was one of a reason-ruled future, but Rationality was written in the wake of Donald Trump, whose rise and popularity as an emotive demagogue triggered a reactionary flare-up of identity politics on the cultural Left. This is the world where David Hume observed that “Reason is, and ought only to be the slave of the passions”. To a great extent, the project of making Bayesian reasoning, game theory and machine learning accessible and palatable to the public seems born of the disappointments of the past generation. If you are a deep reader of Scott Alexander’s Slate Star Codex, Rationality may not offer you much new food for thought. But if you are terrified by the insanities of the present, and wish to replenish your intellectual armoury in the fight for progress and reason in the 21st century, then Rationality is what you will want to bring to the battlefield.

Let’s say that you see that people who rate themselves highly on the scale are more likely to have quit than people who don’t. For example, people who gave themselves an eight or higher were twice as likely to be smoke-free after six months than people who gave themselves a four or below.

What’s your conclusion here? If you are Professor Ian Robertson, a clinical psychologist and author of How Confidence Works: The new science of self-belief, why some people learn it and others don’t, then the conclusion is obvious. Confidence helps you quit. If you believe — if you truly believe — that you have the willpower to quit, then you can do it! You have the “can happen” and the “can do” mentality, and that will drive you over the finish line!

If you’re anybody else, of course, you might point out that all this shows is that people can judge their own abilities quite well. I’m not confident in my ability to beat Magnus Carlsen at chess, and I expect Carlsen would be very confident in his ability to beat me. If we then played and I lost, I don’t think that would be because I was inadequately confident. In the same vein, if I know I’ve got a really bad 40-a-day habit and shitty willpower, then I’ll probably judge that I won’t be able to quit, and I may well be right. But it’s not that I’ll fail because I’m not confident; I’m not confident because I rightly think I’ll fail.

I’ve made up the smoking example, but Robertson uses several similar ones. “Studies have shown,” he says, right at the start of the book, “that you will live longer after heart failure if your partner feels confident about your condition.” If you find the study (not studies) he’s referring to,[1. Which is a pain in the arse to do, by the way, because he doesn’t use SODDING FOOTNOTES. Use footnotes! It’s fine! Readers don’t get derailed by little superscript numbers in the text!] you see that it does indeed find that heart disease patients are more likely to live longer if their spouses have high confidence that they will survive.

Suggested reading

How science has been corrupted

By Matthew Crawford

But… if I were to develop heart disease tomorrow, and you were to ask my wife whether she thinks I’ll survive the next six months, her answer wouldn’t simply be a product of whether she’s a confident, happy-go-lucky sort of person. It would also be based on her assessment of my health, my ability to stick to exercise and diet regimes, whether or not I tend to stick my fingers in electrical sockets for fun. She might say: “My husband is relatively young, and reasonably fit, so that’s positive, but on the other side of the ledger he’s a lazy git and eats too many burgers, so overall I’ll give it 60%.”

(To be fair, the study tries to look out for this by controlling for disease severity. But I would imagine that spouses are better judges of a person’s overall health than a four-category NYHA classification.)

For Robertson, though, confidence is everything. It’s the most important thing in the world. And everywhere he looks, science backs this up. It’s astonishing. A Duke University study looking at twins found that if one member of a pair of twins assesses their family’s social status as lower, that twin will likely do worse in life: worse jobs, more crime, etc. For Robertson, “something happened during adolescence that made some of the twins feel that they had lower social status than their sibling. Once they felt that, their behaviour, mental health, education, job and optimism suffered accordingly.”

The possibility that the causal arrow points the other way – that when a twin’s life starts to go down the toilet, they start to assess their (and their family’s) status more negatively – simply does not occur to him.

There are dozens of things like this. People who are pessimistic die young, people who are less confident in their memory are more likely to get dementia, people who walk more slowly are less confident in their health and will therefore die young. The possibility that people are accurately judging their own health status is occasionally raised but then dismissed. (Sometimes the studies controlled for other possibilities, but did they control for enough?)

Which is a shame. There is a fascinating book to be written by someone with a bit more self-reflection — someone a bit less confident, perhaps, in their thesis. Robertson divides confidence into two constituent parts: a “can happen” attitude and a “can do” attitude. If we’re trying to lose weight, say, someone might tell us to eat a healthier diet and take more exercise.

If we believe that salad and jogging will have the desired effect – that we would, if we ate better and ran more, lose weight — then we have a “can happen” attitude. And if we believe that we are personally capable of eating better and running more, then we have a “can do” attitude. Confidence, he says, is a “bridge to the future”: it is our mind’s ability to visualise our ability to get from here, where we are, to there, where we want to be.

More from this author

Can Dominic Cummings read the room?

By Tom Chivers

Robertson illustrates the idea with stories of CEOs and sportspeople and so on who have, as he says, this can happen/can do attitude, and how they therefore changed the world. 

But there’s another way of thinking about confidence: it’s just your brain predicting, given the information it has, how likely it is to succeed. You could imagine an AI that was built to perform some task, say image recognition. You train it on eleventy billion pictures of dogs and cats, and then you make it look at some other images of dogs and cats and say which ones are which. You feed it a thousand pictures and it gets 980 of them right.

Then you ask it (it’s also a natural-language AI) how confident it is that it will get the next one right. It says “98%.” Then it gets the next one wrong.

Why did it get the next one wrong? Is it because it wasn’t confident enough? No: it has correctly assessed that it will get it right 49 times out of 50. If the AI had said it was 99% confident, it would have been overconfident. The AI has learnt, from its experience of its own capabilities and the difficulty of the dog-cat-recognition task, that it gets it right 98% of the time.

And if the AI has to make some decisions based on its ability to tell a picture of a cat from a picture of a dog, it can only make those decisions rationally if it assesses its own skill accurately. If you offered it a bet at £1 to £25, it would be rational to take that bet; if you offered it a bet at £1 to £100, then it would not. Similarly, a human being asked “Can you quit smoking?” is making an assessment of his or her abilities, and will take action according to how likely he or she thinks they are to succeed.

Of course, you might say, AIs aren’t human. We do misjudge our own abilities, either overestimating or underestimating them, and it does feel from the inside as though confidence drives success as well as success driving confidence. And there might be some scientific evidence supporting that. This would have been the interesting book Robertson could have written: the book teasing out the difference between confidence-as-assessment-of-skill and confidence-as-motivation.

More from this author

Will science survive politics?

By Tom Chivers

Unfortunately, Robertson’s actual book is, again, so confident in the importance of confidence that he overlooks the shakiness of the science he cites. I’ve written and read a lot about the “replication crisis” over the last few years, and you get a bit of a sense for the sort of psychology studies which will turn out to be garbage. And my alarm bells were ringing constantly as I read this book.

For one thing, it repeatedly mentions psychological findings, such as stereotype threat, social priming, impostor syndrome, the “hot hand” effect, the Dunning-Kruger effect, and growth mindset, which have variously either been debunked or at least need some sort of acknowledgment that they’re controversial. 

More generally, it regularly quotes studies which seemed odd — so I went and looked about two dozen of them up. Studies into things like whether wearing a lab coat makes you better at concentrating, or whether being told that you’re smarter makes your brain look different in an fMRI scanner. Time after time, it was an unpreregistered study looking at 27 undergraduates which barely reached statistical significance. I am, I’m afraid, extremely not confident that most of these studies would replicate (and several of them definitely have not).

I have three hypotheses for why Robertson, who as a neuroscientist and psychologist is presumably aware of the replication crisis, might overlook the shakiness of the studies supporting his thesis. One is charitable: the others less so.

The charitable explanation is that Robertson is a clinical psychologist. If this book were a book of robust science, it would be full of caveats and on-the-one-hand-on-the-others. But perhaps it’s not. Perhaps it’s a self-help book: “Get confident, stupid!” Robertson wants to get his readers to maximise their confidence, so he pushes its importance and overstates the effectiveness of various tricks to improve it. It is full of pretty common-sense advice (tell yourself you’re excited rather than anxious, etc); maybe he feels that’ll do more good if it’s backed up with neurobollocks about how it’s all to do with dopamine receptors in the ventral striatum or whatever.

The first less charitable explanation is that Robertson, like many other moderately famous psychologists, has his eye on the lucrative after-dinner speaking market, and the way to get on that is to say that This One Weird Trick is all you need to totally change your life, whether it’s growth mindset or “grit” or power posing or whatever. In this model, Robertson has decided that “confidence” is the new “positive psychology”, and he can get £10,000 a go for 45 minutes talking to postprandial advertising executives in a conference hall in the Runnymede Marriott.

Suggested reading

What makes us hate?

By Stuart Ritchie

The second less charitable explanation is … well. Robertson tells the story of Ally McLeod, who took Scotland to the World Cup in 1978, promising not only to win it but to retain it the next time around. Scotland were instead knocked out in the group stages . “Ally MacLeod was what is colloquially known as a bullshitter,” he says. My own understanding of “bullshit”, in the Harry Frankfurt sense, is speech in which the speaker doesn’t care whether he is telling the truth or not; but for Robertson, it’s essentially problematic overconfidence, misjudging your own expertise and abilities. It is possible that one or both of those two definitions might apply to this book.

But this book is what it is. One of Robertson’s key anecdotes is of Padraig Harrington, the golfer, having a moment of doubt while comfortably in the lead in the 2007 British Open, and then repeatedly whiffing his ball into a lake, almost throwing his lead away, and then getting his confidence back and going on to win. There’s a fascinating story there to be told — was the confidence (and its disappearance) a product of how well he was doing or wasn’t, or the other way around? Did they feed back into each other in a complex, self-reinforcing loop? And was that initial moment of doubt even real, or was it added into his mind with hindsight? (It’s not as if memory is infallible.) I would love to have read someone treating that complicated story with the subtlety it deserves.

But for Robertson, the story is simple. Harrington was confident, so he was winning; then he stopped being confident, so he stopped winning; then he got confident again, so he won. The end. Get confident, stupid.

The people, some commentators caterwaul in dismay, just don’t trust the experts anymore. But an essential part of this story is that the experts have to a significant extent lost trust in each other and in their institutions. This shift came, in large part, due to the replication crisis in science.

In 2010 Dana Carney, Amy Cuddy and Andy Yap wrote a paper in the academic journal Psychological Science detailing a finding about a certain kind of “powerful” pose — most iconically, one in which the poser has their hands on their hips and their legs spread apart. The authors, based on a study with 42 participants, asserted that “minimal posture changes” could “over time and in aggregate… potentially… improve a person’s general health and well-being.”

They went on to say: “This potential benefit is particularly important when considering people who are or who feel chronically powerless because of lack of resources, low hierarchical rank in an organization, or membership in a low-power social group.” In 2012, Cuddy gave a popular TED Talk called “Your Body Language May Shape Who You Are,” and in 2015 she published a self-help book called Presence: Bringing Your Boldest Self to Your Biggest Challenges.

It was a New York Times best-seller, but right around the same time her psychologist colleagues were starting to try, and fail, to “replicate” her findings — to redo her experiments and achieve similar results. They could not find independent evidence of the effect she had become rich and famous for trumpeting.

Power posing was like catnip for a lot of people, and it’s symptomatic in many ways of the problems with public engagement by experts in social psychology. First, it suggests that humans and our social world are highly malleable, our identities susceptible to massive alterations as consequences of tiny, simple interventions. There’s no particular a priori reason to think that these interventions will have the claimed effects: no first principles, no overarching theory. Second, it presents a kind of feel-good solution to problems of a vulnerable group: here, the homogenised “chronically powerless.” Third, while it originated within the scientific community, it made its way to the public consciousness at least in part through being commodified for an audience that was only superficially scientifically literate — the TED Talk crowd.

The replication crisis is an ongoing event across the sciences involving the failure of published and often celebrated results to replicate in subsequent experiments. It is the most important science story of our times and the subject of psychologist Stuart Ritchie’s new book Science Fictions.

Suggested reading

Do any Covid-19 'cures' actually work?

By Tom Chivers

Ritchie tells the story very well; the achievement of the book as a work of popular science comes from the way it switches smoothly back and forth among three tasks, all of which it performs with considerable success.

First, it gives the unfamiliar reader an overview of the details of professional science — grants, journals, peer review, that kind of stuff. Second, it develops evocative and engaging narratives, sometimes funny, sometimes shocking, of specific instances of scientific misconduct and error. Third, it delves into quite technical material about statistics and experimentation, covering problematic research practices like p-hacking and potential stopgaps and solutions like specification-curve analysis.

Ritchie focuses on psychology, his area of expertise and the field most prominently hit by the replication crisis. Recent papers seemingly established things like “college students having psychic powers” (Ritchie was personally involved with the unsuccessful attempt to replicate that paper) and “messier or dirtier environments cause a rise in prejudice and stereotyping” (the author of that paper turned out to have wholly fabricated the data).

As these results started being discredited, classic results came under scrutiny too. The Stanford Prison Experiment of Philip Zimbardo, which found students assigned to be “guards” in a week-long simulation abusing students assigned to be “prisoners,” has been famously used to suggest the near-universality of human cruelty; and the “priming” studies, in which tiny, unconscious suggestions can lead participants to behave in very different ways, were taken by Nobel laureate Daniel Kahneman to “threaten our self-image as conscious and autonomous authors of our judgments and our choices”. A meta-replication in 2015 found that of 100 chosen studies from three psychology journals, only 39% of experimental results replicated.

Other such studies found rates of 62%, 77%, 54% and 38%. Not good numbers! And Ritchie notes that similar meta-replications have produced worrying results in economics, neuroscience, evolutionary biology, marine biology and organic chemistry, and, perhaps most worryingly, medicine. In some cases, studies fail not replication but “reproduction” — other researchers analyse the data set actually obtained in the original study and come to different conclusions due to errors, unspoken assumptions or unjustified analytic decisions.

Ritchie begins his book with a humorous overview of the scientific process. His goal is to show that this process is ineluctably social, and he does this convincingly. Scientists must justify their work to other scientists, both in terms of its importance and in terms of its accuracy, and scientists must check other scientists’ work. When one scientist publishes a result, if that result is true, another scientist should, barring chance deviations, be able to replicate it in their own lab. If such a replication cannot be done, the result means much less, because it now describes not how the world is in general, but at best how one scientist’s experiment went.

Suggested reading

Don’t trust the psychologists on coronavirus

By Stuart Ritchie

Why might a published result fail to replicate — that is, why might an academic journal publish a result that’s ultimately false? Ritchie mentions four underlying causes: fraud, bias, negligence and hype. Fraud can be objectively very obvious while still escaping notice, as in studies which use the same image for purportedly different situations. And scientific institutions often fail to hold fraudulent scientists to account when those scientists are prestigious, due to both gullibility and self-interest.

“Bias” includes various kinds of political biases and identity-based stereotypes, but Ritchie stresses that biases in favor of intellectual novelty and impact are if anything more significant in the current scientific landscape, including the well-known bias against publishing negative results — that is, against publishing papers that tell us only that something exciting did not happen.

Ritchie’s section on bias also includes a clear account of some technical elements of statistical reasoning, failures to adhere to which are Ritchie classes as “analytic biases.” These biases include things like only reporting a portion of collected data, excluding certain data points for arbitrary reasons, and deciding whether or not to continue looking for data based on initial results. These biases can in fact creep into a scientist’s work rather innocently, based merely on their conviction that their hypothesis is correct — the “bias” in question.

Surveys found that 65%, 40% and 57% of psychologists respectively had engaged in those three practices. The section on negligence also includes some great descriptions of statistical techniques and instruments that have been developed to detect various kinds of scientific error, and the section hype mentions NASA’s claim, later debunked, to have found arsenic-based life on Earth, as well as the “growth mindset” fad, the “probiotics” fad, and the ever-changing directives of nutritional science.

Suggested reading

The myth of the anti-science Middle Ages

By James Hannam

Serious though this is, there is also something more specifically pernicious about the replication crisis in psychology. We saw that the bias in psychological research is in favour of publishing exciting results. An exciting result in psychology is one that tells us that something has a large effect on people’s behavior. And the things that the studies that have failed to replicate have found to have large effects on people’s behavior are not necessarily things that ought to affect people’s behaviour, were those people rational. Think of the studies I mentioned above: a mess makes people more prejudiced; a random assignment of roles makes people sadistic; a list of words makes people walk at a different speed; a strange pose makes people more confident. And so on.

All of these studies involve odd effects from environmental cues. Taken together, they suggest an image of human behaviour which is highly irrational and highly manipulable. This is, in fact, the image of human behaviour which I think a great many educated people hold at the moment.

Of course, the existence of the replication crisis does not by itself mean that this image is the wrong one. But as these overhyped and ultimately flawed studies have made their way through the press releases and TED Talks Ritchie discusses into the broader consciousness of a certain class, I think they’ve helped persuade that class that that image is indeed the right one. As we reevaluate our scientific institutions, it may pay to reevaluate this image as well, and its influence on our social organisation more broadly.

My six-year-old son’s hands were red and raw too, from his conscientious efforts at washing his hands at school. He would proudly tell me that his teacher had said he was the best hand-washer in his class, and it broke my heart even as I praised him for it. I’m not an anxious or depressive person, and I’m not usually stressed. But the coronavirus pandemic — the uncertainty, the fear, and the god-damn handwashing — got to me.

It’s easier now: you get into routines, the weird becomes normal; plus I started to backslide a bit on the handwashing, once they shifted the emphasis to social distancing. But, for a while, I could feel myself struggling. 

So when people say that Covid-19 is bringing with it a secondary epidemic of mental health problems, I can easily believe it. I can believe it because I could see how tough being out of school was on my kids, and because I heard other people saying the same; and because we were all well-off people of privilege, so I could only imagine that it was worse for others. I can believe it because I know so many people who went from being full-time workers to full-time workers and full-time parents and educators. It’s a bloody stressful time.

Mental health matters. If people are suffering significant mental health impacts because of coronavirus, or because of the lockdown, then we need to know, because that will affect the correct way to respond, personally and as a society. We need to honestly address the trade-offs, the costs and benefits, of the policies our country takes to minimise the damage of the virus.

That means that research into the mental health impact is important — and there’s lots of research going on. The trouble is that, just as in so much of science at the moment, everything has to happen fast. That’s understandable, but it can cause real problems. 

More from this author

Will there be a second Covid wave?

By Tom Chivers

Pete Etchells, a professor of psychology at Bath Spa University, told me that “there’s a lot of scrabbling around to do research on the wider effects of Covid. Not just in psych. And it comes from a good place; people want to help and do something useful.” But, he says, we need to be extraordinarily careful. “When there’s stuff going out there that either has the aim of, or has an impact on, policy decisions, it has a real potential to have a big impact on people’s lives.” 

Careless science can lead to wrong conclusions amazingly easily. Psychology has learnt that lesson more harshly than most disciplines, having suffered huge and embarrassing setbacks during the “replication crisis”. What it has discovered is that some precautions are vital. Some of these are slightly technical-sounding but important, such as preregistration of hypotheses, and committing to publishing regardless of the results, which avoid innocent and deliberate statistical malpractice. And, perhaps most importantly, it’s vital to share your data and methods, so other researchers can check your work.

I want to flag two papers looking at Covid and mental health, each with very different approaches to how you do science, to show you what I mean. One of them came out in May, and was intended to establish the “validity” of a new measure that one of the authors had developed, a “fear of Covid-19” scale, back in March. The authors have since used it in further research.

The “validity” of a scientific measure is how well it applies to the real world. For instance, most research into smartphones and mental health has been carried out using one of several questionnaires which asked people about their phone use. But a study last year discovered that how much people thought they used their phones was only tangentially related to how much they actually used it, as recorded by a special app. So the phone-use scales are not very valid: they aren’t very accurate at measuring the thing they’re supposed to be measuring. 

Validity is crucial in psychology, because so much of it is based on questionnaires and surveys. If I make a questionnaire to give to patients, and which then gives them a score, and that score is called their “suicidality score”, then it is really important that people who score highly on it are at higher risk of suicide than people with low scores. 

It’s important because, if that scale is picked up, other people will do research using it. So they might do a trial of some antidepressant, and measure subjects’ suicidality scores on this scale before and after. Then, if the subjects’ scores improve, the authors can say “this drug improves scores on the Chivers Suicidality Scale by an average of 11 points”. And it might get picked up by NICE and given to patients by the NHS. 

So, if the suicidality score is actually not valid, if it doesn’t relate to real-world outcomes, then it is worse than useless: it is directing resources and future research away from the places where they can do good.

More from this author

Can you really teach kids not to be racist?

By Tom Chivers

That means the paper establishing the validity of the new “fear of Covid-19” scale is very important. The authors seemed to take it seriously, recruiting 8,500 Bangladeshi students to test it on. And, vitally, they said that “data will be available on request” if other researchers want to look at their findings.

But — disturbingly — they haven’t gone ahead with that. Another researcher, Brittany Davidson — one of the authors of the smartphone self-report validation study I mentioned earlier — asked them for the data several months ago. She had wanted to look at the data because, she says, she was wary of psychometric scales coming out with a “lack of rigour”, and she had questions over how the authors had managed to recruit such a huge sample size in a very short time, apparently without offering to pay the subjects anything; that is, she says, very unusual.

So far, none of the data has been forthcoming. This is really out of the ordinary, and worrying, especially since — as far as we can tell — the data is not sensitive. Researchers keeping data to themselves was a significant part of why the replication crisis got so bad. It’s especially bad when they say in the paper that they’ll share it.

You may feel this is all a bit technical and unimportant, but it’s not. The original “fear of Covid-19 scale” paper has more than 120 citations already. The validation paper was carried out in Bangladesh, but the authors have also created English, Turkish, Iranian, Spanish and other versions of the scale, in each of those languages. A small but non-negligible percentage of the research into the mental health impacts of Covid-19 is carried out using this scale. 

And its authors make serious claims about those impacts: they say in one paper that Covid-19 has led to people “suffering from elevated anxiety, anger, confusion, and posttraumatic symptoms”, to “unusual sadness, fear, frustration, feelings of helplessness, loneliness, and nervousness”, and that in “extreme cases, it may trigger suicidal thoughts and attempts and, in some cases, actually result in suicide”. This is important, non-trivial stuff.

More than that: the authors use the scale to propose policies for national governments, such as banning and removing websites that host “misinformation (e.g., false COVID-19, treatment remedies, COVID-19 conspiracy theories)”, or having government health agencies “make online counseling sessions available to help ease the mental health concerns and worries of the general population”. It is very possible that, somewhere in the world, a government white paper will be written citing the “fear of Covid-19” scale as support for some policy or other. (Stranger things have happened.) And yet other researchers cannot look at it to see how it was validated. 

(There are, I should note, other concerns with some of the research of one of the authors, some raised here by psychologist Dorothy Bishop and others here by the psychologist Nick Brown.)

As we said earlier, it seems obvious that Covid-19 is dragging a mental health crisis in its wake. It certainly seems obvious to me. So you might think it doesn’t really matter if some papers cut corners a bit in order to tell us things that we already know.

That’s why I wanted to bring up another piece of work, which seems to me an example of careful research in this fast-moving and scary topic. It looked at 800 adults living alone in the UK and USA, and measured their mental health on well-established, well-validated scales (the standard measures of depression, anxiety and loneliness). 

The paper measured their responses at three points, one some time before the crisis had really taken off, and two after lockdown had started. Vitally, also, it was a “registered report”: that is, the authors said in advance what they were looking for, and the journal agreed to publish it as long as it was carried out according to its methods, regardless of its results. That rules out lots of the bad statistical practice that can distort science.

Just as the existence of a Covid-19 mental health crisis seemed obvious to me, it seemed obvious to the authors of this study, Dr Netta Weinstein of the University of Reading and Dr Thuy-Vy Nguyen of the University of Durham. They just assumed that it was true; so much so that they were only measuring them to look at other things, such as whether introverted people were coping better.

But to both authors’ surprise, they simply didn’t find any increase at all on any of the three scales. The average level of loneliness, anxiety and depression did not go up. It was — to me, and to the authors — a startling result.

Suggested reading

Why Covid won't kill off populism

By Matthew Goodwin

It’s not the last word. There could be all sorts of reasons why it didn’t find any impact. Weinstein wondered if the study, looking at people living alone, missed the stress placed on families. Or it could be that depression, anxiety and loneliness aren’t the main things that went up. “We didn’t look at stress,” she says: “I can remember that moment when I realised I was a full-time mum and a full-time academic, that sense of panic, but I don’t think I felt depressed.” 

But the point is: we would all have confidently predicted, like Nguyen and Weinstein, that lockdown would make people living alone more lonely, more anxious and more depressed. They didn’t find that. Psychological research is messy and hard; it demonstrates the importance of really good, careful science, especially in messy topics like psychology, and especially in fast-moving situations like this. “This is a good opportunity to practise all the stuff that came out of the replication crisis to make sure that the answers we get are the right ones,” says Etchells. Open data, preregistered hypotheses, well-validated scales: we know this stuff is important. 

My stress levels have returned to normal now; partly that’s routine, partly it’s kids being back at school, partly it’s my hands being back to their usual washing-up-liquid-advert softness. But the issues surrounding Covid-19 and mental health have not gone away; if nothing else, we can expect economic hardship and widespread unemployment in the coming years, and we know that is linked to mental health problems. Research into those issues will only get more important, to help mitigate those problems. But it needs to be good research. The lessons of the replication crisis need to be learned.

All scientists are part of this to some degree: most prominently, the peer-review system, at least in theory, involves scientists critiquing each other’s studies, throwing out the bad or mistaken ones, and suggesting improvements for those that are salvageable.

Some scientists, though, make their entire reputations as critics, loudly drawing attention to the flaws and failings in their fields. It’s hard not to respect these often-eccentric characters, who stand up against groupthink and intellectual inertia, telling entire fields of research what they don’t want to hear. But recent months have given us some cautionary tales about these scientific watchmen, and have shown in excruciating detail how even the most perceptive critics of science can end up bafflingly wrong.

In my own field of psychology, one of the most prominent examples of an uber-critic was Hans Eysenck. From the 1950s all the way to his death in 1997, Eysenck wrote blistering critiques of psychoanalysis and psychotherapy, noting the unscientific nature of Freudian theories and digging into the evidence base for therapy’s effects on mental health (I should note that Eysenck worked at the Institute of Psychiatry, now part of King’s College London, which is my employer).

In one typically acrimonious exchange in 1978, Eysenck criticised a study that had reviewed all the available evidence on psychotherapy. Eysenck argued that this kind of study — known as a “meta-analysis” because it tries to pool all the previous studies together and draw an overall conclusion — was futile, owing to the poor quality of all the original studies. The meta-analysis, he said, was “an exercise in mega-silliness”: a “mass of reports—good, bad, and indifferent—are fed into the computer”, he explained, “in the hope that people will cease caring about the quality of the material on which the conclusions are based.”

Whether or not this was a sound argument in the case of psychotherapy, Eysenck had put his finger on an important issue all scientists face when they try to zoom out to take an overall view of the evidence on some question: if you put garbage into a meta-analysis, you’ll get garbage out.

More from this author

Don’t trust the psychologists on coronavirus

By Stuart Ritchie

You would think that a scientist so concerned with the garbage in the scientific literature would do his best to avoid producing more garbage himself. But around the same time as he was excoriating meta-analysis, Eysenck began collaborating with one Ronald Grossarth-Maticek, a therapist working at Heidelberg University. Grossarth-Maticek had, he claimed, run large-scale studies of how personality questionnaires could predict fatal disease, and how his special kind of behavioural therapy could help people avoid cancer and heart disease. Eysenck worked with him to get these data into the scientific literature, eventually producing many dozens of scientific papers and reports.

The first indication that something might be a little off in this research comes if you look at the questions being asked. Usually a personality questionnaire would include questions like “Can you get a party going?” or “Do you enjoy sunbathing on the beach?” The Eysenck-Grossarth-Maticek questionnaire included questions like the following — to which a yes/no answer is required:

“Do you change your behaviour according to consequences of previous behaviour, i.e., do you repeat ways of acting which have in the past led to positive results, such as contentment, wellbeing, self-reliance, etc., and to stop acting in ways which lead to negative consequences, i.e., to feelings of anxiety, hopelessness, depression, excitement, annoyance, etc.? In other words, have you learned to give up ways of acting which have negative consequences, and to rely more and more on ways of acting which have positive consequences?”

Go on. Yes or no? Quickly, please.

The second issue is with the results. Frankly, they’re unbelievable. Answers to the kind of question quoted above could, the pair claimed, classify people into “Cancer-Prone”, “Heart Disease-Prone” or “Healthy” personalities — with massive implications for their lives. Cancer-Prone personalities were an astonishing 120 times more likely to die of cancer than Healthy personalities in the next ten years (the equivalent number for Heart-Disease Prone personalities dying of heart disease was 27 times, also amazingly high). And in a trial of Grossarth-Maticek’s therapy, not one treated patient died of cancer, whereas 32% of the control participants, who received no therapy, did.

As one of Eysenck’s critics, Anthony Pelosi, has argued, “such results are unheard of in the entire history of medical science”. There are only three possibilities: they’re either the most important results ever discovered in medicine (a 100% chance of avoiding cancer after some talking therapy!), grossly mistaken in some way, or made up. I suspect the answer isn’t the first one.

These results were criticised in the early 1990s (including by a statistician who’d seen the raw data, and who more or less argued they were fraudulent), though they were vociferously defended by Eysenck. He wrote that the criticisms, “…however incorrect, full of errors and misunderstandings, and lacking in objectivity, may have been useful in drawing attention to a large body of work, of both scientific and social relevance, that has been overlooked for too long.”

It was only in 2020 that the self-correcting nature of science really started to bite: after a resurgence of criticism from Pelosi and others, King’s College London investigated Eysenck’s work with Grossarth-Maticek, listed many (though not all) of the articles that used the data as “unsafe”, and wrote to the relevant scientific journals advising them to retract the papers. So far, 14 have been pulled from the literature (with a further 64 given an editorial “expression of concern”) — and given how many papers the duo published on these bizarre studies, this may end up being the tip of a rather large iceberg.

How had such a strong advocate of rigour in science ended up presiding over one of the most ludicrous sets of scientific papers ever published? There are allegations that he received funding from tobacco companies, who would have stood to benefit if it was personality rather than cigarettes that caused cancer, and that this might have influenced his reasoning (a conflict of interest that was never fully declared).

But deeper explanations relate to Eysenck’s personality. When he wasn’t railing against psychotherapy, he was publishing and debating almost every other contentious issue in the book, including crime and violence, astrology, extra-sensory perception, and the genetics of race and intelligence. This was someone who loved argument, loved controversy — and most importantly, refused in almost any case to back down under criticism (see this cringeworthy video for further evidence). Once he found himself deeply involved in the Grossarth-Maticek studies, he felt beholden to defend them despite their transparent absurdity.

The strange story of Eysenck —  an arch-critic who conspicuously failed to see the flaws in his own work — has come to mind several times while I’ve been following a far more contemporary controversy: the case of John Ioannidis and COVID-19.

It’s fair to say that Stanford University’s John Ioannidis is a hero of mine. He’s the medical researcher who made waves in 2005 with a paper carrying the firecracker title “Why Most Published Research Findings are False”, and who has published an eye-watering number of papers outlining problems in clinical trials, economics, psychology, statistics, nutrition research and more.

Like Eysenck, he’s been a critic of meta-analysis: in a 2016 paper, he argued that scientists were cranking out far too many such analyses — not only because of the phenomenon of Garbage-In-Garbage-Out, but because the meta-analyses themselves are done poorly. He’s also argued that we should be much more transparent about conflicts of interest in research: even about conflicts we wouldn’t normally think of, such as nutrition researchers being biased towards finding health effects of a particular diet because it’s the one that they themselves follow.

Ioannidis’s contribution to science has been to make it far more open, honest, and self-reflective about its flaws. How odd it is, then, to see his failure to follow his own advice.

First, in mid-March, as the pandemic was making its way to America, Ioannidis wrote an article for STAT News where he argued that we should avoid rushing into big decisions like country-wide lockdowns without what he called “reliable data” on the virus. The most memorable part of the article was his prediction — on the basis of his analysis of the cursed cruise ship Diamond Princess — that around 10,000 people in the US would die from COVID-19 — a number that, he said, “is buried within the noise of the estimate of deaths from ‘influenza-like illness’”. As US deaths have just hit 125,000, I don’t need to emphasise how wrong that prediction was.

So far, so fair enough: everyone makes bad predictions sometimes. But some weeks later, it emerged that Ioannidis had helped co-author the infamous Santa Clara County study, where Stanford researchers estimated that the number of people who had been infected with the coronavirus was considerably higher than had been previously supposed. The message was that the “infection fatality rate” of the virus (the proportion of people who, once infected, die from the disease), must be very low, since the death rate had to be divided across a much larger number of infections. The study became extremely popular in anti-lockdown quarters and in the Right-wing populist media. The virus is hardly a threat, they argued — lift the lockdown now!

More from this author

Our dangerous addiction to prediction

By Stuart Ritchie

But the study had serious problems. When you do a study of the prevalence of a virus, your sample needs to be as random as possible. Here, though, the researchers had recruited participants using Facebook and via email, emphasising that they could get a test if they signed up to the study. In this way, it’s probable that they recruited disproportionate numbers of people who were worried they were (or had been) infected, and who thus wanted a test. If so, the study was fundamentally broken, with an artificially-high COVID infection rate that didn’t represent the real population level of the virus (there were also other issues relating to the false-positive rate of the test they used).

Then, an investigation by Stephanie Lee of BuzzFeed News revealed that the study had been part-funded by David Neeleman, the founder of the airline JetBlue — a company that would certainly have benefited from a shorter lockdown. Lee reported that Neeleman appeared to have been in direct contact with Ioannidis and the other Stanford researchers while the study was going on, and knew about their conclusions before they published their paper. Even if these conversations didn’t influence the conduct of the study in any way (as argued by Ioannidis and his co-authors), it was certainly odd — given Ioannidis’s record of advocating for radical transparency — that none of this was mentioned in the paper, even just to be safe.

Ioannidis didn’t stop there. He then did his own meta-analysis of prevalence studies, in an attempt to estimate the true infection fatality rate of the virus. His conclusion — once again — was that the infection fatality rate wasn’t far off that for the flu. But he had included flawed studies like his own one from Santa Clara, as well as several studies of the prevalence that only included young people — biasing the death rate substantially downwards and, again, not representing the rate in the population (several other issues are noted in a critique by the epidemiologist Hilda Bastian). That German accent you can hear faintly in the background is the ghost of Hans Eysenck, warning us about the “mega-silliness” of meta-analysing low-quality studies.

Suggested reading

Slate Star Codex must remain anonymous

By Tom Chivers

His most recent contribution is an article on forecasting COVID-19, upbraiding the researchers and politicians who predicted doomsday scenarios with overwhelmed hospitals. His own drastic under-prediction of 10,000 US deaths? Not mentioned once.

Although Ioannidis has at least sounded as if he’s glad to receive criticism, some of his discussion of the more mainstream epidemiological models has sounded Eysenckian — for instance, where he described the Imperial College model of the pandemic as having been “astronomically wrong”. There is, of course, a genuine debate to be had on how and when we should lift our lockdowns. There’s also a great deal that we don’t know about the virus (though more reliable estimates suggest, contra Ioannidis, that its infection fatality rate is many times higher than that of the flu). But Ioannidis’s constant string of findings that all confirm his initial belief — that the virus is far less dangerous than scientists are telling you — gives the impression of someone who has taken a position and is now simply defending it against all comers.

And for that reason, it’s an important reminder of what we often forget: scientists are human beings, and are subject to very human flaws. Most notably, they’re subject to bias, and a strong aversion to having their cherished theories proved wrong. The fact that Ioannidis, the world’s most famous sceptic of science, is himself subject to this bias is the strongest possible confirmation of its psychological power. The Eysenck and Ioannidis stories differ in very many ways, but they both tell us how contrarianism and iconoclasm — both crucial forces for the process of constant scepticism that science needs to progress — can go too far, leading researchers not to back down, but to double-down in the face of valid criticism.

Above, I should really have said that John Ioannidis was a hero of mine. Because this whole episode has reminded me that those self-critical, self-correcting principles of science simply don’t allow for hero-worship. Even the strongest critics of science need themselves to be criticised; those who raise the biggest questions about the way we do research need themselves to be questioned. Healthy science needs a whole community of sceptics, all constantly arguing with one another — and it helps if they’re willing to admit their own mistakes. Who watches the watchmen in science? The answer is, or at least should be: all of us.

Stuart Ritchie is a Lecturer in the Social, Genetic and Developmental Psychiatry Centre at King’s College London. His new book, Science Fictions: Exposing Fraud, Bias, Negligence and Hype in Science, is published on July 16.

Then some of the most famous studies from social psychology’s 1970s heyday fell apart on closer scrutiny. The Stanford Prison Experiment, where people were assigned roles as “prisoners” and “guards”, and the guards ended up treating the prisoners abominably? Probably misreported. The study where “pseudopatients” admitted themselves to psychiatric hospitals, acted entirely sane, and were locked up and medicated regardless? Possibly fraudulent.

Now, psychologists are disgracing themselves anew over the coronavirus.

It started with articles relying on psychological “insights” to downplay the severity of the problem. In early February, social psychologist David DeSteno wrote a piece in the New York Times arguing that people get so caught up with their fear of the virus, they fail to understand that they’re unlikely to get it. Referencing some of his own lab experiments, DeSteno wrote that “…quarantine or monitoring policies can make great sense when the threat is real and the policies are based on accurate data. But the facts on the ground, as opposed to the fear in the air, don’t warrant such actions.”

Two days later, the New York Times’s Interpreter column quoted psychologist Paul Slovic, who noted that “[o]ur feelings don’t do arithmetic very well”, and that focusing on the coronavirus fatalities, and not the “98% or so of people who are recovering from it and may have mild cases” is skewing our judgement. The article argued that our fears, triggered by disturbing reports of “city-scale lockdowns and overcrowded hospitals”, overload our critical faculties, making us overreact to the threat the virus poses. The thought that those city-scale lockdowns and overcrowded hospitals might be a mere month away from the United States didn’t seem to occur.

Further psychological insights were provided by Cass Sunstein, co-author of the best-selling book Nudge, which used lessons from behavioural economics (essentially psychology by another name) that could inform attempts to change people’s behaviour. In an article for Bloomberg Opinion on 28 February (by which point there were over 83,000 confirmed coronavirus cases worldwide), Sunstein wrote that anxiety regarding the coronavirus pandemic was mainly due to something called “probability neglect”.

Because the disease is both novel and potentially fatal, Sunstein reasoned, we suffer from “excessive fear” and neglect the fact that our probability of getting it is low. “Unless the disease is contained in the near future,” he continued, “it will induce much more fear, and much more in the way of economic and social dislocation, than is warranted by the actual risk”.

On 12 March, the day after Italy had announced its 827^th death from the virus, the eminent psychologist Gerd Gigerenzer published a piece in Project Syndicate entitled “Why What Does Not Kill Us Makes Us Panic”. It was, to say the least, confused: it opened with an acknowledgement that we don’t know how bad this epidemic could be, but immediately went on to make the case that we’d likely overreact, and failed to consider any opposing arguments.

Gigerenzer’s article discussed the 2009 swine flu pandemic, which affected hundreds of millions but killed relatively few, particularly in Europe, where the fear was nonetheless high. It also likened the coronavirus pandemic to terrorist attacks, which similarly kill small numbers in a relative sense, but — according to Gigerenzer — garner disproportionate attention. If only we could improve our “risk literacy”, Gigerenzer maintained, then we could “approach situations such as the COVID-19 epidemic with a cooler head”.

At the time of writing, Italy’s death toll is now above 10,000, with over 900 fatalties being reported in a single day. The epidemic in the UK has reached 1,000 deaths; in France, 2,000; in Spain, nearly 6,000. New York City is setting up makeshift morgues for the first time since 9/11. The suspiciously uniform daily death figures from Iran might belie a truly devastating epidemic in that country. The world’s economy has largely closed down, as we all anticipate the pandemic getting far worse before it gets better. How could all those psychologists have gotten it so disastrously wrong?

Suggested reading

Would you take a coronavirus risk?

By Tom Chivers

To back up his points about “probability neglect”, Sunstein had referred to a 2001 paper in the journal Psychological Science. It reported three experiments; Sunstein focused on the third one, which included 156 participants, all of whom were undergraduate students reasoning about how much they’d pay to avoid an imaginary electric shock. It’s not a criticism of the scientists to say that this experiment is only tenuously relevant to a global pandemic.

Indeed, alongside talk of the “replication crisis” there’s been discussion of a “generalisability crisis”, with renewed realisation that results from lab experiments don’t necessary generalise to other contexts. A global pandemic of a completely novel virus is, by its very definition, a context never encountered before. So how can we be sure that the results of behavioural science experiments — even those that are based on bigger or more representative samples than 156 undergrads — are relevant to our current situation?

The answer is that we can’t. Exploring the human capacity for bias and irrationality can make for quirky, thought-provoking articles and books that make readers feel smarter (and can build towards a tentative scientific understanding of how the mind works). But when a truly dangerous disease comes along, relying on small-scale lab experiments and behavioural-economic studies results in dreadful misfires like the articles we encountered above.

Although there are many behavioural phenomena that certainly seem relevant to today’s news — bias, sunk costs, the tragedy of the commons — it’s not at all clear how these concepts would be practically applied to do what needs to be done right now: slowing the spread of the disease.

We can even play the psychologists at their own game: there are also biases in the opposite direction to those discussed above. For instance, “exponential growth bias” might mean that our standard ways of reasoning break down in a situation where a threat accelerates like this virus. As observed in an insightful article on Italy’s ongoing horrific experience, there’s also “confirmation bias”, where we seek out evidence that confirms our previous beliefs or desires.

In this case, many were understandably desperate to believe that the virus wouldn’t be too much of a problem, and it seems to have led them to underreact very badly indeed. None of the psychological writers above seemed to consider these opposing biases — and ironically, it might have been because they were biased against thinking about them.

Suggested reading

I'm frightened I'll die of Covid-19

By Stephen Pollard

That latter idea is just speculation on my part — which is the whole point. As the psychologist Stefan Schubert has noted, “[i]t’s too easy to make up a just-so-story” about us being biased in one direction or another. “The ‘psychology of X’ (e.g. risk, disease) can only tell us so much about X. We should mostly look at X itself”.

Quite so: the critical task for now is not to try and get inside people’s heads, but to learn from the experience of other countries (like Taiwan, South Korea and Singapore) that have been more successful in containing covid-19; and to build realistic mathematical models of the epidemic, and to pour resources into medical advances — like tests, vaccines, and antivirals — that might help us to track and inhibit the transmission of the virus and prevent its symptoms.

Sunstein himself appears to have undergone something of a Damascene corona-conversion. On 26 March, he wrote another column for Bloomberg Opinion, entitled “This Time the Numbers Show We Can’t Be Too Careful”. In an argument that was directly opposed to his own reasoning of a month earlier, Sunstein suggested that the potentially ruinous effects of the pandemic mean that “the benefits of aggressive social distancing greatly exceed the cost”. Remarkably, he made no reference whatsoever to his previous article on this specific crisis — an article which, as the writer Ari Schulman has pointed out, had said that if you held the view Sunstein now holds, you were the sad victim of a cognitive malfunction.

It’s impossible to know how many people changed the way they thought about the virus after reading one of the misconceived articles above. But a consistent set of articles by credentialed scientists telling people they shouldn’t worry so much about the pandemic was the precise opposite of what was required in a situation where governments would eventually, and rightly, be imploring their citizens to stay at home at all costs. At best, the advice from psychology was unhelpful. At worst, it constituted dangerous misinformation.

And what of those governments? Could the “don’t overreact” advice from behavioural science have led politicians to drag their feet in dealing with the crisis, as some clearly have? Again, it’s hard to say — but it’s very possible that the broad message got through to the decision-makers.

Thankfully, the advice given to the UK government by behavioural scientists, which was published just over a week ago, didn’t contain any minimisation of the risks. In fact, it was — in my view at least — mostly banal and common-sensical, noting for instance that using the word “isolation” might have “negative overtones for older adults”, and that there might be unintended consequences if we closed schools.

Suggested reading

Nineties Russia prepared me for coronageddon

By Daniel Kalder

Other government behavioural advisors, though, were less circumspect. The Behavioural Insights Team, a consulting company nicknamed the “Nudge Unit”, has been brought in to assist the UK’s response. At first it seemed their focus was on how to encourage handwashing, but there appears to have been mission creep.

For example, the team’s head, David Halpern, was interviewed as a “government coronavirus science advisor” about broad policies of “cocooning” older people. He was also quoted in support of the idea — which might yet seem grievously misguided in hindsight — that social-distancing measures should only be brought in gradually, to avoid people becoming fatigued. After he was reportedly “bollocked” by No.10 a fortnight ago for introducing the unfortunate phrase “herd immunity” into the national conversation, Halpern hasn’t (to my knowledge) been heard from again in public.

And perhaps that’s for the best. As intriguing as many psychological studies are, the vast majority of the insights we’ve gained from our research are simply not ready for primetime — especially in the case of a worldwide emergency where millions of lives are at stake. Much of the useful advice behavioural scientists can give isn’t really based on “science” to any important degree, and is intuitive and obvious.

Where they try to be counter-intuitive — for instance, arguing that people are wrong to find a global pandemic frightening — they simply end up embarrassing themselves, or worse, endangering people by having them make fewer pandemic preparations. This isn’t to say that psychology isn’t useful when it stays in its own lane: it’ll be important to ensure that as many people as possible have access to psychotherapy for the mental-health effects of the pandemic, for instance. But that’s a secondary effect of the virus: my argument here is that psychology can give little reliable counsel about our immediate reaction to the pandemic.

Psychologists should know their limits, and avoid over-stretching results from their small-scale studies to new, dissimilar situations. Decision-makers should, before using psychology research as the basis for policy, know just how weak and contentious so much of it is. And everyone else should stay at home, wash their hands — and beware psychologists bearing advice.

At the absolute heart of our current political and cultural turmoil is the issue of authority. Epistemic authority, if you want to be fancy about it: the difficulty of knowing what’s true, and who we can trust to tell us the truth. This ties together the widespread mistrust of “elites” and the “mainstream media”, the fear about the way in which data can be used to manipulate us, the notorious rejection of “experts” and the universal scourge of “fake news”.

So if you were looking for a book that touches on this issue, but does so in an original enough way to mark it out from its hordes of competitors – and an enjoyable enough way to be read on a sun-lounger – you could do a lot worse than Michael Blastland’s thought-provoking The Hidden Half: How The World Conceals Its Secrets.

More from this author

What's the point of political fiction?

By Sam Leith

There are scores of books – from Daniel Kahneman’s Thinking Fast and Slow to Hans Rosling’s Factfulness – that investigate the cognitive biases and blind spots in the human brain, which form the basis for behavioural economics and the dark psychological arts of political campaigning. We’ve all now read a lot about the recency effect, availability biases, confirmation bias, anchoring and all the rest of it; we’re quite happy to trust the idea that our own minds are untrustworthy.

Blastland’s approach differs a little bit. Instead of concentrating on how our brains lead us astray, he looks instead at how much more mysterious and unknowable the outside world actually is – even were we able to look at it straight.

And, most winningly of all, he does so with special reference to a species of parthenogenetic crayfish.

Suggested reading

Do leaders bluff their way to the top?

By Peter Franklin

Here’s a thing. In 1995 in an aquarium in Germany, this crayfish called the marmokrebs appeared out of nowhere. It hadn’t teleported in: it had come into being through (we assume) a random mutation. And the first thing scientists noticed about this new species was that there weren’t any males. Every marmokrebs was genetically identical: a clone of a single ancestor, the marmokrebs Eve.

This offered scientists a fantastic opportunity to disentangle nature and nurture. If members of the species are all genetically identical, any differences must – as anybody would happily concede – down to environmental influences, right? So experiments were made in which marmokrebs were raised in utterly identical controlled environments – same water pH, same food, same light, same temperature, the whole works.

And here’s the marmalade-dropper: these genetically identical, identically raised crayfish were wildly, wildly different from each other. They varied in size by as much as twenty times; they had different numbers of mouthparts and wholly different markings; they reproduced at different times; they exhibited entirely individual behaviours and preferences; they showed preferences for different foods.

In practically every respect, they were as different as could be.

Suggested reading

The latest front in the gender wars

By Tom Chivers

So what caused these differences if not genetics or environment? Answer: we don’t know. And most laypeople – myself included, before I’d read Blastland’s book – didn’t even know we didn’t know. You, like me, probably thought that the argument in science was between genes and environment; not between genes and environment and… this other thing. Yet this other thing – this hidden half, called “enigmatic variation” – doesn’t just apply to crayfish. As much as half of human variation can’t be accounted for, writes Blastland, by either genetic or environmental factors.

The story is a caution against the idea that such knowledge as we have is a stable or reliable thing. And from there, Blastland roams across vast areas of inquiry, showing us how little we actually know.

You all know by now, for instance, that economic forecasting isn’t hugely reliable; perhaps it seems obvious that that’s in the nature of the thing. Animal spirits, irrational exuberance and all that, right?

But economic reporting, it turns out, is just as dodgy. Not only do we not know what’s going to happen, we don’t know what did happen. ONS figures for the economy two or three years ago continue to be revised in light of what has followed – and are often subject to confidence margins that can make the difference between a boom and a recession (Blastland cites one where a fall in unemployment of 3,000 was sombrely reported with a confidence margin of +/-77,000 – i.e. the figure could be a rise of 74,000 rather than a fall of 3,000).

And then there’s the “replication crisis” in the social sciences, where results on which whole subsequent fields of research have been built turn out to be, literally, junk science. Again, as many as half of the accepted results in the whole of social science or medicine are feared to be unreliable or plain wrong. The experiments simply don’t replicate. Even medicines that we know work may only work for a tiny percentage of patients – and we can’t predict which ones and we don’t know why.

More from this series

How to end Westminster's tribal warfare

By Polly Mackenzie

Blastland’s book is invigorating, though, because it’s not making the case for disregarding all our knowledge about the world, rejecting any sort of expertise or academic data and going with our tribal instincts. Rather, he argues that we need to be humbler in our approach to what we do and don’t know and the methods we use to arrive at our conclusions.

Politicians, scientists and reporters should put aside the blustering pretence of absolute confidence and – by sharing their doubt and caution – become more trustworthy. The first step to understanding the world better is acknowledging the limits of our current understanding. To be able to say: it’s more complicated than it looks. The wise man, as it’s sometimes said, knows how little he knows.