A scientist, searching for her blind spot. (Illustration by GraphicaArtis/Getty Images)
![](https://unherd.com/wp-content/uploads/2020/03/ritchie-100x100.png)
You know the old joke: a man goes to the doctor and is told he only has a month to live.
“Surely not!” he gasps. “I want a second opinion!”
“Alright then,” says the doctor. “You’re hideously ugly, too.”
The misunderstanding arises because the doctor is arrogant enough to think her patient trusts her as an expert on multiple issues, when the patient was, in fact, worried about error. The doctor might have seen a positive test result for a killer disease and taken it at face value, without considering that the disease is vanishingly rare, so the test result was likely a false positive. That is, the patient might be concerned that the doctor’s judgement — because of her failure to consider the “base rate” of the disease — might have been subject to bias. In other words, skewed in a specific direction.
Alternatively, maybe the patient was concerned that the doctor had carelessly misread the test results, or even read those of a different patient. Another doctor, even one of similar skill, would be unlikely to make the exact same mistake, hence the request for a second opinion. So rather than bias, the patient might have been worried about noise: the tendency for human judgments to vary in unwanted, unpredictable and arbitrary ways.
The first type of error, bias, is well-known, thanks to the work of Daniel Kahneman, who is among the most famous psychologists in the world. As he chronicled in his mega-blockbuster popular-science book Thinking, Fast and Slow, Kahneman spent decades with his colleague Amos Tversky cataloguing all the ways human thinking can go off the rails: not just the “base rate neglect” that we saw above, but all sorts of other biases. These include “anchoring” — best explained by the sales move where a shop gives an item a super-high price and then gives you 50% off, though 50% of the price is higher than you’d have paid for it if you’d never seen that initial value. There’s also “framing”, where asking a question in different ways can affect people’s answers (would you choose to have surgery that has a “10% death rate”? What if I told you it had a “90% survival rate”?). For these and many other contributions, Kahneman remains the only psychologist ever to have won a Nobel Prize, in 2002.
There was, however, a certain irony in Thinking, Fast and Slow. Whereas the biases and heuristics that Kahneman identified have been borne out extremely well by subsequent studies, a good chunk of the rest of the book, where Kahneman talked about other scientists’ work, hasn’t. For instance, Kahneman devotes a chapter to a certain kind of social psychology study where barely noticeable “priming” stimuli are shown to participants in lab studies, with the intention of changing their behaviour. For example, one set of researchers claimed that showing people a screensaver with banknotes on it made them less likely to want to help a struggling student — because it “primed” the idea of money, and thus selfishness, in their minds.
Long story short: those studies were weak, and other scientists can’t find similar results when they try to re-run the experiments. There’s plenty of evidence for priming in language — people react faster when asked to decide which of “CHAIR” and “CHIAR” is a real word if they’ve just seen the word “TABLE”, compared to if they’ve just seen a word unrelated to furniture. But the type of priming study where a barely noticeable prime makes major, measurable changes to people’s subsequent actions? Not so much. And yet, here’s how Kahneman, in Thinking, Fast and Slow, summarised his views on that kind of priming research:
Join the discussion
Join like minded readers that support our journalism by becoming a paid subscriber
To join the discussion in the comments, become a paid subscriber.
Join like minded readers that support our journalism, read unlimited articles and enjoy other subscriber-only benefits.
SubscribeVery often a book could have been written as an essay. The central Idea needn’t be stretched into a long and tedious book because the Idea of a book is more attractive.
Yes, I have said this for some years and have read countless such books, including Thinking Fast And Slow. A couple of notable exceptions, which I would recommend to anyone, are Paper Money Collapse by Detlev Schlichter, and The Master And His Emissary by Iain McGilchrist.
How about ‘The Creature from Jekyll Island: A Second Look at the Federal Reserve’ about the global money elites running the world? Have not read it yet, but sounds great – and I know you read a very great deal Fraser. Unherd is too dainty aesthetically to ever discuss money, so I ask the BTL guys…
“Where does money come from? Where does it go? Who makes it? The money magicians’ secrets are unveiled. We get a close look at their mirrors and smoke machines, their pulleys, cogs, and wheels that create the grand illusion called money. A dry and boring subject? Just wait! You’ll be hooked in five minutes. Reads like a detective story – which it really is. But it’s all true. This book is about the most blatant scam of all history. It’s all here: the cause of wars, boom-bust cycles, inflation, depression, prosperity. Creature from Jekyll Island is a “must read.” Your world view will definitely change. You’ll never trust a politician again – or a banker.”
Anyone who “trusts” a politician or a banker deserves the screwing that will result.
Thank you for the suggested reading. I remain curious and have read on both subjects but was unaware of those authors.
The Bible being a good example.
Well, we do always note the brevity of your posts, not even achieving a sentence on this one.
Why waste words?
Why Not?
It’s not a book. It’s a collection of books: τὰ βιβλία. You’re not expected to read it all at once (or even ever).
Christianity is a desert weed that doesn’t flourish in a lush environment.
As a former publisher of professional/business books, the explanation is simple: only by stretching a small (even if worthy) idea into a book can any money be made.
Maybe Mr Kahneman has his own bias: he believes that humans are rubbish at thinking and reasoning, and is happy to accept any result that confirms his opinion.
When first introduced to his ideas, in a company training course, I was convinced that management was deliberately trying to undermine our trust in our own judgement, in order to make us more obedient to management orders, stupid or otherwise. Crucially, nothing was said about how to identify or correct for bias, only that we are riddled with it. Trying to finish ‘Thinking fast and slow’ later, I saw that the course was actually quite faithful to Kahneman’s ideas. As I remember, he thought that a problem like officer candidate selection could best be handled by removing all human judgement from the process and replacing it with something like a twelve-line Python program. He may or may not be right, but short of handing over the running of our lives to some primitive computer program, what can we use it for?
I have difficulty believing this – I actually upvoted you <G>
You could use it to become more self aware.
And where does peer pressure come into making judgments and even conclusions from observations? Is it the case that institutions are making ‘woke’ decisions because proponents are elevating ‘noise’ and they bow to peer pressure?
One could perhaps argue that this article is influenced by noise in changing the generic pronoun from male to female in the joke. Women don’t make those kinds of personal remarks to people: it could only ever be an insensitive bloke.
When men get together they insult each other and they don’t mean it. When women get together they pay each other compliments and they don’t mean it.
For me, the most interesting article of the year, which makes me a bit weird. One eminent psychologist criticising the work of another eminent psychologist in a way which the ordinary person can understand. Two problems for me in this:
As I have said before, I don’t really think of medics as scientists. The bottom line is that they do make a lot of mistakes, especially the experienced ones. In the USA there was an extensive trial of a computer-based diagnosis system which always beat the doctors but especially made the experienced ones look very bad – because medical ideas change very quickly and the medics don’t change fast enough as they get older.
The explanation of the statistics showed the one important thing – that we focus too much on averages. For example, I have read many times that a lockdown for Covid should not have happened because the average age of death was 82. Here, the average is not necessarily the correct figure to use – the way the average is made up can sometimes be more important.
So, a psychologist criticising a psychologist about the abuse of science – both of them use statistics in their work but neither is a scientist. Arguably, we could do with fewer psychologists and more scientists.
You mean like those “scientists” who created the Covid pandemic and have been furiously trying to cover their trail for the past 18 months?
Lab leak? What lab leak? I didn’t see a lab leak.
Lockdowns are good for you.
Here, have a mask.
Don’t understand. If you mean that all scientists should be treated with suspicion and all findings should be taken with a pinch of salt, I suggest that you should suspect all forms of transport as unsafe, that all bridges are about to collapse, that phones don’t really work (just an evil government trick).
You can only really not believe in science if you are prepared to go back to nature. You should stay in comfort behind your computer and make clever comments. Oh, yes – a computer, another evil trick.
It depends surely. There’s science and there’s science.
There’s practical science and there’s prediction / woo-woo lent spurious authority by claiming that it’s “science”.
Phones and computers objectively do work, and bridges usually don’t fall down.
Models of spacecraft trajectory work pretty well. Models of car crash risk also work pretty well. It’s no coincidence that the science in these – the physics, the statistics – produce replicable results.
No problem with the science in those. Purportedly predictive models of the likelihood of a global financial crash risk, however, or COVID fatality, objectively don’t work at all. Climate science and psychology likewise. They don’t become science just because they happen to use an Excel spreadsheet overlay on top of some political assumptions that fundamentally aren’t science.
The challenge is in how you recognise them from each other and respond.
Climate science and COVID fatality prediction are actually decent,even if they are not quite as good as weather forecasting. The system is too complex to understand fully, but at least the main mechanisms are understood. It is just that a lot of people do not like their conclusions. Finance and psychology are a lot worse off.
Anywhere that the claim “the science is complete” is made is not science. Climate science in terms of forecasting is about as accurate as a 1960s weather forecast for the month after next.
COVID fatality prediction – OK I’ll bite – how do you know?
It is a question of what you have to work with. Pure curve-fitting without understanding is unlikely to work – which is why you cannot extrapolate your way to future stock prices. Economic predictions and psychology have no well-understood laws to make a secure basis for anything. With climate or COVID we have enough of a basis that our predictions should at least bear some reasonable relationship to reality. For more precision that that you would need detailed knowledge.
On the climate we know that CO2 in the atmosphere has increased steadily, we know that the first-order effect is to heat up the planet, we know at least a fair bit about how the different parts of the system interact, and our data do say the planet is heating up and the glaciers are melting faster. If our models fit the data and predict that further increasing CO2 will further increase temperatures that does sound at least plausible.
With COVID we know (or are getting to know) how the disease spreads, how long you are infective, the proportion of deaths in each age group. We cannot really model the social interactions or population behaviour, but again, we have a handle on what could happen.
“We have to get rid of the mediaeval warm period”
https://scienceblogs.com/stoat/2012/10/17/1943#:~:text=In%201995%2C%20Deming%20had%20concluded,research%20on%20ancient%20tree%20rings.&text=his%20guard%20down.-,A%20major%20person%20working%20in%20the%20area%20of%20climate%20change,the%20Medieval%20Warm%20Period.%E2%80%9D‘
A long, complex and contradictory discussion on which unnamed climate scientist may have made an exaggerated throwaway remark, and why that would mean that all scientific conclusions on the future climate can be ignored. Rather like saying that Boris Johnsons (alleged) comment on ‘Let the bodies pile up’ should be proof that lockdowns work. I am just not buying it.
Wade’s article on the lab leak was convincing because it addressed the actual arguments, not the bad faith of the people he disagreed with, and gave a convincing alternative explanation for the available facts. If there is someone who can do the same for climate change it will be worth the time it takes to read it. At least it would hopefully tell us what these people believe is happening, and why, not just why everybody else should not be trusted.
The irony is strong in this one. 🙂
Weather forecasts are only really reliable about 3 days into the future, and they don’t even try to do forecasts more than 10 days into the future (outside of climatology).
So what you’re saying is that epidemiology is a bit worse than something that is only reliable 3 days into the future. Yes, that sounds about right. So why do they try to predict trends over a span of months or years?
Because they are predicting trends, not individual events, which is a lot easier. And because that is the question we need an answer for.
This is an academic distinction – a “trend” in epidemiology is started by an “event”, because they don’t try to predict when outbreaks start at all.
Also, it’s clearly not easier because their predictions are always either wrong, or useless, or both!
“50 million climate refugees by 2010”?
https://www.theguardian.com/environment/2005/oct/12/naturaldisasters.climatechange1
Or in other words it is time to:
“Prepare to repel boarders!”
Yes, the Guardian wrote an exaggerated warning about the possible future in 2005. Again, try telling us what you think is happening and why, that would be a lot more constructive.
I would suggest that people who traffic in absolutes should be viewed with suspicion. We were told unequivocally that lockdowns and mandates and masking were the answers, and that to question our betters was tantamount to heresy. How’d that work out? I have a lot more respect for a person with the humility to say “I don’t know” or “I’m not sure” when one of those is the real answer. The others are not practicing science; they’re resorting to scientism with a public all too happy to parrot them with an appeal to authority.
You are spot on about medical mistakes; they’re among the top causes of death in the US. Turns out doctors are not gods, after all, just men and women with all the inherent flaws that come of being human. Seems having biases is part of that, too.
Were we told ‘unequivocally that lockdowns and mandates and masking were the answers’? I thought they were proposed as the best possible guess as to what to do based on the information available and an assumed priorities of minimising imminent deaths or illness.
Those things were not “proposed.” They were mandated. No business closed its doors voluntarily. In many jurisdictions, masks were not suggestions.
Had various officials just said “we recommend” and let adults make their own adult decisions, that would be quite a different affair.
I see what you mean. Yes, in some places they were made laws. I suppose my point was that the arguments behind the laws wasn’t presented (to my ears, anyway) as unequivocal fact but the best available option. Making them compulsory? If some UK businesses had not been required to shut down then the UK furlough scheme would not have been plausible.
As has been gone over many times, letting adults make their own decisions as to whether they do stuff to protect other people doesn’t necessarily lead to those adults making the decisions that protect other people.
That attitude helps explain why the US did such a poor job of handling the pandemic.
They were mere political manipulation. Lockdowns and masks were not to save anything, but for nefarious political agenda. Take 50 USA States, look at WHO was in charge of each, and what they made their state do, that was the thing Who, not What, not the covid. Left never waste a good crisis.
For me the problem is that far to many “ologies” have been declared science. If you cannot falsify and repeat then its not science. Most of the soft sciences should disappear in a mushroom shaed cloud.
If medical ideas change very quickly, how do you know that today’s ideas are the correct ones? It is not surprising that a program recently written to follow the most modern ideas, well, conforms to the most modern ideas. But unless you have some independent confirmation it does sound a bit like it is marking its own homework.
Yes but you have to ask, “So what?” My point is that medical science comes from scientists, people who are trained scientifically, who use experimentation to confirm or deny their findings.
By this definition, medics (medical doctors) are not scientists – they merely do as they are told. Psychologists have almost no scientific training but they do perform experiments, often badly.
Medical doctors are taught to observe, to count symptoms, how to react when those symptoms occur. They do this by rote. People believe that they get better with age because of experience. In fact, they get fixed in their ways and don’t change as quickly as they should. Therefore, they make a lot of mistakes in diagnosis, in emphasis and in bedside manner.
There was a program in the last year or 2 on do you give oxygen to a person having a heart attack . Over about 3 decades 3 different studies had shown that patient outcomes were worse when oxygen was given. They were trying to get the message across to health professionals.
One of the big learnings for the public in the past year is that medical science doesn’t actually come from generic “scientists” at all, it comes from a grab-bag of pharma firms (who seem to be competent) and academics pretending to be scientists (who don’t). Any type of “science” that makes predictions but never validates them against the real world is not actually a science at all, and medicine is absolutely filled with that sort of thing. Nobody within the scientific institutions seems to notice or care, and they just flatly reject all outside criticism.
That’s before we get to the maths errors, the programming errors, the logic errors … frankly having read their output, it left me terrified of going into a hospital. The health literature is flooded with papers that look superficially like what medical science is meant to look like, but then you scratch the surface and discover the whole thing is a movie set in which 80% of the buildings are actually just facades and everything falls down if you lean on it.
On the other hand, would you rather be treated for a serious illness in a 2021 UK hospital or a 1971 UK hospital, and who do you think did the research that makes 2021 (even under current pressures) the obvious choice?
Easy – its because if the patient is male the system says he wants to me female and have bits chopped off and other bits stuck on, if female she wants to be male and have some bits chopped off and other bits stuck on.
There you go – a fully modern medical diagnostics system.
My wife would agree with you about (most) doctors not being trained as scientist. She would also tell you that the Lancet is not a scientific journal but a magazine for doctors.
Averages are meaningless when the shape of the distribution is too skewed or worse yet multimodal. Anyone in the hard sciences understands that, but I don’t think medics, biologists, or people on the social sciences are very aware of it.
Interesting comment and I’ve never really thought about whether medics are scientists before but the more I think about it the more it rings true that they’re not. Making a diagnosis, prognosis, deciding on treatment uses some data produced by science but then lots of other stuff that is not science as well. And computer programs have been found to be better at reading scans than radiologists.
Maybe we need more statisticians and computer programmers and fewer psychologists and scientists. Though Psychology could then re-badge itself as a sort of interpretive art form – which wouldn’t necessarily make it any less valuable.
Chris, you are a very worthwhile poster to read, but all I got from the above article was gibberish arranged to sound like it had meaning. when I hear “psychologists” I think Astrologer, Phrenologists, Primal Scream Therapy, Freud, Shaman, and so on. I guess unconscious bias.
There is a very good reason that I am “biased” in favor of young doctors and young dentists from only top schools. Experienced members of professions are the most outdated and dangerous and the doctor who came in last at Grenada Medical School is still called Doctor. Continuing Education is a joke–I know, I provided it at a handsome profit for many years to financial advisors and CME (the medical version) is no better.
I’m one of those millions. I gave up not because it was a slog but because it was give up or start ranting about conclusions being drawn from minuscule studies being extrapolated to the whole of humanity.
Later I read Science Fictions by Stuart Ritchie which confirmed my bias.
I’m another one who didn’t finish Thinking Fast and Slow.
Not that I disagreed with it, but because it merely confirmed what a lifetime of experience and self-criticism has taught me. And then repeated it in each chapter.
The really intriguing question was: “Is this stuff really revelatory to academics? They must be even more out of touch than I thought” (with my inbuilt bias).
Humans have biases – News at 11.
Sounds like that may be an interesting read. I first came across the concept of cognitive bias in The Mechanism of Mind, published by Edward de Bono in 1969.
I have no idea whether his model of cognition is widely accepted or not. I suspect that rather fewer people than might be desirable understand the concept of a self-organising system, which is central to the model (as far as I understand it).
Long story short: those studies were weak, and other scientists can’t find similar results when they try to re-run the experiments.
As those who have looked into it know, there is a crisis of replication in the “scientific” world. Strip away the verbiage, and the bottom line is that there is a lot of politics in “science,” going back to Galileo and further. Whether religious (Galileo) or academic careerism, or today’s rush for university grants tied to media attention and public superstition, the scientific method, which depends on independent replication, is all too often observed in the breach.
My favorite example is at the link. This is a British site, so I do not expect readers here to be familiar with it. The example is very far from trivial, and I can cite other recent examples.
http://www.detectingdesign.com/harlenbretz.html
Hey Unherd, what’s with the censorship? Are you trying to become MSNBC or something?
Are you biased if you hate everyone and everything?
I think the sub-editor has inserted a very noisy title.