Too many famous findings in the field have turned out to be rubbish
Everyone’s heard of the marshmallow test, right? You put a child in a room with a marshmallow on a plate in front of them. And you say to that child, “You can have this marshmallow! Or you can have two, if you can wait until I get back.” Then you go away for 15 minutes or something.
There are two things to say about this. One is that it gives you lots of very cute videos of children trying not to eat a marshmallow. And the other is that a famous study found that children who were able to wait 15 minutes and get the second marshmallow would, over the decades to come, do better in life on various metrics than the ones who didn’t. They earned more, they did better at school, they had fewer behavioural problems, they tended to be slimmer.
This is one of the most famous results in psychology — one of those ones, like the Dunning-Kruger effect, that is part of popular culture. It’s fair to say that Cookie Monster and Tom Hiddlestone have never been asked to make a video teaching children about the affect heuristic or psychological anchoring. It’s even been in school curricula. I may even have mentioned it myself.
So it will not surprise you to learn that, like almost every other well-known psychological finding from before about 2011, it probably isn’t real. There have been concerns for a while — a study in 2018 found a much smaller effect which disappeared if you controlled for a few basic things, such as the child’s intelligence and social class. (Vox did a good writeup here.) But recently a re-analysis of the original study’s data, looking at the same now-middle-aged subjects of the original 1960s work, found no correlation between whether they resisted the marshmallow and all the outcomes — BMI, behaviour problems, earnings, etc — that have been linked to it. (Again, there’s a longer-form writeup here.)
There are so many things like this that I have started to lose count. The idea that forcing people to smile makes them happier? Didn’t replicate. Standing in “power poses” makes you more confident? Didn’t replicate. “Priming” people with the concept of money makes them act more selfishly? Honnnnnnnnk. The Stanford Prison Experiment: nope. Entire subfields of psychology have become suspect. Growth mindset, violent video games causing aggression, positive psychology, stereotype threat, implicit bias: so much of it is either shaky or false.
It’s all good news, in a way. In 2011, psychology took three big hits: one, a paper called Feeling the Future was published, which used statistical techniques entirely standard within the discipline to apparently show that people could predict the future. Two, another paper deliberately used similar techniques to apparently show that listening to the Beatles literally made you younger. (I wrote about it here.) And a well-known social psychologist, Diederik Stapel, was found to have been fabricating data in many of his widely cited studies. Scientists realised that their house was built on sand.
Since then, psychology (and science in general) has started to clean up its act: to impose much better statistical and data-gathering hygiene, so this sort of nonsense becomes harder. The “replication crisis” is really a story of a scientific discipline becoming actually disciplined.
But still, this is a reminder. Famous psychological experiments with headline-worthy findings have a worrying tendency to be garbage. Often they feel truthy — they feel like they’re explaining something deep — but all too often they haven’t stood up to scrutiny.