Is Instagram really bad for teenagers? The quality of the company's secret research into mental health is abysmal

Compare that to what’s in the “Facebook Files” (which, by the way, include leaks about several other topics, like the effect of social media on political debate, democracy, and human trafficking). The only evidence we’ve seen thus far is from surveys — teens filling in questionnaires about the effects they thought Instagram had on their wellbeing and mental health. The Journal also mentions larger studies, which link measurements of time spent on Instagram with survey responses, but no details are provided.

So, it would be as if Big Tobacco had sent a questionnaire to smokers, asking whether or not they’d gotten lung cancer, and whether they thought it was down to their smoking habit. If that was all we had, it would hardly be worth writing home about (or, indeed, in the Wall Street Journal): surveys might be an interesting spark for further research, but they’re a million miles from conclusive evidence.

“Correlation is not causation” is simultaneously the most hackneyed phrase and overlooked rule in science: although everyone’s heard it, people writing about studies — journalists and even the scientists themselves — routinely portray purely correlational or observational research as if it was an experiment testing the effect of one thing on another. This is the source of exciting headlines about how being optimistic makes you live longer, or that being more materialistic makes you depressed, or that drinking diet fizzy drinks makes it harder to lose weight — but no useful insight into the causes of health issues. In the Instagram case, the fundamental problem is that we don’t know whether more time spent on Instagram made the teen’s mental health worse, or — an equally-plausible opposite hypothesis — whether teens with worsening mental health for other reasons tended to spend more time on Instagram.

Asking people to introspect on the causes of their own mental health is hardly a reliable way of getting to the truth, given how much is going on in any one person’s life that might positively or negatively affect their wellbeing. Those who identify Instagram as the source of their sorrow could simply be mistaken — as could the vast majority of survey respondents, who said that Instagram either made them happier or had no effect on their mental health. So the headline splash of “Facebook Knows Instagram Is Toxic For Teen Girls” should, at the very least, have been: “Facebook Thinks Instagram Is Toxic For Some Teen Girls” — with a subheading: “On the Basis of Asking Those Teen Girls For Their Opinion”.

Perhaps the fury sparked by the Journal’s story would’ve been more justified if the research we do have in the public domain conclusively suggested Instagram — or social media in general — really is causing harm. This was certainly the case for tobacco: even without Big Tobacco’s covered-up internal research, there was plenty of convincing evidence available in the scientific literature to show the harms of smoking, even before the 1960s. A classic paper from 1959 (reprinted fifty years later) shows just how far researchers had gotten: they used different lines of converging evidence — population surveys, lab research, chemical studies — to demonstrate that smoking, beyond a shadow of a doubt, caused cancer.

But, in contrast, the research on the effects of social media on wellbeing is radically more uncertain. The psychologist Jonathan Haidt and the aforementioned Jean Twenge — both strong proponents of the social-media-causes-harm theory — have put together a useful document summarising the available evidence on the question. And that evidence is something of a mess: an array of correlational studies of widely-varying quality — but all suffering from the same shortcomings as Facebook’s survey research — sits alongside a set of experimental intervention studies, where researchers have, for example, had randomly-chosen participants limit their social media usage and observed the effect on their mental health.

That latter kind of research has mixed results: Haidt and Twenge note that eight of thirteen studies showed a statistically significant improvement in mental health after social media use was limited. But science isn’t democratic: we don’t count studies like votes, since not all studies are equal in their quality — or even necessarily comparable. In this case, some were in undergraduate students and some were in a wider adult age range (few, if any, included teens of the age that people are now concerned about); they used a variety of different methodologies; and looked at a variety of different social media sites.

Of course, the question of whether Instagram (or social media in general) makes teens unhappy is a lot harder to answer than whether smoking causes lung cancer. The effect of smoking is notoriously massive: smokers are by some estimates between 15 and 30 times more likely to get lung cancer than non-smokers (all research is like looking for a needle in a haystack, but the smoking-cancer researchers were looking for a needle the size of the one on top of the Empire State Building).

The effect of social media, for the average user at least, is likely to be far, far smaller. That doesn’t mean it’s not important, but it does mean it’s much harder for researchers to find. Those testing for subtle effects on mental health have the unenviable task of teasing apart an incredible amount of noise — people’s general mental health; the daily fluctuations in their mood; how we even reliably measure happiness in the first place — to find what might only be a modest signal.

But Big Tech ought to see the current furore — the backlash against them keeping their research secret — as an opportunity to do some better science. They’re in control of the platforms: they could run all sorts of useful randomised experiments on social media use, with huge samples of willing participants, who they could follow-up across months or years to assess their mental health. Given the potential conflicts of interest involved, such research would have to be ultra-transparent, and would preferably be done in collaboration with neutral, non-industry-linked academics. But of course, who’s to say that such research hasn’t already been done covertly by Facebook and others, and just hasn’t yet been leaked to the Wall Street Journal?

And that, of course, is the real issue: we are suspicious of Big Tech — and, in many ways, rightly so. It was the industry’s secrecy, not the quality of its evidence, that drove this particular story. The fact that Facebook have been hiding something from us has made commentators overlook the obvious limitations of the company’s research — Haidt and Twenge, for example, updated their document with the “Breaking news” that the Facebook survey had been leaked, despite knowing that survey studies are worth relatively little. Still, drawing comparisons to Big Tobacco’s conspiracies is ultimately facile, even if it gets the blood pumping.

Nobody comes out of this story looking good: not the secretive social media giants, nor the journalists exaggerating harms for attention, nor the scientists who’ve generally failed to provide us with a good evidence base. If they could all work together, of course, we might get some answers; but given their radically divergent motives, we might be asking the same questions for a while.

---

Stuart Ritchie is a psychologist and a Lecturer in the Social, Genetic and Developmental Psychiatry Centre at King’s College London

StuartJRitchie