How we made a monster

It wasn’t just Harry Potter fanfic. Sarah Ditum at the New Statesman writes about how knitting blogs at around the same time were full of people competing to have the most tragic backstory, not all of which was necessarily true. When I worked at the Telegraph many years ago, I helped edit our blog site, and the comments had more than their share of really nasty racists. The old blogosphere was not a prelapsarian paradise.

But communities have scaled up. Twitter and Facebook and Tumblr each have hundreds of millions of users, and it only takes a fraction of them to be awful to poison the stream. There’s this rule of the internet, the 1% rule, which states that almost all of the content is produced by a hyperactive hundredth of the community. We saw this at Telegraph Blogs: a post might be read by tens of thousands of people, but you’d see the same 50 or 100 names commenting. And, of course, the ones most likely to comment the most were the angriest, the most politically extreme, the maddest.

That’s manageable in small communities. You’d maybe find one or two really awful people on each of the small islands of the old internet. They were there, but when you know everyone, it’s easier for mods to ban bad actors and look out for their return under a different name. On Twitter, though, you can spend all day blocking and muting (I know, I’ve tried) and you’ll never get rid of them all. If 99.9% of people are lovely and 0.1% are awful racists and liars, then in a community of a million then you’ve got 10,000 racists and liars. And they’ll tend to be the loudest and most prolific.

More than this: the internet selects for the mad people. So do the traditional media, of course. But it’s much more direct on social media. The whole incentive structure of social media rewards bullshit and partisanship and tribalism. Exciting, interesting stories get upvoted or retweeted or liked, and it’s much easier to come up with exciting, interesting stories if you’re not constrained by whether or not they actually happened.

More subtly, controversy is incentivised. Stuff that everyone agrees with, people read, say “obviously”, and move on. Stuff that’s controversial gets one lot of people sharing it to signal that they are members of the tribe that believes that stuff, and the other lot hate-sharing it to show that they aren’t. You don’t get big fights on Twitter any more about gay marriage, because even Conservatives are in favour of gay marriage. Instead you get big fights about statements like “biological sex isn’t real” or “we should arm primary school teachers”, because a strident opinion on that marks out tribal allegiances much more clearly.

It’s an incentive structure that, when you have enormous communities as we do now, leads to Paul Joseph Watson and Novara Media.

To some extent, these big communities have made the small but open communities of the past impossible. Sure, they still exist. But they exist in the greater sea of Facebook and Twitter, and are exposed to the storms in them. My favourite blog, Slate Star Codex, has an associated page on Reddit, a subreddit. The nature of the blog is that it is a place for people to talk across political difference, and to take each other seriously and treat each other charitably. It is relatively evenly split in terms of political leanings. Inevitably enough, a lot of the conversation was about culture war topics – trans rights, abortion, race, gender, guns. Those topics were restricted to a “culture war thread”, so the rest of the subreddit wasn’t swamped by it.

Recently, the author had to ask the moderators of the subreddit to remove the culture war thread, because although the thread itself was always relatively polite and easy to moderate, it had become the subject of a campaign on Twitter and elsewhere, in which it was portrayed as hive of homophobic neo-Nazi hatred. The author was doxxed; people tried to get him fired from his job.

This is what happens when small communities – like a blog, or a subreddit – reaches the attention of the wider world. I’m sure 99.9% of the hundreds of thousands of people who became aware of it through social media were perfectly nice and would never try to get anyone fired at all. But it only takes two or three people trying to do that before things become seriously unpleasant.

So what happens next? Sir Tim Berners-Lee is trying to put together a “contract for the Web”, where everyone – companies, governments, citizens – agrees to abide by certain rules, keeping it free and civil. I wish him all the best with it, and hope it works, but I suspect that even if a voluntary agreement works for 99.9% of people, it still (as we’ve seen) only takes a tiny few to break the trust, to defect in the prisoner’s dilemma, to cause chaos.

Another thing would be to break the quasi-monopolistic social media companies up. The US Democratic Party wants to do that. But the plan, as I understand it, is to do separate out the companies within them: break Facebook apart from Instagram, Amazon from Whole Foods. It won’t be to make the actual Facebook social media community smaller. How would that even work, anyway? They smash it down the middle, and you suddenly find that you’re no longer Facebook friends with your aunt June because she’s been shunted over into the rival BookFace?

We could set up new rival social media sites! New, better ones! But that won’t work, because the reason we’re all on Facebook is not because Facebook is objectively the best website, it’s because everyone’s already there. If Google can’t make a new one work, you can’t. (That’s why those “Right-wing alternatives to Twitter” won’t work.)

Realistically, the mega-communities are – I think – here to stay. First, it’s worth saying that’s not necessarily a bad thing. There are obvious downsides, but people are at least partly on these sites because they make it easier to stay in touch, to make friends, to find work. (I’m a journalist, so my experience is obviously unusual, but I have landed two actual jobs, scores of commissions, and countless contacts through Twitter.) They’re not prisons.

And second, I am hopeful that, as the technology becomes more mature, we will become more used to it. The old story about Orson Welles’s War of the Worlds radio broadcast causing mass panic appears to have been largely a myth, sadly. But we are more susceptible to information reaching us via new technologies: look at propaganda ahead of World War One, which would be laughable now. The internet has convinced us that 30 years is a long time. It’s not. Give it another 50 or so, and we’ll probably have worked out how to use it.

---

Tom Chivers is a science writer. His second book, How to Read Numbers, is out now.

TomChivers