The issue isn’t exactly black and white either, with some arguing that the provision of a safe space for struggling people is a necessary step towards combatting the feelings of alienation inherent within eating disorders and even the recovery which follows. The problem with pro-ED content in the social media era, however, is that while these communities used to be self-contained in forums, new dangers emerge when outsiders are exposed to it on their TikTok For You page or X (formerly Twitter) timeline through algorithms. This includes women and girls who are susceptible to self-harm themselves, but also predators who either see an opportunity to take advantage of vulnerable young people or just have an anorexia fetish.

Another moderation issue with pro-ED content is that in the 22 years since Oprah Winfrey introduced the mainstream to “pro-anorexia” content on her talk show in 2001, the culture surrounding it has been increasingly normalised outside eating disorder communities. But since the advent of social media it has gone into overdrive: it is now common for adults and adolescents to make jokes in favour of anorexia online, partially as a reaction to what they see as the ugliness and oppressive nature of “body positivity”. Alluding to having an eating disorder, both in images and in text, is practically a mainstay of being an e-girl. 

But where does one draw the line between posting aspirational images of ultra-thin, bikini-clad supermodels and “thinspiration” that breaks the terms of service? And at what point should users be free to make decisions about how they conceive of and talk about their own bodies, as well as the bodies of others? There’s a certain impossibility to moderating human-generated ED content, something that might be reflected in AI-generated material, too. 

In general, AI is used in sometimes impressive, sometimes downright disturbing ways in (often youth-dominated) digital subcultures. In the true crime community, in which fandom for murderers and mass shooters thrives, people were using character.ai, an AI chatbot, to simulate conversations with school shooters like Eric Harris and Adam Lanza, and murderers like Jeffrey Dahmer. 

In another kind of true crime community, there was recently a debate around TikToks that featured deep-faked murder victims explaining the story of their deaths “from their perspective”. In the Stranger Things fandom, at least one Tumblr user created AI-generated voice recordings featuring characters from the show so that they could role-play sexual encounters. The latter might initially read as quirky until you remember Stranger Things is a TV programme about children and high school students. 

In each of these situations, AI is only amplifying a morally ambiguous and existing cultural norm or behaviour. The problem, though, is what happens if it’s making it worse. That is something with which we are going to have to come to terms before we know it.

---

Katherine Dee is a writer. To read more of her work, visit defaultfriend.substack.com.

default_friend

---