It is no exaggeration to say that soon almost everything we see or hear online will be synthetic — that is, generated or manipulated by AI. Machines that can “learn” to do almost anything when ‘trained’ on the right data — and they’ve never had access to more data, nor so much power to churn through it all. Some experts estimate that within 5-7 years, 90% of all video content online will be synthetic. Before long, anyone with a smartphone will be able to make Hollywood level AI-generated content.

One synthetic-text generating model can already generate articles that appear to have been written by a human. AI can be trained to clone someone’s voice even if they’re already dead: an old recording of JFK’s voice has been used to make a clip of the former president reading the Book of Genesis. AI trained on a dataset of human faces can generate convincing fake images of people who do not exist, and it can be taught to insert people into photographs and videos they were not originally in. One YouTuber is working on a project to insert actor Nicholas Cage into every movie ever made.

More from this author

Could Russian race-baiting tear the US apart?

By Nina Schick

All this sounds weird at worst and hilarious at best, but this technology has a dark side. It will, inevitably, be misused, and for that most obvious of male-driven reasons. It was reported last week that the messaging app Telegram is hosting a “deepfake pornography bot,” which allows users to generate images of naked women. According to the report, there are already over 100,000 such images being circulated in public Telegram channels; considering that they are almost certainly being shared privately too, the actual number in existence is likely much higher. The women who appear to feature in this cache of publicly-shared fake porn are mostly private individuals rather than celebrities. More disturbingly, the images also include deepfake nudes of underage girls.

Henry Ajder, the lead author of the report, told me that “the discovery of the bot and its surrounding ecosystem was a disturbing, yet sadly unsurprising, confirmation that the creation and sharing of malicious deepfakes is growing rapidly.” When he wrote to Telegram to request a takedown, he “received no response”. The bot and the fake pornography — including that of minors — is still live at time of writing.

Deepfake pornography first emerged less than three years ago, when an anonymous user started posting it on Reddit. He later revealed his methodology: he was using open-source AI software to insert celebrity faces into pornographic films, by training AI on images and videos of his intended target. The end result was surprisingly convincing. When other Redditors started making their own deepfake pornography and news of this AI-assisted community broke to the world, there was a furore: Reddit shut down the community and banned deepfake pornography. But the genie was out of the bottle.

Since its early days on Reddit, an entire deepfake pornography ecosystem has developed online. With a few clicks, it is possible to access deepfake pornography of every (female) celebrity imaginable, from Ivanka Trump and Michelle Obama to Ann Coulter. And celebrities are not the only targets. AI can clone any woman: all that is needed is some training data, and the rapid acceleration of the technology means that less and less training data is required. These days a single picture, a few seconds of a voice recording, or a single video would be enough.

Suggested reading

Just how harmful is porn?

By Tom Chivers

The tools to make synthetic media are becoming more and more accessible. Anyone can try to create their own by using free software; support communities online explain how to use it. It is even possible to buy “learn to deepfake” courses, or to commission a “deepfake artist” for a bespoke “creation” from little as $20. Recently,entrepreneurs have begun to wrap up the technology in easy-to-use app interfaces, so millions of consumers will be able to experiment with making their own AI-generated fake content.

One such app launched last year, calling itself “DeepNude”. It allowed users to upload photos of a woman to generate a deepfake image of her naked, and for a fee of $50, users could pay to remove a watermark so that the image would look authentic. (Because the underlying AI was developed on training data of female bodies, the app only worked on women.) When the DeepNude was released, the demand was so high that the app’s servers crashed under a stampede of downloads. Facing a barrage of negative press, its developers eventually took their creation offline, saying “the world is not ready for DeepNude app”. Weeks later, they quietly sold their software in an anonymous auction for $30,000.

While deepfake pornography ostensibly seems to be a “woman’s issue”, it provides an early and worrying case study for how synthetic media could be weaponised against us all. It is inevitable, for example, that deepfakes will be used as a potent new tool of identity theft and fraud.

Last week, federal prosecutors in the US revealed the case of a Californian widow who was scammed out of nearly $300,000 by an unidentified overseas con man who romanced her using deepfake videos in which he posed as the superintendent of the US Naval Academy. The widow, only identified as “M.M.”, thought she was building a relationship with an admiral named “Sean Buck”, who told her he was stationed on an aircraft carrier in the Middle East. They communicated for months via Skype, during which “Buck” always appeared dressed in his military uniform. According to the prosecutors, “While M.M. believed she was communicating with [Buck] via live chat on Skype, what she was seeing were actually manipulated [deepfake] clips of preexisting publicly-available video of the real Admiral Buck, and not the live video chats that M.M. believed them to be.”

Suggested reading

Why doesn't porn ever get cancelled?

By Sarah Ditum

The costs of identity theft and fraud are vast and on the rise. According to an annual report by Javelin Strategy & Research, identity fraud-related losses grew 15% in 2019 to $16.9 billion — in the US alone. This is largely due to the fact that financial institutions’ methods of identifying and responding to fraud are no match for criminals’ high-tech schemes to steal money, which increasingly incorporate deepfakes.

Of course, they will not only be utilised against consumers, but businesses too. The first serious case of deepfake business fraud emerged last year when the Wall Street Journal reported that a British energy company lost €250,000 when scammers used AI to clone the voice of the company’s CEO to demand (over a phone call) that the money transfer be made.

Libel, identity theft and fraud are nothing new — but the potency of such ventures will increase exponentially as synthetic media becomes prolific. Because of its unique ability to “clone”, AI presents a dire threat to an individual’s right to privacy and security. At the moment it is almost impossible to distinguish between authentic and synthetic media, and the quality of the latter is improving. This, then, is the alarming reality: a world in which our identities can be “hijacked” by almost anyone and used against us.

We have reached the critical moment to set standards — to create ethical and legal frameworks — to define how synthetic media should be created, labelled and identified. Given that the AI behind synthetic media is still nascent, we still have (a little) time to influence the effect this technology will have on our societies and the individuals within them. Too often we build exciting technology without considering how it might amplify the worst parts of human nature, or hand weapons to the cruellest impulses.

It soon transpired that Professor Segal was a retired 76-year-old biophysicist, living in the Soviet-controlled puppet state of East Germany. But by then it was too late. Critical faculties, as we know, can go out the window during pandemics — whisper ‘scientific evidence’, and frightened people will believe what you say.

Suggested reading

Are the conspiracy cranks a little bit right?

By Gavin Haynes

The Soviets knew this. As the AIDS epidemic worsened, the lie that the virus was created in a Pentagon laboratory proliferated in the Soviet press, which widely cited the Segal report. The Soviet news agencies TASS and RIA Novosti, which together had over 100 bureaus worldwide, amplified the story; it began to go viral, appearing in dozens of sympathetic or unsuspecting newspapers around the globe. By the end of the decade, it had appeared in major outlets in more than 80 countries. This disinformation campaign, codenamed ‘Operation Infektion,’ was the most successful of the Cold War.

Its legacy is felt to this day. The genocidal conspiracy theory is disproportionately believed in black American communities — to the extent that it hinders HIV prevention. One survey of black Americans found that 48% believed that AIDS was an artificially made virus, and 27% believed it was made in a government laboratory.

And although the Cold War is over, the old Soviet strategy of race-baiting as a form of information warfare against America lives on. Indeed, the techniques have only become more potent, as our modern information architecture — the Internet, smartphones and social media — makes it infinitely easier for bad actors to disseminate disinformation quickly and widely.

Suggested reading

America is losing the information war

By Nina Jankowicz

As the battle for America’s soul threatens to rip the country apart ahead of November’s election, race has become one of the key partisan battlegrounds. This is being exploited with devastating precision by the Russians — who’ve been refining their methods for years. As Special Counsel Robert Mueller found in his investigation into Russian interference in the 2016 US election, the Kremlin started laying the groundwork in 2013 — well before Donald Trump had even announced his candidacy. That year, it set up the Internet Research Agency (IRA) — a unique mash-up between a digital marketing and intelligence agency, whose task was to experiment with social media to fuel polarisation in American society. Race was one of the most fertile battlegrounds.

In mid-2014, IRA agents travelled to the United States in an “intelligence gathering mission” in order “to obtain information and photographs for use in their social media posts”. Then, from their base in St Petersburg, they built fake pages, communities and personas that looked authentically American. These included ostensibly Left-wing groups like ‘Black Matters’, which targeted black Americans by appearing close to ‘Black Lives Matter’. This disinformation project was known as ‘Operation Lakhta’.

A paid ad sponsoring the IRA’s ‘Black Matters’ page on Facebook

Paid ads publicising the group went straight for the jugular: this one features photos of three young black Americans — Michael Brown, Tamir Rice and Freddie Gray — who were killed either in police shootings or custody. Rice was only 12. He had allegedly been shot on sight for playing with a toy gun.

Once they had ensnared followers, and as the election grew closer, groups like ‘Black Matters’ became a vehicle for the IRA’s attempts to suppress the black Democrat Vote. Agents posted emotive narratives of political alienation, suggesting that Hillary Clinton did ‘not care for Blacks’, or that a race between Trump and Clinton was ‘one that Black people should sit out’.

Suggested reading

Britain’s race pain is not the same as America’s

By Ralph Leonard

But the IRA wasn’t just baiting black voters on the Left; it was also targeting white voters on the Right. Again, race was the trump card. Agents set up pro-police social media groups that promoted the so-called ‘Blues Lives Matter’ movement. In a sponsored ad for one of these pages called ‘Being Patriotic’, the IRA urged white conservative Facebook users to ‘like’ its page by pushing anti-BLM content. These ads were just as emotive as the ones aimed at black voters. The one pictured below, in the full text, attacks Hillary Clinton — warning that she is the ‘main hardliner against cops,’ and that Donald Trump is the ‘one and only who can defend the police from terrorists’.

‘Being Patriotic’ Promoted Page on Facebook

As the IRA’s operation grew, it found it could socially-engineer the real-life actions of American citizens — manipulating many, for instance, into attending political rallies organised and promoted from St Petersburg. Donald Trump’s campaign unwittingly boosted one such event on its Facebook page, but the IRA also ran anti-Trump rallies after he won the election. One, organised on the ‘Black Matters’ page, managed to convene up to 10,000 protestors in Manhattan’s Union Square, who then marched on Trump Tower, four days after the Republican victory. ‘Join us in the streets! Stop Trump and his bigoted agenda!’ said the Facebook event page for the rally. ‘Divided is the reason we just fell. We must unite despite our differences to stop HATE from ruling the land.’ The tragic irony, of course, being that division was exactly what the Russians were exploiting.

Suggested reading

Abolishing the police is a cop out

By Mary Harrington

The Democratic House Intelligence Committee has publicly released a cache of IRA social media advertisements from between 2015 and 2017, demonstrating how prolific they were. On Facebook alone, the IRA purchased 3,393 ads, which 11.4 million Americans were exposed to. They built 470 Facebook pages, and created 80,000 pieces of organic content, which reached over 126 million Americans.

It is difficult to quantify the extent of the IRA’s influence; it’s worth bearing in mind that the election that was decided by just 70,000 votes. It would be naïve to attribute Trump’s victory wholly to Russian interference, but it is nevertheless clear that the poison the IRA injected into US public discourse has played some part in making it increasingly febrile.

And now the 2020 election looms. In the wake of George Floyd’s death and global BLM protests, race has re-emerged as the critical dividing line. And Russia is better placed than ever to exploit it, with methods that grow more sophisticated with every passing month. Though the situation is developing as you read this, there are already some clues as to how their tactics are evolving.

Suggested reading

The liberals are falling into Putin's trap

By Peter Franklin

On 12 March 2020, CNN, Twitter, Facebook and the network analytics firm Graphika exposed another disinformation operation. While Project Lakhta was run by IRA agents in St Petersburg, by 2020 the work had been outsourced to Ghana, where the IRA was operating behind the front of a fake NGO called Eliminating Barriers to the Liberation of Africa. EBLA had office buildings, employees and a website, which at first glance looked legitimate — it referred to an array of ‘projects’ and had a big ‘donate’ button. On closer inspection, the façade crumbled. The donate button did not work, the site was populated with Latin-stock text, and one of its projects claimed to have raised $231.53 billion — over three times Ghana’s annual GDP​.

The IRA had employed Ghanaian operatives to entice black Americans to join online communities, then post positive messages that would encourage them to have pride in their racial identity. Agents would then start to feed these manufactured communities negative messages, making followers feel angry and isolated. The aim was to alienate black voters from the rest of society. The Ghanaian operatives were encouraged to play with content to see what would resonate and what would go viral, just as the IRA agents have done since 2013. EBLA employees had access to a library of memes and images — many simply recycled from the IRA’s stock, built up at the time of the last election.

It was not hard for these operatives to guess that they were targeting American audiences — particularly because they were told to post content on social media during the late afternoon and night in local Ghanaian time, to correspond with US daytime. Still, it would have been quite a leap for them to figure out that they were being unwittingly used as agents of a Russian disinformation campaign. This split-level subterfuge — practised on both Americans civilians and Ghanaian operatives — led the operation to be dubbed ‘Double Deceit.’ That it had been outsourced to African soil made it even harder to trace, and fight, the true perpetrators of this assault on America.

It’s reassuring to think that Operation Double Deceit was exposed, but it would be surprising if Russia doesn’t have other irons in the fire ahead of this year’s election. Perhaps more worrying, though, is that Russian disinformation against the United States has become a hyper-partisan political issue. With one side paranoid about Russia — seeing it everywhere — and the other denying that Russia is a threat at all, the Kremlin’s strategy is working beautifully.

While armed vigilantes and pro-and anti-BLM protestors were filling the streets of America earlier this year, these divisions were being weaponised by foreign actors; Americans, meanwhile, were too busy downplaying or magnifying the Russian threat for their own ends to organise any coherent response. Obviously, this does nothing to help race relations. In fact, extreme partisanship undermines the security of the entire nation, by blinding Americans to the real threats.

When the dust settles in the wake of the 2020 race, the US is likely find that, yet again, it was too slow to understand and react to foreign psychological operations.