Chat Control, formally known as the Child Sexual Abuse Regulation, was first proposed by the European Commission in 2022. The original plan would have made it mandatory for email and messenger providers to scan private, even encrypted, communications — with the purported aim of detecting child sexual abuse material. 

The tool was sold as a noble crusade against some of the world’s most horrific crimes. But critics argued that the tool risked becoming a blueprint for generalized surveillance, by essentially giving states and EU institutions the ability to scan every private message. Indeed, a public consultation preceding the proposal revealed that a majority of respondents opposed such obligations, with over 80% of explicitly rejecting its application to end-to-end encrypted communications.

Yet despite repeated blockages, and widespread criticism for violating privacy and fundamental rights, the text was never abandoned. Instead, it was repackaged, and continually pushed forward from one Council presidency to the next. Each time democratic resistance stopped the original plan, it kept returning in new forms, under new labels, each time dressed up as a “necessary” and “urgent” tool to protect children online, yet always preserving its core logic: normalizing government-mandated monitoring of private communications on an unprecedented scale. 

In May, the European Commission once again presented its proposal. Yet several states objected. That included Germany, but also Poland, Austria and the Netherlands. As a result, Denmark, which currently holds the rotating presidency of the European Council, immediately began drafting a new version, known as “Chat Control 2.0” and unveiled earlier this month, which removed the requirement for general monitoring of private chats; the searches would now remain formally voluntary for providers. All this happened under the auspices of Coreper, the Committee of Permanent Representatives — one of the most powerful, but least visible, institutions in the EU decision-making process. It is where most EU legislation is actually negotiated; if Coreper agrees on a legislative file, member states almost always rubber-stamp it. 

The gamble worked. Yesterday, this revised version was quietly greenlit by Coreper, essentially paving the way for the text’s adoption by the Council, possibly as early as December. As digital rights campaigner and former MEP Patrick Breyer put it, this maneuver amounts to “a deceptive sleight of hand” aimed at bypassing meaningful democratic debate and oversight. 

While the removal of mandatory on-device detection is an improvement on the first draft, the new text still contains two extremely problematic features. First, it encourages “voluntary” mass scanning by online platforms — a practice already allowed in “temporary” form, which would now become a lasting feature of EU law. Second, it effectively outlaws anonymous communication by introducing mandatory age-verification systems. 

Suggested reading

The EU’s new censorship machine

By Thomas Fazi

An open letter signed by 18 of Europe’s leading cybersecurity and privacy academics warned that the latest proposal poses “high risks to society without clear benefits for children”. The first, in their view, is the expansion of “voluntary” scanning, including automated text analysis using AI to identify ambiguous “grooming” behaviors. This approach, they argue, is deeply flawed. Current AI systems are incapable of properly distinguishing between innocent conversation and abusive behavior. As the experts explain, AI-driven grooming detection risks sweeping vast numbers of normal, private conversations into a dragnet, overwhelming investigators with false positives and exposing intimate communications to third parties. 

Breyer further emphasized this danger by noting that no AI can reliably distinguish between innocent flirtation, humorous sarcasm — and criminal grooming. He warned that this amounts to a form of digital witch-hunt, whereby the mere appearance of words like “love” or “meet” in a conversation between family members, partners or friends could trigger intrusive scrutiny. This is not child protection, Breyer has argued, but mass suspicion directed at the entire population. Even under the existing voluntary regime, German federal police warn that roughly half of all reports received are criminally irrelevant, representing tens of thousands of leaked legal chats annually. According the Swiss Federal Police, meanwhile, 80% of machine-reported content is not illegal. It might, for example, encompass harmless holiday photos showing nude children playing at a beach. The new text would expand these risks dramatically. 

Further concerns arise from Article 4 of the new compromise proposal, which requires providers to implement “all appropriate risk mitigation measures”. This clause could allow authorities to pressure encrypted messaging services to enable scanning, even if this undermines their core security model. In practice, this could mean requiring providers such as WhatsApp, Signal or Telegram to scan messages on users’ devices before encryption is applied. 

The Electronic Frontier Foundation has noted that this approach risks creating a permanent security infrastructure, one which could gradually become universal. Meta, Google and Microsoft already scan unencrypted content voluntarily; extending this practice to encrypted content would merely require technical changes. Moreover, what begins as a voluntary option can easily become compulsory in practice, as platforms face reputational, legal and market pressure to “cooperate” with the authorities. Furthermore, this doesn’t affect just people in the EU, but everyone around the world, including the United States. If platforms decide to stay in the EU, they would be forced to scan the conversations of everyone in the bloc. If you’re not in the EU, but you chat with someone who is, then your privacy is compromised too. 

Another major danger is the introduction of mandatory age-verification systems for app stores and private messaging services. Though the Council claims these systems can be designed to “preserve privacy”, critics insist that the very concept is technologically unworkable. Age assessments inevitably rely on biometric and behavioral data, both of which require invasive data collection. Far from protecting children, these systems would increase the volume of sensitive personal information being stored and potentially exploited. 

Suggested reading

Will we ever escape the ECHR?

By Thomas Fazi

Requiring official identity documents for online verification would exclude millions of people who lack easy access to digital IDs or who won’t provide such sensitive documentation merely to use a messaging service. In practice, this would spell the end of anonymous communication online, forcing users to present ID or face scans simply to open an email or messaging account. Breyer has warned that such measures would be particularly disastrous for whistleblowers, journalists, political activists and others reliant on online anonymity. It would also push under-16s towards less safe, poorly regulated alternatives that lack encryption or basic safety protections. 

Ultimately, critics argue that mass surveillance is simply the wrong approach to combating child sexual exploitation. Scanning private messages does not stop the circulation of child abuse material. Platforms such as Facebook have used scanning technologies for years, yet the number of automated reports continues to rise. Moreover, mandatory scanning would still fail to detect perpetrators who distribute material through decentralized secret forums or via encrypted archives shared using only links and passwords — methods that scanning algorithms cannot successfully penetrate. The most effective strategy would be to delete known abuse material from online hosts, something Europol has repeatedly failed to do. 

Chat Control, in short, would do little to actually help victims of child sexual exploitation while harming everyone else. Every message would become subject to surveillance, without any judicial oversight, contrary to long-standing guarantees of private correspondence. There’s a legal question here too. The EU Court of Justice has previously ruled that general and automatic analysis of private communications violates fundamental rights, yet the EU is now poised to adopt legislation that contravenes this precedent. Once adopted, it could take years for a new judicial challenge to overturn it. 

The confidentiality of electronic communication — essential for personal privacy, business secrecy and democratic participation — would be sacrificed. Sensitive conversations could be read, analyzed, wrongly flagged or even misused, as past scandals involving intelligence officials and tech employees have shown. One of the most notorious cases of intelligence abuse came from the US National Security Agency, in which multiple NSA employees were caught using the agency’s powerful surveillance tools to spy on romantic partners and ex-lovers. Leaked documents have also shown that the UK intelligence agency GCHQ captured and stored images from Yahoo webcam chats, including millions of sexually explicit images of completely innocent users. There have also been several cases of Big Tech employees — from Google to Facebook — using internal tools to spy on unsuspecting users. 

Furthermore, secure encryption, a foundation of cybersecurity, would be compromised by introducing backdoors or client-side scanning tools that foreign intelligence services or criminal actors could exploit. At the same time, the responsibility for criminal investigations would shift from democratically accountable authorities to opaque corporate algorithms, with minimal transparency or oversight. 

Opponents therefore argue that the EU should instead adopt a fundamentally different approach: one that protects children without undermining fundamental rights. They propose ending the current voluntary scanning of private messages by US internet companies — restoring the principle that targeted surveillance requires a judicial warrant and must be limited to individuals reasonably suspected of wrongdoing — and maintain that secure end-to-end encryption, and the right to anonymous communication, must be preserved. 

Particularly worrying is the issue of function creep, the process by which a technology introduced for a narrowly defined purpose gradually expands to serve broader, and sometimes entirely different, purposes over time. The UK’s Online Safety Act, passed in October 2023, obliges firms to develop child sexual abuse detection systems, even though the British government itself admits that such infrastructure is not yet technically available, creating legal authority awaiting technical capability. In the United States, “temporary” surveillance measures introduced under the post-9/11 Patriot Act became permanent, and indeed expanded in scope. Once a technological infrastructure for comprehensive online surveillance exists, it can easily be repurposed and is hard to dismantle. Technologies designed to detect harmful content can quickly be extended to political repression; examples from authoritarian states demonstrate how similar systems are used to identify and target dissidents. 

Breyer summarized this pattern starkly: “They are selling us security but delivering a total surveillance machine. They promise child protection but punish our children and criminalize privacy.” The implications are ominous. Europe effectively stands on the threshold of building a machine that can see everything. Once constructed, it will serve not only the current political authorities — the idea of Ursula von der Leyen spying on everyone’s messages is disturbing enough — but whoever wields power next. With yet another vote approaching, the window to stop Chat Control is narrowing. 

Chief among these is Lewis Mumford’s two volume Myth of the Machine (1966 and 1970) which examines the relationship between technology and culture from early man to now. This is everything Yuval Noah Harari has written in Sapiens or Homo Deus, and more (although Mumford is harder to read). Mumford was the pre-eminent tech critic of the 1960s and saw it as a dehumanising system of control.

In a similar vein, check out Neil Postman’s Amusing Ourselves to Death (1985). He argued that Huxley (distraction and entertainment) not Orwell (surveillance and boots) better predicted our dystopian future. The core idea is that the medium of communication dictates the sort of ideas and culture that can be sustained in a society (he was a student of philosopher and media theorist Marshall McLuhan after all). His focus on – and critique of – cable TV in the US is brilliant, and more relevant for social media than he could have predicted.

More from this author

How social media makes fascists of us all

By Jamie Bartlett

I find the radical fringes of thought are often a source of unexpected insight that mainstream writers dare not think or say. To that end, take a look at – and hold your breath for – the Unabomber’s Manifesto (1995). Ted Kascinski is a convicted terrorist who killed three people in a wave of anti-technology violence, and Manifesto is a rage against technology, which is in places violent and racist. But tucked inside are also fascinating speculations and predictions, some of which look surprisingly insightful twenty-five years on. (And don’t rule out a new wave of luddist violence – something I predicted here).

In a similar vein – Timothy C. May’s book-length blog ‘Cyphernomicon’ (1994) is a wild, bizarre, yet visionary, set of predictions about how encryption would transform society: crypto-currencies, dark net markets, ubiquitous encryption are all in there. May believed this would one day fatally weaken governments. He celebrated these trends; many of us will not.

Both of these works will shock and annoy you, but if you read them sceptically and carefully you’ll get a lot from them. And they’re both free to download.

More Christmas Reading

A survivor's guide to trauma

By Bidisha

Looking ahead rather than behind, two recent books set out a big problem heading our way. Both Cathy O’Neil’s Weapons of Math Destruction and Sayifa Umoja Noble’s Algorithms of Oppression (who both had decent stints in the private sector before writing these books) show how an increasing number of decisions are taken by smart machines and algorithms – but that doesn’t mean they will be less biased. In fact, they might simply replicate, or even worsen, existing economic, racial, gender or other inequalities, because they will appear objective and credible.

This problem – invisible algorithmic bias through search engines, or data discrimination – will become a major public preconception very soon, perhaps as soon as next year. The sooner we start thinking about it the better.