The machines aren't out to get you. Credit: ARUN SANKAR/AFP via Getty Images
When the apocalypse comes, most of us will barely notice it’s happening. Most technology-driven dystopias are far too interesting to be realistic: the end of the world will be a grinding, bureaucratic affair, its overriding spirit one of weary confusion — about how things work and who’s to blame when things go wrong.
Forget for a moment the flashier signals of technological process: AI-powered personal assistants, Boston Dynamics back-flipping robots or blockchain cheerleaders. The two most important trends in the field of technology are quiet and relentless: increasing volumes of data and declining cost of computing power. In the long run they mean machines will, despite frequent hiccups, keep improving. They already outperform humans in a small but growing number of narrow tasks, but it’s unlikely we’ll see general artificial intelligence any time soon — much less the AI-goes-rogue scenario. Still, machines will gradually take over more and more decision-making in important areas of life, including those which have ethical or political dimensions. Already there are signs of AI drifting into bail conditions, warfare strategy, welfare payments and employment decisions.
The problem isn’t whether machine decisions are better or worse — that’s often a question of values anyway — but whether it’ll get to the point where no one will be able to understand how decisions are made at all. Today’salgorithms already deal with millions of inputs, insane computing power and calculations far beyond human understanding. Frankenstein-like, most creators no longer really understand their own algorithms. Stuff goes in and stuff comes out, but the middle part is becoming a mysterious tangle of signals and weighting. Consider the example of AlphaGo, the AI system that astonished the world by thrashing the world’s best Go player, before astonishing it a second time by thrashing itself. Aeronautic engineers know precisely why their planes stay in the air; Alpha Go’s inner workings were and are a mystery to everyone. And by 2050, Alpha Go will be fondly remembered as a child-like simpleton.
There will be seminars, lessons, bootcamps, and online training courses about how to work with The Algorithm. Don’t worry yourself overly, human! Singletons: Learn the best combination of words to secure your dream date! Join our “beat the algo” seminar where you will learn how to ensure your CV outwits the HR filtering systems. Use our VPN to trick web browsers into thinking you’re from a poorer neighbourhood to secure a better price! A few months back a handful of bootcamps opened, where parents pay $2,000 for experts to teach their kids how to succeed on YouTube. Some scoffed, but I suspect similar courses will soon be the norm. These will be the warning signs of a confused and frightened society.
Imagine a 21-year-old happily bouncing through life in the 2050s. His entire life will have been datafied and correlated. His sleep patterns from birth captured by some helpful SmartSleep ap; his Baby Shark video consumption aged 2 safely registered on a server somewhere. All those tiny markers will help guide his future one day: his love life determined by a sophisticated personality matching software, while his smart fridge lectures him about meat consumption (insurance premiums may be impacted you know!); his employment prospects determined by a CV checking system 100 times more accurate than today’s. His cryptocurrency portfolio automatically updating every half nano-second based on pre-determined preferences. His political choices and opinions subtly shaped by what pops up on his screen controlled by AI-editors using preference algorithms that have been running for 50 years.
It sounds bad, but not apocalyptically bad, right? But imagine, now, that our 21-year-old is so impudent as to question or object to what these brilliantly clever systems are offering him up. There would probably be no obvious number to call with a complaint. He might try to sue the CV-checking software designed for the subtle discrimination he suffered — but the judges will throw the case out because the software designer has been dead for 30 years and they still don’t really understand what an algorithm is anyway.
The problem with such a machine-dependent world, then, is not what you might think. AI theorists spend a lot of time worrying about something called “value alignment”. It is a hypothetical future problem where a hyper-powerful AI takes instructions literally, with disastrous results. The most famous example is the “paperclip maximiser” where an unsuspecting factory owner asks an AI to make as many paperclips as possible — and it ends up turning the entire universe into paperclips. But I doubt you’ll need to worry about paperclips: you’ll be too busy on the phone to machine-like bureaucrats who can’t help with your application, because the machine has made a decision and the person who okayed it is off sick and the person who built the tech now works in Beijing and…
Confusing machines will annihilate accountability, which is one reason powerful people will like them. A couple of years ago UK health secretary Jeremy Hunt told the House of Commons that “a computer algorithm failure” meant 450,000 patients in England missed breast cancer screenings, and as a result many as 270 women might have had their lives shortened as a result. Who was responsible for this murderous and despicable “computer algorithm failure”? The tech guy who wrote the software, in good faith, years ago? The person who commissioned it? The people feeding the data in? Unsurprisingly, a subsequent inquiry into all this found that “no one person” was to blame. Nothing has been done in response, and nothing will. More recently, Boris Johnson blamed a “mutant algorithm” for the A-level fiasco — how convenient! Expect algorithms to become every politicians’ non-apology apology by the 2030s.
Around this time, the first casualties from driverless car accidents will start arriving in A&E. The subsequent enquiries will conclude that “no one person” is responsible for the deaths, either. It will instead be the fault of “unforeseen system incompatibilities” and “data corruptions” that make no sense, and offer no comfort, to anyone.
Presumably all this will be accompanied by a mild identity crisis. Some of us will pray to these God-like systems in the hope their mysterious inner workings are good to us. (An Uber driver was recently overheard muttering that, “The Algorithm has been good to me today”.) The less sanguine will presumably try to smash them to pieces. That will be destined to fail because, unlike the Spinning Jenny, software can’t be destroyed with a bat or arsonist. It’s somewhere you can’t reach it.
What will our leaders do about it? When people aren’t held to account, they tend behave worse — especially if someone or something tells them it’s OK. In his infamous experiment on the nature of authority, Stanley Milgram asked people to (they believed) electrocute other participants, which they generally did if a man in a white doctor’s jacket told them it was OK. He called this “agentic shift” — the process by which humans shift responsibility to abstract processes and systems, and in the process lose their own sense of right and wrong. People are worryingly good at following orders without question. Adolf Eichmann, the chief bureaucratic mastermind behind the Holocaust, is history’s most infamous rule-follower, but there were thousands like him inside the Nazi machine, telling themselves that they were only following orders, and so they were not to blame.
The Adolf Eichmanns of the future will be hip, jean-wearing technologists and bureaucrats who confidently assure everyone that they need to follow the complicated data models and respect the analytics. Outsourcing morality to a machine, writes Virginia Eubanks in her book Automating Inequality, gives the nation:
“the ethical distance it needs to make inhuman choices: who gets food and who starves, who has housing and who remains homeless, and which families are broken up by the state.”
Some form of ‘ethical distance’ is probably necessary for fair and objective government, but if it goes too far, the result is decision-makers who see little relationship between their decisions and the effect on people’s lives. Smart machines will likely make things worse because rather than just following rules and making sure your little jigsaw piece fits, bureaucrats will have a machine to rely on, an intelligence apparently smarter and wiser than they’ll ever be. The ultimate form of deniability.
If, one day in the future, a world-ending cyberwar breaks out — the most likely form the bureaucalypse might take — it won’t be caused by SkyNet going rogue. It will be initiated by a group of well-dressed and well-meaning civil servants who lack the courage or conviction to disagree with the machine-modelling and AI-Strategists which told them that overall well-being would be improved by 13.2 percentage points, that the risk of retaliation was minimal. Having spent the previous decades relying on machine advice for everything from music choices to cancer diagnosis, disagreeing with the supercomputers will seem impossible, maybe even immoral.
Obviously, we humans are too thin-skinned to give up on the idea that we’re the ones in charge, so we’ll still have the plebiscites, the MPs, the Select Committees and the opinion pages. But the whole point and purpose of democracy — to hold powerful people to account, to ensure well-informed citizens are ultimately in charge — would be reduced to a charade. Real power and authority will become centralised in a tiny group of techno-geniuses and black boxes that no-one understands.
If anything, as the range of problems politicians can actually solve shrinks, the fabricated outrage and manufactured disagreements will grow. Around the same time machines get to decide the most efficient tax rate, politicians will be literally throwing themselves onto pyres over survey question options or toilets signs. While, in the real world, algorithms will sort us by intelligence, ambition and attractiveness, politics will become at best an empty ritual, at worst a form of entertainment, like a WWE wrestling match. And the scariest thought of all is this: a world run by machines and rubber stamped by humans who’ve forgotten how to think — all divorced from a democracy that has been reduced to pure content — might not worry people at all. In fact, plenty of us will probably quite enjoy it.
Join the discussion
Join like minded readers that support our journalism by becoming a paid subscriber
To join the discussion in the comments, become a paid subscriber.
Join like minded readers that support our journalism, read unlimited articles and enjoy other subscriber-only benefits.
SubscribeThis is the kind of article for which I subscribe to Unherd. Thoughtful, deep, well-written, unrushed in its manner and form – a work of art all its own. My thanks to the author and to Unherd. More of such would be most welcome!
True, but so painful to read.
My suggestion is don’t go to university, don’t rely on an art gallery to legitimise your art, don’t bother with the arts community, stop believing in the idea of being an artist. They are not savants, they’re merely entertainers. If you think you’re something special and you want to live off it then you’re going to have to play the game. You don’t need them to create, but you do to make money. Art is the ultimate elite game. This story just confirms it. The arts community talk about AIs destroying art because it’s not human. There’s nothing human about the arts today. It’s a closed, suffocating shop ticking off the boxes and bending everything into some shape that satisfies their warped egos, including antisemitism, the latest “trend”.
Having spent a professional life encouraging students to develop their thinking skills and knowledge by attending university, previously known as institutes of higher learning (but no longer deserving of that title), I would counsel any youngster possessed of an independent spirit and a modicum of intelligence to avoid these conformist echo chambers like the plague.
I entirely agree. And the world of contemporary literature is fast behind it: despite having an even bigger claim to being a medium meant to make us think, see other perspectives and empathise.
We have a number of tax-funded literary presses in the UK pumping out anti-western propaganda on a daily basis. 87 press is one of the most pernicious. Its editor is also studying for a tax-funded PhD in [whatever], while denying anyone was raped and any ‘innocents’ were harmed on October 7th.
It’s funny that art schools preach transgression but oddly seem to produce conformism.
I guess this is what happens when the traditional battle grounds of sexuality and politics have effectively been conquered.
I’m trying to remember my own art school days and what agitated us back then. As I remember it was ban the bomb, anti Thatcher and the after shock of the AIDS epidemic. I genuinely think we still had boundaries that needed to be crossed.
What we have now, some 35 years later, is a borderless, valueless society where everything is relative and nothing is sacred.
Maybe we are doomed to repeat cycles of hate and conflict. All that remains is for the people to decide which group of others to go to war against.
Yes, herdish transgression is the new “plat du jour” and relativism the latest absolute
The reoccurring theme here with the artists mentioned isn’t that they’re Jewish, it’s that they’re Israeli, which isn’t the same thing. Of course no one should blame each and every Israeli citizen for the actions of their government, that’s absurd, but assuming all criticism of the Israeli state is motivated by hostility to Jewish people is also absurd, especially when many of Israel’s staunchest critics are themselves Jewish.
Sorry but as a British Jew I can tell you that while you think this to be the case, it isn’t. If you are Jewish there are parts of this country that will shun you unless you are on ‘their’ side. We are guilty by association unless we denounce – sounds eerily familiar.
You missed his point.
He was speaking of the references in the article, not of your (alleged) personal experience.
With respect I didn’t miss his point and although most of the artists quoted were Israeli there is one that isn’t, and undoubtedly more that didn’t want to be named as suggested in the article. My point is that we as British Jews cannot disassociate or indeed are not allowed to disassociate ourselves from the conflict. People consistently try to separate the two, but it is impossible without us denoucing what is going on. When doors were kicked in on my brother’s road a few years ago it wasn’t because those houses all had Israeli flags, it’s because they displayed symbols of Judaism and were automatically associated with Israel. This is what people do not understand when they make comments like the above. It applies to any religion/religious conflict but is most pronounced with respect to Jews and Israel.
Answered as a new comment
Just to be clear, it is eerily familiar of the authoritarian left (Stalin,etc). A Jewish artist who is explicitly and resolutely anti-Zionists and denounces Israel will be lauded by the art world which is overwhelmingly dominated by the left.
This is exemplified by the open letter published in July 24 by Artists for Palestine UK, the signatories, including more than 100 Jewish creatives, decrying as “shameful” the Royal Academy’s removal of a photograph of a protestor holding a placard that reads, “Jews Say Stop Genocide on Palestinians. Not In Our Name.
More than ever, the West’s art world is a leftist propagandas machine. Jews and non-Jews alike will be exhibited if they keep to the left.
assuming all criticism of the Israeli state is motivated by hostility to Jewish people is also absurd,
did you read the article? Did you get to the part where the author speaks to a particular piece and is told “how could any country promote an artist like her?”
Hi Chris, it’s the same. Israel is not just a Jewish state, but the land of the Jews (the Jewish people), and its name is Israel. It’s easier to say “Israeli” than “Jew” to cover up hate, and people use “social justice” arguments to denounce one country. Go check the number of deaths and refugees in Ukraine/Russia. Where is the venom there? You start calling the Jews “Semite,” then kill them all and start saying “antisemitism.” And the Jews start using this to explain the hate. So what do you do? Start saying “Zionist” – a movement that ended 75 years ago. I’m waiting for the next name.
how could any country promote an artist like her?
Ah, the ever-present desire to silence, stigmatize, and scapegoat people of a certain identity. What are the odds of the woman quoted saying the same of a black artist, a Latino, a gay, or anyone else from the congregation within the church of the aggrieved and offended?
Art has become another word for another institution paralyzed by mind-numbing groupthink that mistakes itself for wisdom. Voltaire was right in saying that the people who can make you believe absurdities can make you commit (or accept) atrocities.
It’s appalling this antisemitism is allowed to continue. There’s a post on another forum from a concerned Jewish mother whose child is being subjected to anti-semitic bullying at her school which has 30% Muslim pupils. Quite a few posters have either disbelieved her or asked ‘why are you surprised?’
Zionists like talking about ‘light’ while obscuring it as much as possible. When they talk about art, it is invariably about the artifice of conflating Judaism with zionism and Jews with zionists. Such subversion of language is not artistic.
As ever, _what_ are you talking about. Zionism= believing the Jewish people should return to their historic homeland. What do you know about art anyway? Your religion only permits shapes.
When left to their own devices, Jews create Israel, whilst Muslims create Daesh. By their works ye shall know them.
The zionist state was created by the Brits, not the zionists, like Northern Ireland. And much like NI, it created a trail of misery stretching to, but not ending in, the present day.
Historically illiterate. It was _recommended_ by the Brits as only fair that the Jews should have a state and the Muslims too: Transjordan, which they got. The UN voted in favour of Israel’s creation & the trail of misery has been the abject refusal of all the surrounding Islamic countries refusing to live alongside a single Jewish one ever since.
Some day historians will write about this time the same way they described Europe in the 1930s and 40s: the Jew-haters who wanted to kill us Jews because we’re Jewish, and those who fought that medieval barbarity. We live in such a time again.
100% solidarity with the innocent victims of the hypocritical anti-jewish, leftist arts world.
“Which now of these three, thinkest thou, was neighbour unto him that fell among the thieves?
And he said, He that shewed mercy on him. Then said Jesus unto him, Go, and do thou likewise.”
More Levites and priests around nowadays than Good Samaritans. Pious hypocrisy is much in vogue. Not much mercy.
The Sokal Affair last century showed what the art world has become. A propaganda machine for leftist nonsense.
This article points out the real threat from the far left, socialist anti west ideology which threatens our way of life. Cancel culture which attacks free speech, free expression and liberty writ large, is a baked into the cake principle of Marxism. It would be a good idea for everyone to brush up on Lenin, Stalin and Mao tactics of implementing Marxist Communism and Socialism with regards to those who did not get in line with their doctrines. Over 75 million people were murdered or died from starvation in those regimes, and freedom of the people did not exist. It is not alarmist to say we need to be very alarmed and fight back against this scourge to Western Civilization values. Antisemitism as described in this article is just the beginning. We are all at risk.