One reason for our indifference is that the true scale of the theft has been hidden from us. Social media platforms have for years been speeding up our sense of time — effectively shortening our lives — and yet they do this in such a devious manner that we rarely realize what we’ve lost.

Every social media user has had their time pickpocketed. You may log on to quickly check your notifications, and before you know it, half an hour has gone by and you’re still on the platform, unable to account for where the time went. This phenomenon even has a name: the “30-minute ick factor”. It also has empirical support. Experiments have found that people using apps such as TikTok and Instagram start to underestimate the time they’ve spent on such platforms after just a few minutes of use, even when they’re explicitly told to keep track of time.

This is no accident. Sean Parker, Facebook’s founding president, said: “The thought process that went into building these applications… was all about: ‘How do we consume as much of your time and conscious attention as possible?’” Unsurprisingly, Parker himself doesn’t use social media, saying it’s “too much of a time sink”.

To understand how Big Tech steals our time, we must first get our heads around time perception, or chronoception. Time doesn’t always feel like it’s moving at a constant pace. The weightier an experience is, the slower time will often feel. It’s why people tend to overestimate the duration of earthquakes, accidents, or any other scary situation.

Generally, an event feels longer in the moment if it heightens awareness. But we seldom think of time in the moment; rather, our sense of time tends to be retrospective. And our sense of retrospective time is determined by memory. The more we remember of a certain period, the longer that period feels, and the slower time seems to have passed.

Sometimes an experience can seem brief in the moment but long in memory, and vice versa. A classic example of this is the “holiday paradox”: while on vacation, time speeds by because you’re overwhelmed by new experiences. But in retrospect your vacation feels longer, because you made many strong memories, and each adds depth to the past. Conversely, when you’re waiting at a boring airport, constantly checking the clock, your hyperawareness of time causes it to pass slowly in the moment. But since the wait is uneventful, you don’t make strong memories of it, and thus, in retrospect, it seems brief.

The sinister thing about social media is that it speeds up your perception of time both in the moment and in retrospect. It does this by simultaneously impairing your awareness of the present and your memory of the past.

More from this author

Is liberal society making us ill?

By Gurwinder Bhogal

Try to recall what you saw on social media the last time you scrolled. You’ll notice you can barely remember any posts, even if you scrolled for hours. This phenomenon has been confirmed by studies that have found that social media impairs both short-term and long-term memory. A social media feed is like the Lethe, the mythical river in whose waters lost souls sought absolution, and received it in the form of oblivion.

But what explains this “Lethe effect”? Theoretically, a social media feed should heighten awareness and memory, and dilate time, because it selects for content that’s exciting, outrageous, and scary. And yet we seldom remember such content. The reason for this discrepancy is simple: when every post is alarming, your brain quickly becomes desensitized, and starts to interpret alarming content as routine. And routine, being passive and therefore immemorable, speeds up time.

There are other, more cunning ways that social media impairs awareness and memory. The use of “attention engineers” is one of them. These are people employed specifically to design interfaces to capture as much of your time and attention as possible.

To understand how attention engineers deliberately warp your time, we must look to the history of casino floor design. In the Seventies, a man named Bill Friedman turned from gambling addict to casino manager by studying the tricks used to manipulate him — and perfecting them. He eventually published his ideas in several books, which would become bibles of casino design.

Friedman’s philosophy seems to have been borrowed from the retail industry. Supermarkets have long been designed like mazes, with everyday items like milk and eggs deep in the heart of the store, so you must pass by countless other products to access them. The purpose of this layout is to evoke the Gruen effect: the moment when a shopper loses track of what they entered the store for, and begins aimlessly wandering and impulse-buying.

Friedman proposes a similar strategy: to arrange casinos like mazes, where even the paths to the toilets and exits would spiral and meander through rows of enticing games machines. Such an overwhelming environment would distract people from themselves, so they’d remain instinctual rather than intentional, and hence compliant rather than resistant.

Friedman rejected open-plan designs, instead advocating for casinos to be sectioned into cubicles so players could only see their immediate surroundings. This was partly to restrict their awareness, but also to create constant FOMO: players would hear excited hoots and cheers coming from a nearby cubicle, and go searching for the cause of the commotion. In doing so, they’d wander further into the maze.

The pathways in Friedman’s mazes had as few right-angle turns as possible. This is because sharp bends jolt pedestrians into awareness by forcing a change of direction. And when someone has to decide where to go, they’re liable to think about whether it’s time to leave. For this reason, Friedman advocated for curvilinear paths that had no discernible corners, beginnings, or ends, and could thus be perpetually navigated on autopilot. The intention behind this hugely successful design was to keep people unaware by maximizing immersion, distraction, and disorientation.

Friedman’s ideas revolutionized casinos around the world. And many of his tactics would later resurface in social media design, where they proved even more successful at hindering awareness. For instance, in the early days of social media, it was possible to reach the end of a feed. These ends acted much like right-angle turns, snapping the scroller out of autopilot. Soon, however, the feeds were made “curvilinear” by the infinite scroll and autoplay function — features which, we now know, impair awareness and memory by lulling people into passivity.

Meanwhile, social media platforms have become increasingly labyrinthine, with the aim of trapping people in them. Every webpage is littered with links, each a path to another maze. And many of these links are deliberately placed where they don’t belong: search results are sneakily scattered with recommendations unrelated to your search, and personal notifications often have generic news links hiding among them.

What makes social media even more disorienting than a casino is that our feeds are not just mazes in space, but also in time. The opposite of a maze is a route, and a route through time is a story. This is because stories are linear and syntagmatic — each moment of the tale semantically follows from the previous — and such collective meaningfulness anchors the whole thing in memory. This is why studies have consistently found that people are much better at memorizing information when it’s presented in narrative form.

The memorable and sequential nature of stories makes them good timekeepers. As such, the way we make sense of time is through emplotment: by turning time into stories. It’s why research finds that people who are similarly engaged in a story will tend to converge in their estimates of how much time has elapsed. If we can’t turn a duration into a story, we struggle to keep track of it.

Now here’s the issue: your social media feed resists emplotment because it’s the opposite of a story. It’s a chronological maze. It has no beginning, middle, or end, and each post is unrelated to the next, so that scrolling is like trying to read a book in a windstorm, the pages constantly flapping, randomly switching between scenes, so you can never connect the dots into a coherent and memorable narrative.

Thus, not only do you forget time while scrolling through posts, but you also forget the posts themselves. We have no problem recounting the plot of a good book we read or movie we saw last year, yet we can barely remember what we saw on social media yesterday.

More from this author

The Menendez brothers prey on empathy

By Gurwinder Bhogal

Despite not having much memory of your social media feed, you may have a vague sense that you at least enjoy scrolling. This, too, is likely a trick. Research suggests that people judge an experience as being more enjoyable if they believe they underestimated its duration. In other words, not only does time fly when we’re having fun, but we also believe we had fun if time flies. In this way, attention engineers reduce the likelihood that you’ll regret the time you waste online.

Even if you do regret it, social media excels at making you return. Friedman’s cubicles were designed to spark FOMO by letting you hear the cheers of excited players without allowing you to see the cause — unless you entered. Similarly, social media push notifications periodically tease you with what you’re missing out on, and the only way for you to find out more is to re-enter the maze. The result of having your day punctuated by these notifications is that your attention is constantly intercutting between the real world and the virtual one, so you can never fully settle in either.

This creates problems of its own. When attention is constantly switching between concurrent tasks, it imposes a “switch-cost effect” that can make people lose track of time. By constantly interrupting you, social media platforms can therefore impair your awareness and shorten your days even when you’re not on them, so that you end up scrolling through the real world as shallowly as the virtual one.

Now that we understand this, we can try to do something about it. Fortunately, there are ways to actively slow down how we experience time. One way is to quit social media — or at least restrict your usage. Research suggests that abstaining from social media can lead to immediate time dilation, particularly for compulsive users. Further, a study of 35,000 people found that quitting social media tended to slightly improve perceived mental health after just a few weeks.

However, that same study also found that the time people saved by quitting social media was often just spent browsing other apps, which are increasingly emulating the time-warping features of social media. Take chatbots, for instance. They are inherently mazelike; not only do they frequently hallucinate red herrings, but they’re also prone to “verbosity compensation”, which means they often ramble and equivocate in their responses, raising more questions with every answer, and creating a kind of verbal Gruen effect. They also have a tendency to validate users’ delusions, leading them further down deceptive and dangerous rabbit-holes.

Chatbots are also becoming curvilinear, increasingly ending their answers with a question or an offer for further help to create a kind of conversational infinite scroll. And now Meta plans to train its chatbots to message you unprompted — the AI equivalent of Friedman’s cubicles. These developments pose problems because there is emerging evidence that chatbots, when used carelessly, can impair awareness and memory just like social media.

The deeper problem, then, is not social media or chatbots. It’s curvilinear mazes. If we can avoid following smooth spirals to nowhere, and instead take sharp turns to clear destinations, we can stay aware and keep track of time. We could even learn to use social media and chatbots — two incredibly useful technologies — in ways that enrich our lives rather than impoverish them.

We now know the ways social media platforms speed up time. If we can adopt the opposite techniques, we might just be able to slow time. What the human brain needs to make strong memories are salient stimuli — surprises, stories, (strong) sentiments, and selections. Social media gives you the illusion of these things, but it is actually their killer. Instead, we must seek out each of these in the real world.

One way of doing this is to always choose the life experience that’s most likely to lead to a good story: reading books instead of scrolling social media; going on adventures instead of staying at home. Focus too on the sentiments these stories inspire. The simplest way to strengthen a feeling is to savor experiences. So stop idly scrolling through life as if it’s a feed, and learn to focus your attention on the here and now. A review of 47 research articles found that well-practiced mindfulness is associated with slowed subjective time.

The more you avoid living on autopilot, the more of life you’ll remember, and the longer it will feel. So make a habit of resisting habit, and instead, choose a life of choices. Whenever you find yourself doing something — for instance, habitually checking your phone — ask yourself why you’re doing it, and whether there are better alternatives. If there are, or if you don’t know why you’re doing it, do something else.

This brings us to perhaps the greatest time dilator of all: surprise. We tend to remember novel experiences much more than repeated ones. It’s why time seems to speed up as we age: as we grow older, fewer of our experiences are new, so fewer stick in memory. It’s also why old people tend to remember their early lives better than their adult lives. The power of new experiences to stick out to us would explain why studies have consistently found evidence for the “oddball effect”, a phenomenon where a surprising stimulus presented among predictable stimuli is perceived to last longer. Recent research has found that the stimulus that immediately follows the surprising stimulus is also perceived to last longer, suggesting that novel experiences slow time by heightening awareness.

So there you have it. To make life feel longer, choose experiences that are novel over familiar, intentional over habitual, narrative over disjointed, and emotional over neutral. Above all, don’t let online algorithms set the pace of your life, as they’re not operating in your interests.

Seneca once said: “Life is short and anxious for those who forget the past, neglect the present, and fear the future.” Social media makes you do all three. But you have the choice to do the opposite, and to expand time, for living long is not just about maximizing the days in your life, but also the life in your days. This moment is one your future selves will wish they could have back. Don’t waste it scrolling through posts you won’t even remember tomorrow.

Forget for a moment the flashier signals of technological process: AI-powered personal assistants, Boston Dynamics back-flipping robots or blockchain cheerleaders. The two most important trends in the field of technology are quiet and relentless: increasing volumes of data and declining cost of computing power. In the long run they mean machines will, despite frequent hiccups, keep improving. They already outperform humans in a small but growing number of narrow tasks, but it’s unlikely we’ll see general artificial intelligence any time soon — much less the AI-goes-rogue scenario. Still, machines will gradually take over more and more decision-making in important areas of life, including those which have ethical or political dimensions. Already there are signs of AI drifting into bail conditions, warfare strategy, welfare payments and employment decisions.

The problem isn’t whether machine decisions are better or worse — that’s often a question of values anyway — but whether it’ll get to the point where no one will be able to understand how decisions are made at all. Today’salgorithms already deal with millions of inputs, insane computing power and calculations far beyond human understanding. Frankenstein-like, most creators no longer really understand their own algorithms. Stuff goes in and stuff comes out, but the middle part is becoming a mysterious tangle of signals and weighting. Consider the example of AlphaGo, the AI system that astonished the world by thrashing the world’s best Go player, before astonishing it a second time by thrashing itself. Aeronautic engineers know precisely why their planes stay in the air; Alpha Go’s inner workings were and are a mystery to everyone. And by 2050, Alpha Go will be fondly remembered as a child-like simpleton.

There will be seminars, lessons, bootcamps, and online training courses about how to work with The Algorithm. Don’t worry yourself overly, human! Singletons: Learn the best combination of words to secure your dream date! Join our “beat the algo” seminar where you will learn how to ensure your CV outwits the HR filtering systems. Use our VPN to trick web browsers into thinking you’re from a poorer neighbourhood to secure a better price! A few months back a handful of bootcamps opened, where parents pay $2,000 for experts to teach their kids how to succeed on YouTube. Some scoffed, but I suspect similar courses will soon be the norm. These will be the warning signs of a confused and frightened society.

Imagine a 21-year-old happily bouncing through life in the 2050s. His entire life will have been datafied and correlated. His sleep patterns from birth captured by some helpful SmartSleep ap; his Baby Shark video consumption aged 2 safely registered on a server somewhere. All those tiny markers will help guide his future one day: his love life determined by a sophisticated personality matching software, while his smart fridge lectures him about meat consumption (insurance premiums may be impacted you know!); his employment prospects determined by a CV checking system 100 times more accurate than today’s. His cryptocurrency portfolio automatically updating every half nano-second based on pre-determined preferences. His political choices and opinions subtly shaped by what pops up on his screen controlled by AI-editors using preference algorithms that have been running for 50 years.

More from this series

Is this the end of the world?

By Douglas Coupland

It sounds bad, but not apocalyptically bad, right? But imagine, now, that our 21-year-old is so impudent as to question or object to what these brilliantly clever systems are offering him up. There would probably be no obvious number to call with a complaint. He might try to sue the CV-checking software designed for the subtle discrimination he suffered — but the judges will throw the case out because the software designer has been dead for 30 years and they still don’t really understand what an algorithm is anyway.

The problem with such a machine-dependent world, then, is not what you might think. AI theorists spend a lot of time worrying about something called “value alignment”. It is a hypothetical future problem where a hyper-powerful AI takes instructions literally, with disastrous results. The most famous example is the “paperclip maximiser” where an unsuspecting factory owner asks an AI to make as many paperclips as possible — and it ends up turning the entire universe into paperclips. But I doubt you’ll need to worry about paperclips: you’ll be too busy on the phone to machine-like bureaucrats who can’t help with your application, because the machine has made a decision and the person who okayed it is off sick and the person who built the tech now works in Beijing and…

Confusing machines will annihilate accountability, which is one reason powerful people will like them. A couple of years ago UK health secretary Jeremy Hunt told the House of Commons that “a computer algorithm failure” meant 450,000 patients in England missed breast cancer screenings, and as a result many as 270 women might have had their lives shortened as a result. Who was responsible for this murderous and despicable “computer algorithm failure”? The tech guy who wrote the software, in good faith, years ago? The person who commissioned it? The people feeding the data in? Unsurprisingly, a subsequent inquiry into all this found that “no one person” was to blame. Nothing has been done in response, and nothing will. More recently, Boris Johnson blamed a “mutant algorithm” for the A-level fiasco — how convenient! Expect algorithms to become every politicians’ non-apology apology by the 2030s.

Around this time, the first casualties from driverless car accidents will start arriving in A&E. The subsequent enquiries will conclude that “no one person” is responsible for the deaths, either. It will instead be the fault of “unforeseen system incompatibilities” and “data corruptions” that make no sense, and offer no comfort, to anyone.

Suggested reading

The narcissism of apocalyptic thinking

By Sam Leith

Presumably all this will be accompanied by a mild identity crisis. Some of us will pray to these God-like systems in the hope their mysterious inner workings are good to us. (An Uber driver was recently overheard muttering that, “The Algorithm has been good to me today”.) The less sanguine will presumably try to smash them to pieces. That will be destined to fail because, unlike the Spinning Jenny, software can’t be destroyed with a bat or arsonist. It’s somewhere you can’t reach it.

What will our leaders do about it? When people aren’t held to account, they tend behave worse — especially if someone or something tells them it’s OK. In his infamous experiment on the nature of authority, Stanley Milgram asked people to (they believed) electrocute other participants, which they generally did if a man in a white doctor’s jacket told them it was OK. He called this “agentic shift” — the process by which humans shift responsibility to abstract processes and systems, and in the process lose their own sense of right and wrong. People are worryingly good at following orders without question. Adolf Eichmann, the chief bureaucratic mastermind behind the Holocaust, is history’s most infamous rule-follower, but there were thousands like him inside the Nazi machine, telling themselves that they were only following orders, and so they were not to blame.

Suggested reading

The calculated evil of Adolf Eichmann

By Alexander Faludy

The Adolf Eichmanns of the future will be hip, jean-wearing technologists and bureaucrats who confidently assure everyone that they need to follow the complicated data models and respect the analytics. Outsourcing morality to a machine, writes Virginia Eubanks in her book Automating Inequality, gives the nation:

“the ethical distance it needs to make inhuman choices: who gets food and who starves, who has housing and who remains homeless, and which families are broken up by the state.”

Some form of ‘ethical distance’ is probably necessary for fair and objective government, but if it goes too far, the result is decision-makers who see little relationship between their decisions and the effect on people’s lives. Smart machines will likely make things worse because rather than just following rules and making sure your little jigsaw piece fits, bureaucrats will have a machine to rely on, an intelligence apparently smarter and wiser than they’ll ever be. The ultimate form of deniability.

If, one day in the future, a world-ending cyberwar breaks out — the most likely form the bureaucalypse might take — it won’t be caused by SkyNet going rogue. It will be initiated by a group of well-dressed and well-meaning civil servants who lack the courage or conviction to disagree with the machine-modelling and AI-Strategists which told them that overall well-being would be improved by 13.2 percentage points, that the risk of retaliation was minimal. Having spent the previous decades relying on machine advice for everything from music choices to cancer diagnosis, disagreeing with the supercomputers will seem impossible, maybe even immoral.

Obviously, we humans are too thin-skinned to give up on the idea that we’re the ones in charge, so we’ll still have the plebiscites, the MPs, the Select Committees and the opinion pages. But the whole point and purpose of democracy — to hold powerful people to account, to ensure well-informed citizens are ultimately in charge — would be reduced to a charade. Real power and authority will become centralised in a tiny group of techno-geniuses and black boxes that no-one understands.

If anything, as the range of problems politicians can actually solve shrinks, the fabricated outrage and manufactured disagreements will grow. Around the same time machines get to decide the most efficient tax rate, politicians will be literally throwing themselves onto pyres over survey question options or toilets signs. While, in the real world, algorithms will sort us by intelligence, ambition and attractiveness, politics will become at best an empty ritual, at worst a form of entertainment, like a WWE wrestling match. And the scariest thought of all is this: a world run by machines and rubber stamped by humans who’ve forgotten how to think — all divorced from a democracy that has been reduced to pure content — might not worry people at all. In fact, plenty of us will probably quite enjoy it.

That was the beginning of her voyage of discovery into the biases of computer code; one which would — as a new Netflix documentary, Coded Bias, shows — lead her to testify before lawmakers in the U.S. Congress. Facial recognition technology (FRT), Buolamwini proved, is often poor at identifying black faces and female faces — largely because it is “trained” on data sets with far more images of white men. The bias may be inadvertent, but, as Buolamwini points out, has very real consequences.

Still, in a biased world, do you really want FRT to perform well in recognising your face? If you’re trying to unlock your phone, probably yes. If you want to project a fearless lion’s face onto your reflection in the morning, I suppose so. But if you’re walking past a Metropolitan Police FRT van, possibly not. In Coded Bias, Buolamwini’s story is interwoven with that of Big Brother Watch, as the organisation challenges police use of FRT. In a sequence filmed on London’s streets, a 14-year-old black schoolboy is stopped and searched by plain clothes officers after his face is wrongly matched by FRT with a suspect on their database.

Previous surveys in the UK have found that ethnic minority respondents were less keen for the police to have unfettered use of FRT. Not just because the technology is less likely to wrongly tag a white schoolboy, though that is true. Ethnic minorities’ experience of bias — not their knowledge of the technology — makes them less likely to see the enhancement of police powers as a good thing. More accurate technology, trained on more diverse databases, won’t help that lack of trust in the police.

More from this author

The danger of digital strip searches

By Timandra Harkness

“What is the purpose of identification?” asks Apartheid historian Patric Tariq Mellet when Buolamwini visits South Africa. Mellet displays the racial classification papers that allocated South Africans to categories — based not on who they felt themselves to be, but what the state decided they were. The purpose of this identification was clear: to control where the carrier could go, what they could do, even whom they could marry.

With facial recognition technology, there is no need to carry papers. Your face is your ID card. You could be sorted by any system, not just by visible characteristics like skin colour or sex, but by linking your face to all the unseen information sitting in databases. Not only your credit history, but your internet search history. Not just your home postcode, but your mood as expressed in walking speed.

This is where the visible bias of the inadequately trained FRT systems and the invisible bias of algorithms, trained on data from an unequal past, come together. It’s relatively easy to object that a program has misclassified your face or failed to recognise you as a person. It’s much harder to discover that you’ve been wrongly tagged with a poor credit history or tendencies towards political extremism.

In any case, “wrongly” becomes meaningless when algorithms apply population-scale predictions to individuals. The best predictor of your likelihood to turn to crime is your postcode. Is it fair to tag everyone from the same neighbourhood as future criminals? The best predictor of your future educational attainment is your school’s past performance. Is it fair for an algorithm to allocate exam marks on that basis, as the A Level algorithm did last year?

Suggested reading

How corporations can delete your existence

By Gavin Haynes

Big data links together disparate sources of information to profile each of us. Systems using FRT can attach each data profile to a specific physical person in the real world. This is clearly a gift to anyone with power, political or economic. But it also threatens to transform our personal relationships.

In a beautifully simple and telling sequence filmed in China, a skateboarding young woman explains how she uses the ubiquitous Facial Recognition systems all the time. She uses her face to buy groceries, because her face is linked with her bank account. It’s also linked to her Social Credit score, which combines data from official records with “bad behaviour” like “making false reports” online. People with a low score may be denied travel on trains and aeroplanes.

Unlike most profiling in the West — where you may never know that you are paying more for flights, or that you are on a police watch list for domestic extremism — China’s Social Credit scores are public. Your trustworthiness is not a personal quality to be discovered by trial and error, but a numerical value calculated by an algorithm and available for anyone to see.

This is a good thing, says our Chinese woman. When she meets somebody new, she doesn’t have to use her own judgment to decide whether to trust him. She can save time by checking his Social Credit score before deciding whether to be his friend. But trust is not a substance to be quantified, like a bank balance. It’s a relationship. We trust our friends and family because we have built up bonds of mutual commitment, of empathy and intimacy, by getting to know them, by sharing our lives with them, by opening ourselves to each other and taking the risk of being let down or betrayed.

More from this author

How Ofqual failed the algorithm test

By Timandra Harkness

Where in China the algorithmic sorting is compulsory, in the democratic West choices remain. Of course, applicants for welfare, and those arrested suspects seeking bail, can’t opt out of systems run by algorithms — systems in which the stakes are high, and algorithms encoded with prejudice. But because we live in democracies, we can object to how these algorithms are used. There is no reason, in principle, why algorithmic relationships between institutions and individuals should not also be subject to democratic oversight, and in some cases they are.

Suggested reading

Blair's constant war on freedom

By Maajid Nawaz

But this brings us to the hardest, most important question: why don’t we object more often? Why are we so relaxed about being identified and sorted by machines? Most people in the UK do not think that police use of Facial Recognition Technology should be banned. The idea that we should all have digital “Vaccine Passports” or “Immunity Certificates” to resume public life is welcomed by many. In spite of repeated scandals about how our data is collected and used, most of us continue to use social media to interact with our friends.

Of course, when unfair outcomes emerge from machines programmed to learn from the past, like the Amazon hiring algorithm that discriminated against female applicants because past employees tended to be men, we object. But the general principle that machines should be able to recognise us and sort us into categories that will determine our options in life does not seem to bother most of us. In many ways, we still put our faith in machines to be fairer than humans, less biased, more objective.

Is this fatalism or ignorance of the true extent of the power relationships embedded in the algorithms? Or do we, like the Chinese interviewee in Coded Bias, simply not want to exercise our own judgment about the people we meet? Perhaps we like the ways that technology mitigates the riskiness of unmediated human life. Instead of taking responsibility for hiring this person and rejecting that one, why not write some code that can choose your next employee?

We tend to think of algorithms as tools in the hands of the powerful, guided by super-intelligent people to achieve their sinister ends. But although the technology has powerful effects on the lives of individuals, it also veils the weakness of those who use it. They lack the courage to exercise judgment. They lack a clear vision of a future to steer towards. They automate relationships with the people they should be persuading, or inspiring, or helping, or leading. It’s not just the bias in these systems that should trouble us. It’s the rush to abandon human agency to them.

Many image classification systems tend to misclassify black women’s faces as male. That’s if they recognise them as human faces at all. Google’s image classification system got it badly wrong in 2015 when it mislabelled black people as ‘Gorillas’.

It’s not unusual for algorithms to make mistakes about humans. Sometimes those errors are harmless. You might see adverts for inappropriate products, or jobs for which you are wildly unsuitable. Other times, they are more problematic, by sending people to jail with the wrong sentence, or rejecting potential job candidates because they don’t resemble recruits from past years.

It’s not just that the programs aren’t intelligent enough, it’s that they don’t share our values. They don’t have goals like “fairness” or even “not being racist”. And the more we delegate decisions to machines, the more that matters.

That’s the problem Brian Christian tackles in his new book, The Alignment Problem, subtitled, How can machines learn human values? One pithy question encompassing several others: How can machines learn anything? What are “human values”? And do either machines or humans learn values in the same way as learning information, strategies, skills, habits, goals?

More from this author

It's not our job to protect the NHS

By Timandra Harkness

The book begins with an account of how machines came to learn at all. It’s a history of brilliant and often odd people, ideas that seemed absurdly far-fetched, inspiration found in unlikely places, and creators confounded by their own creations.

From the early 20th century, computer development and research into human cognition were parallel projects with myriad connections. Engineers looked to the human mind to help them design machines that would think, but equally, neuroscientists and psychologists looked to mathematics and logic to build conceptual models of human thought. Those machines which began with the simple task of recognising whether a square was on the left or the right of a card, and progressed to playing Go better than any human, were seen as working models of the human mind.

In this way, familiar ideas in computer science map, very broadly, onto more familiar ideas in psychology. For example, cognitive scientist Tom Griffiths builds reward-motivated AI systems. He describes his daughter dropping food on the floor to earn praise for sweeping it up again in the same terms he might use at work. “As a parent you are designing the reward function for your kids, right?” He learned to praise her for a clean floor, not for the act of sweeping up.

When they couldn’t get their computer programs to succeed at given tasks, researchers looked back at humans to help them see what was missing. Machines quickly overtook humans in certain ways: speed of logical reasoning, or processing more information than one human could handle at once. But that was not enough. What did Neural Networks lack, that real human brains used to solve new problems, and learn to navigate new environments?

Computers built to mimic human logical reasoning were missing fundamental human drives, among them curiosity. Novelty and surprise, it turned out, were as important as information processing, not only for human life but for machines playing the Atari game Montezuma’s Revenge. The human brain’s dopamine system suggested a new model to reward experimentation and persistence in game-playing computer programs, and that equipped them to win.

More from this author

Can your smartphone crack Covid?

By Timandra Harkness

Alongside the machines, their human creators were also learning: not only how to build them, and what data to train them on, but also how to teach them. Learning by imitation, which comes naturally to humans almost from birth, can also work for robots using Artificial Intelligence. With a method called Inverse Reinforcement Learning, AI can even infer what a human is trying to do, and outstrip its teacher, in complex tasks like flying helicopter drones.

Machines can learn to outperform their human teachers at playing games, sorting images, or even controlling a vehicle on real roads. A robot can imitate your behaviour, and infer your goals, but can it learn the right thing to do? Can machines learn human values?

Ask a philosopher this question and you would probably get a question in return: What are “human values”? Most of us muddle along with an impure, cobbled-together morality made up of habits, boundary lines we took from family, religion, law or social norms, other lines we drew ourselves as a result of experience, instincts of fairness, loyalty, love and anger, and intuitions that we’d struggle to explain. How we behave is seldom as simple as applying an ordered list of moral principles. We’re influenced by what other people are doing, whether other people are watching, and who we think will find out what we did.

We certainly don’t have one, unanimous set of Human Values ready to be inserted into a computer. So if we humans sometimes struggle to know what is right or wrong, how can we expect a machine to get the correct answer? This is a problem that Brian Christian is slow to address, though it could just be that he comes at the question almost from the point of view of an AI program.

Christian goes to a problem that can be expressed in mathematical terms to a machine: uncertainty. How do we make decisions with incomplete information? That is something that can be programmed. Just as the human brain turned out, after all, not to run on binary logic, computers designed to use probability, instead of deterministic pictures of the world, cope better with real conditions. Can this approach help either humans or robots make decisions when we are not sure what is right or wrong?

More from this author

Don't blame Big Tech for society's problems

By Timandra Harkness

Christian likes the example of the effective altruism movement, which uses a strong utilitarian approach to morality. The right decision is the one maximising the good that will result from the act. For example, Effective Altruism recommends funding the fight against malaria as the most good you can do with your money.

This approach lends itself more neatly to mathematical reasoning than the usual mess of categorical principles, consequentialist calculations and spontaneous intuitions that most of us use to make decisions. It also poses the question: what is “good”? But if we can agree on a broadly acceptable answer, it would be easy for a machine to apply.

AI designed to make optimal utilitarian decisions might result in more good overall, but it might also mean, for example, sacrificing innocent lives to save others. It would not always be aligned with our moral instincts, or with the way humans like to treat one another.

If moral values can’t be expressed as mathematics, can machines ever learn to share our values? Researchers trying to improve how machines learn stumbled on an illuminating insight. Stuart Russell was part of a team using Inverse Reinforcement Learning, a program designed to learn by observing what a human does, and infer what that person is trying to do. Building on IRL, his team developed Co-operative Inverse Reinforcement Learning, or CIRL. With CIRL, the robot takes on the (inferred) human goal as its own goal, but as helper, not usurper. Instead of driving, or flying, or playing Go, better than its human teacher, the AI joins the human’s support crew. The human’s goal becomes a shared goal. Machine and human goals are aligned.

This seemed to be a revelation for the researchers. “What if, instead of allowing machines to pursue their objectives, we insist that they pursue our objectives?” Christian quotes Russell as saying. “This is probably what we should have done all along.”

Humans are social. Moral values are social. Thinking is social. The whole of human society is based, not on a solitary thinker deriving everything from abstract reasoning, but on people co-operating, learning from each other, testing and revising our ideas and values, and persisting in profound disagreements some of which, in time, result in seismic changes in how we live together.

More from this author

The Government's Covid scaremongering

By Timandra Harkness

“Human Values” are not an abstraction living in one human mind, let alone one that can be programmed into an artificial intelligence. For machines to become aligned with human values, they would have to work alongside us, learning and adapting and changing, and they would still only align partially, with some of us more than others. And we would have to accept that.

But because we are social creatures, if we live alongside machines that learn from us and interact with us, will we not also be changed?

Christian briefly describes a future in which our machine helpers influence our behaviour, but this world is already here. Whether to sell us products or nudge us into healthier habits, our ubiquitous devices give us feedback and reinforcement learning, as much as the other way around. The more machines become our medium of relating to one another, the more we ourselves become understood as systems that are measurable and predictable in mathematical terms.

Alongside the history Christian tells, of AI as working models of the human mind, is a parallel shadow history, of human beings understood as machines. Our understanding of the brain, the mind, and the human person, have all been influenced over the last century by mathematics, logic and computer science. Aspects of human life that can be quantified, gamified, datafied, constitute the working model of ourselves for which products and policies are designed.

In the conclusion, Christian seems to recognise that this redefinition of humans, as walking neural nets being trained by the systems around us, is the real danger. While our attention is focused on the spectre of super-intelligent AI taking over the world, “We are in danger of losing control of the world not to AI or to machines as such but to models, to formal, often numerical specifications for what exists and for what we want.”

And, he might add, for what we are.

Exams are a blunt instrument. They assess performance on the day, not ability. But when they were cancelled, they left a gaping hole in an education system that depends on the grades they spit out. If only we had an oracle that could see into the mind of each student and judge them: a statistical model, objective, fair, and well-fed on data from every student in the country. So that’s what Ofqual built.

Then, around 40% of the A level grades awarded by the algorithm fell below the teacher predictions for the student in that subject. Cue teenage demonstrations and widespread political recrimination. But, contrary to what students may have expected, those teacher predictions were never the starting point for the awarded grades. In many cases, they didn’t even form part of the calculation.

More from this author

Was the two-metre rule one big lie?

By Timandra Harkness

Instead, the system was designed to give an overall distribution of grades that looked similar to previous years, with similar numbers of A*, A, and all the other grades for each subject — though they did allow more A and A* grades than usual. Ofqual even went so far as to check that the proportion of grades handed out to different subpopulations (by gender, ethnicity and deprivation, for example) would look similar to recent years. If your definition of fairness is that boys, or claimants of Free School Meals, won’t do demonstrably worse than last year, you should be happy.

Ofqual’s Direct Centre Performance model is based on the record of each centre (school or college) in the subject being assessed. Whatever the range and distribution of grades achieved by previous students over the last three years, that is the range of grades allocated to the class who would have taken A-levels in 2020. There was some adjustment, if your class has shown better (or worse) performance than its predecessors in GCSEs or other previous assessments, or if other changes would leave the national distribution of grades looking too different from previous years.

Meanwhile, each teacher was asked to rank each class from highest to lowest in expected achievement. That ranking, not predicted grades, was used to slot each student into the predetermined range of grades from A* to U. The exception to this was small groups, less than 15 in most cases, where Ofqual did resort to teachers’ predicted grades. That is why less popular subjects and smaller schools have seen less marking-down from expected grades.

Any one individual’s achievements so far, or their potential in the view of teachers who know them, had less influence on their eventual results than the attainments of others who attended the same school in past years. It’s closer to buying car insurance than taking an exam for which you have worked for nearly two years. Just enter postcode, make and model and we will predict your likelihood of making a claim, and hence your premium.

This wasn’t inevitable. “Any statistical algorithm embeds a range of judgments and choices; it is not simply a technically obvious and neutral procedure,” wrote the Royal Statistical Society (RSS) in a scathing statement published on 6 August. “Calibrating this year’s estimated grades to previous years’ exam results is one such choice. How to take account of evidence of individual students’ prior attainment is another. How to take account of uncertainty is another.”

More from this author

It's worth taking a risk over Covid-19

By Timandra Harkness

I asked Professor Guy Nason, Chair in Statistics at Imperial College, London, and fellow of the RSS, what Ofqual could have done differently. He was surprised that the UK had not attempted any kind of socially-distanced, in-person assessments, as some other European countries had done. But, given that some kind of statistical grade allocation was needed, Guy pointed out some specific pitfalls that could have been avoided, if a wider range of experts had been involved at an earlier stage.

“Overall, I think they ignored, or in some cases, underestimated uncertainty in many steps of their process. For example, the teacher-provided rankings for students within a subject were treated as if they were correct, when, in all likelihood, they are subject to considerable uncertainty. So, for example, tied rankings were not permitted, which might have resulted in students being assigned different grades even though their Centre thought that they were indistinguishable.” He also thought it was unfair to assess students by a completely different method if they happened to be part of a small class.

Nason had serious concerns about how Ofqual tested the predictive ability of their algorithm. “You’ve got to run a fair test, one that runs under the same conditions as the real deal.”

The normal way to test a predictive algorithm is to see how good it is at predicting the past. That is, you run the program for the previous year and see how well its predictions match what happened in real life. Ofqual did that for 2019, but because teachers in previous years were not asked to rank students, Ofqual could not use 2019’s teacher-generated rank orders for a test run.

Instead, it used the rank order that emerged from the 2019 exam results. Which is like showing you can predict the results of a horse race by including data about the order in which the horses crossed the finish line in that same race. “If a test uses aspects of the same data that it is trying to predict, then it results in a false sense of security,” says Nason.

Even by including some of the data they were trying to predict, Ofqual found their accuracy in predicting exact grades ranged from two thirds for History to one in four for Italian. For most non-language subjects, over nine in 10 students would be within one grade of the true result, but 3% of Maths students (for example) missing a fair result by two grades or more adds up to a lot of teenagers. Over 10% of Further Maths students, ironically the only ones who can understand the tortuous workings of the algorithm that betrayed them, would be over a grade away from a fair result.

“Their algorithm’s predictability is, especially for some subjects, not that good anyway, but if you then realise that they are over-optimistic and cannot be trusted, then one has to really question whether the algorithm is fit for purpose,” says Guy.

This isn’t just hindsight talking. The RSS offered to nominate two distinguished experts to the Ofqual technical advisory group in March. Guy was one of them. But Ofqual wanted to impose a Non-Disclosure Agreement that would bar them from public comment on the model for five years, in direct contradiction of the Society’s commitment to transparency and public trust.

Ofqual also ignored the House of Commons Education Select Committee’s call to publish details of their methods before releasing the results. They might have been spared some of the post-hoc dissection of their work, before public outcry and political pain caused Monday’s abandonment of the algorithm.

Because the grading algorithm has been withdrawn for political, not statistical reasons.

Its workings seem to have hit harder the very students who already felt the cards were stacked against them, and the communities to whom this government promised “levelling up”. Students in state schools and FE colleges, especially, and more deprived students, saw their awarded grades fall well short of their teacher-predicted grades.

It’s no surprise that small teaching groups, and less popular A-levels like Law, Ancient Greek and Music, are more common at private schools, which insulated those students from being marked down. Nor that previous results in selective and private schools would have been higher, bequeathing a higher range of grades to this year’s cohort.

More from this author

Can your smartphone crack Covid?

By Timandra Harkness

But the picture is messier than that. Historically, teachers in large state schools, and of more deprived students, have been more likely to over-predict. There may be good reasons for this. Teachers may consciously give a student the benefit of the doubt, figuring that it at least gives them a shot at a good university place. If they fall short, they can haggle later. If they exceed more honest expectations, it might be too late for them to raise their sights.

And in a high-attaining school where students routinely get A and A* exam results, there is not much headroom for over-optimism, unlike schools whose students walk away with the full range from A* to U.

Whatever the causes, that over-prediction means that every year state school students are more likely to find their exam results lower than their predicted grades. Universities are often flexible, recognising that it’s easier to get good grades in a good school, and that students who fought harder for OK results often do better at university than the ones who got good results in easier circumstances.

It’s bitter to be disappointed with your exam results. Perhaps, like me in those Stats exams, you turned over the paper and finally acknowledged, too late, how poorly your work matched the standards expected for the subject. But even if you were unlucky on the day with what was on the paper, or with your own state of mind, you still had your chance to do the best you could.

To find that a faceless system has allocated you to a lower grade, simply because your school hasn’t previously achieved much in this subject, looks like the epitome of systematic unfairness. Why did you bother to put in all that work, only to be pre-judged on the assumption that you’re homogenous with your older schoolmates?

That’s the embarrassing truth about algorithms. They are prejudice engines. Whenever an algorithm turns data from the past into a model, and projects that model into the future to be used for prediction, it is working on a number of assumptions. One of the more basic assumptions is that the future will look like the past and the present, in significant ways.

You may think you can beat the odds stacked against you by your low-attaining school, and your lack of extra-curricular extras, and your having to do homework perched on your bed in a shared bedroom, but the algorithm thinks otherwise. Isn’t it strange that we are repelled by prejudice in other contexts, but accept it when it’s automated?

Until now. Now school students shouting “Fuck The Algorithm” have forced a Government U-turn. Some of them seem to think the whole business was an elaborate ploy to punish the poor, instead of a clumsy attempt at automated fairness on a population scale. But some of them must be wondering what other algorithms are ignoring their human agency and excluding them from options in life because of what others did before them: Car insurance? Job adverts? Dating apps? Mortgage offers?

Some disgruntled Further Maths students will no doubt go on to write better algorithms, but that won’t solve the problem. As the RSS wrote to the Office of Statistical Regulation, “‘Fairness’ is not a statistical concept. Different and reasonable people will have different judgments about what is ‘fair’, both in general and about this particular issue.”

You don’t need Maths, or Further Maths, or even a 2:2 in Maths and Statistics, to question what assumptions are being designed into mathematical models that will affect your chances in life. Anyone can argue for their idea of what fairness means. Algorithms, and what we let them decide, are too important to be left to statisticians.

What helps us do these things is efficiency: the one thing of which we can never have too much. But what if our constant striving to streamline is actually having a negative effect?

That’s the argument advanced by Edward Tenner in his new book, The Efficiency Paradox. Efficiency is “something self-evidently desirable,” he says, “until it isn’t”.

His beef is with big data (in which large collections of information are analysed computationally to reveal patterns, trends and associations), which is great at helping us to do more with less, or even to do less with much less. It’s not so good at helping us do something new and hitherto unimagined –those things that extend our capabilities.

Suggested reading

"Sneckdowns" and desire lines: A parable for the big data age

By Peter Franklin

Things like flight. It wasn’t big data that invented the aeroplane, but risk-taking humans who took a chance on heavier-than-air flight being possible, in many cases throwing themselves from high places in pursuit of their dream. Once you have aeroplanes, efficiency – through the use of big data – can do wonderful things, such as minimise unwanted environmental effects, reduce prices to put flight within reach of ordinary people, and let us choose the seat we want online.

This is what Tenner, quoting Clayton Christensen, Derek van Bever and Bryan Mezue, calls “efficiency innovation”, typical of the platform-based disrupters such as Uber and Amazon, that “makes existing goods and services available to more people at lower prices”. But “market-creating innovation” that generates the new products or services, such as photocopiers or aeroplanes, takes longer to bear fruit.

Take the labour market. Optimists about the future of jobs in an increasingly automated age point to previous waves of mechanisation, when new industries arose to employ redundant workers. Cars created new jobs for people who previously worked with horses, for example, and then aviation created an entirely new industry that today employs about 10 million people worldwide, directly and indirectly.

Suggested reading

All hail the AI dictatorship

By James Bloodworth

To some extent, I share that optimism; but for that pattern to persist in future, we need market-creating innovation to keep creating new industries. Those bigger gambles that won’t pay off for years, if ever, are less appealing to investors than the quick hits of market-disrupting efficiency innovation.

The most valuable companies in the US in 1917 made their money from steel, oil and the telegraph or telephone business. Today, the top five worldwide by market value are all in data, including Facebook and Google, whose business model rests on delivering advertising with ever-greater efficiency.

You could argue that those platforms are providing radically new products and services, albeit as a side product. Certainly, their existence has transformed the way we communicate with each other and find information. We learn more every week about how much data they collect about us – their users – and yet we continue to use the platforms we know are digitally surveilling and profiling us.

Suggested reading

Only lateral thinking can save us from economic stagnation

By Peter Franklin

This means that our experience of news and of our friends’ opinions, of videos created by users like us, or of political campaigns, is significantly mediated by algorithms that filter the endless tide of digital output. When those algorithms are gamed to feed us untruths, or horrific live videos of a suicide or murder, we ask why they can’t be made more efficient in preventing that kind of abuse.

The problem is, as Tenner puts it, that these “algorithms are written neither to advance a political outlook nor to establish the truth, but to maximise traffic and increase advertising revenue for the platform company”. The efficiency that the algorithms are designed to optimise is not for the benefit either of society or of the individual service user.

But “efficient at what, and for whose benefit?” is only the first question we should be asking. Tenner’s argument goes much deeper, challenging whether efficiency in itself is achievable or even desirable.

Suggested reading

Can Britain build an innovation economy?

By Vicky Pryce

Let’s go back to the “market-creating innovation” that produces new products and services. How can we make that innovation process itself more efficient? “The paradox of efficiency,” says Tenner, “is that progress towards greater efficiency is wasteful.” To make leaps of insight and creative imagination, human beings need to be inefficient. We need boredom, serendipity and “desirable difficulty”. We understand concepts better when we have to work harder to get them.

Experiments with reading on paper versus onscreen, with handwritten versus typed note-taking, and even with more and less legible typefaces, point to this “desirable difficulty” as a help in grasping underlying ideas. If you want to win a pub quiz, browse the internet on your smartphone. But to grasp big ideas, read a paper book in an archaic font, or listen to a lecture while making notes by hand.

Tenner has many other examples of how pursuing efficiency through technology and data can lead to its opposite. Using wearable tech to monitor physical activity and nudge us towards healthier lifestyles turns out, in experiments, to reduce activity, weight loss, and even happiness and well-being. Time and energy saved by home automation through the Internet of Things will probably be expended in the battle to secure networks against hackers.

Suggested reading

In the age of Big Data, politicians should not believe their eyes

By Peter Franklin

None of this means we should turn our backs on the algorithms – only that we should not rush to defer to their mechanical judgment. “Balancing algorithm and intuition to achieve inspired inefficiency,” as Tenner puts it, can free us from repetitive and time-consuming tasks to be creative and inventive. Combining contextual knowledge of a subject with the processing power of search engines or data analysis can deliver breakthroughs in discovery or invention.

The danger comes when we outsource our judgment, underestimating our human minds, shaped through the messy process of life in human society, and overestimating what algorithms can do. We may be fallible but, unlike computers, we understand what it means to live in this world, and we can imagine other worlds in which we might live in the future.

If we never go beyond doing the same things with increasing efficiency, we are missing opportunities that are no less important for being hard to quantify. Inefficiency is an unlikely rallying cry for the digital age, but it opens doors we might otherwise not even see.

Chief among these is Lewis Mumford’s two volume Myth of the Machine (1966 and 1970) which examines the relationship between technology and culture from early man to now. This is everything Yuval Noah Harari has written in Sapiens or Homo Deus, and more (although Mumford is harder to read). Mumford was the pre-eminent tech critic of the 1960s and saw it as a dehumanising system of control.

In a similar vein, check out Neil Postman’s Amusing Ourselves to Death (1985). He argued that Huxley (distraction and entertainment) not Orwell (surveillance and boots) better predicted our dystopian future. The core idea is that the medium of communication dictates the sort of ideas and culture that can be sustained in a society (he was a student of philosopher and media theorist Marshall McLuhan after all). His focus on – and critique of – cable TV in the US is brilliant, and more relevant for social media than he could have predicted.

More from this author

How social media makes fascists of us all

By Jamie Bartlett

I find the radical fringes of thought are often a source of unexpected insight that mainstream writers dare not think or say. To that end, take a look at – and hold your breath for – the Unabomber’s Manifesto (1995). Ted Kascinski is a convicted terrorist who killed three people in a wave of anti-technology violence, and Manifesto is a rage against technology, which is in places violent and racist. But tucked inside are also fascinating speculations and predictions, some of which look surprisingly insightful twenty-five years on. (And don’t rule out a new wave of luddist violence – something I predicted here).

In a similar vein – Timothy C. May’s book-length blog ‘Cyphernomicon’ (1994) is a wild, bizarre, yet visionary, set of predictions about how encryption would transform society: crypto-currencies, dark net markets, ubiquitous encryption are all in there. May believed this would one day fatally weaken governments. He celebrated these trends; many of us will not.

Both of these works will shock and annoy you, but if you read them sceptically and carefully you’ll get a lot from them. And they’re both free to download.

More Christmas Reading

A survivor's guide to trauma

By Bidisha

Looking ahead rather than behind, two recent books set out a big problem heading our way. Both Cathy O’Neil’s Weapons of Math Destruction and Sayifa Umoja Noble’s Algorithms of Oppression (who both had decent stints in the private sector before writing these books) show how an increasing number of decisions are taken by smart machines and algorithms – but that doesn’t mean they will be less biased. In fact, they might simply replicate, or even worsen, existing economic, racial, gender or other inequalities, because they will appear objective and credible.

This problem – invisible algorithmic bias through search engines, or data discrimination – will become a major public preconception very soon, perhaps as soon as next year. The sooner we start thinking about it the better.