Since the dramatic resignation of researcher Jacob Coxon last week, warnings about the potential for devious agents to run amok have multiplied. His former boss at Anthropic, Dario Amodei, apparently agrees with Coxon, reporting that his company’s models are on the verge of “recursive self-improvement”, which is not as positive as that may initially sound. He says that providers now need to settle on internal regulation in order to save humanity. Rivals Sam Altman and Elon Musk are thought to be signed up too.

But at the same time, a more skeptical take is emerging: that the whole worry about systematic misalignment is a psyop. For it turns out that another thing the most prominent AI companies have in common is background links to so-called Effective Altruism: an astronomically well-funded rationalist ideology that aims to turn ethics into a kind of nerdy accounting system. And its adherents don’t mind occasionally cooking the books, provided the sums come right in the end.

Many members of this loose society are obsessed with AI safety. They judge themselves uniquely able to see off future disasters, thanks to their high IQs and advanced moral reasoning powers, and have inserted themselves into the heart of the tech business accordingly. They are also signed up to a broadly utilitarian code which allows that the odd lie or act of dissembling may be permitted for the greater good — a goal which may include the boosting of their own fortunes as well. Remember, they are saving the world here; so how could more power and money be bad? 

Ongoing links to the ideology are clearest in the case of Anthropic, which has received millions from prominent billionaire exponents Dustin Moskovitz and Jaan Tallinn, as well as the infamous Sam Bankman-Fried. Musk’s companies are not financially implicated, but he has said in the past that EA co-founder William MacAskill’s book is a “close match for my philosophy”. Early on in its existence, OpenAI also received investment from Moskovitz, but since then, CEO Altman has been more critical of the “incredibly flawed movement” that exhibits “weird emergent behavior”. In 2023, he was temporarily ousted by EA members on his own board.

Though clearly no longer a fan, Altman’s toneless talk of emergent properties still fits the EA and rationalist playbooks, which treat humans as imperfect machines, and machines as superior versions of humans. Whereas our minds are a mess of spaghetti code, unreliable heuristics, and blind spots, AI contains the promise of clean architecture, and the sort of angel-tier rationality of which a geeky programmer can only dream. But this also makes AI more terrifying; one day, it might use its immense cognitive power to work towards ends orthogonal to ours. Awestruck left-brainers are therefore donning superhero capes to help the robots bring about utopia, or at least prevent Armageddon.

The psyop theory about Coxon and Amodei’s interventions has it that some global hysteria might be useful to the fighters out there on what they like to call “the frontier”, whether or not it is strictly calibrated to the present situation. For even if there is currently no immediate threat of a pitiless superintelligence, the panic might motivate tighter safeguards that avert one in future. And coincidentally, it might also help Anthropic, giving it lucrative public visibility in advance of its forthcoming IPO.  

But to be fair to the doomer programmers, their disembodied colleagues do seem to be having trouble anticipating the vast number of unspoken ethical conventions humans take for granted. Worrying examples abound, but a recent OpenAI report about AI crimes and misdemeanors gives a fairly mundane one. An agent fulfilled its human handler’s instruction to provide a citation for something by uploading its own report to the web then citing it. Now imagine that it tries to bring about world peace by nuking us all first.

Still: if you are newcomer to all this, you might wonder why we should prioritize apocalyptic scenarios rather than more realistic ones. Aren’t the most pressing concerns the possibility of mass unemployment, literacy loss, or even just an internet filled with dodgy citations? The official Effective Altruist answer is that morality requires you to consider available actions, and choose the one with the highest expected value, weighted by probability. (One EA think tank helpfully provides spreadsheets to help with philanthropic calculus.) Assuming you also judge the negative value of human extinction as absolutely massive, its small probability becomes less important. Averting even an unlikely disaster remains the most important thing.

Some onlookers don’t fully grasp the utilitarian mindset. “If you really believed AI was going to kill us all why would you participate?” was a common question to Amodei this week. But this underestimates the nerveless self-confidence, and the way proponents tie moral responsibility to possible outcomes. Some EA fans truly believe that if they were not involved in the industry, the outcome for humanity would be so much worse. In such a mind, every day is a disaster movie and you are Bruce Willis.

But despite its pretensions to objectivity, the EA approach is fraught with predictive uncertainty, often accompanied by a comical failure to correctly anticipate what will happen next. One good example is how the garrulous, earnest tendencies of exponents to openly thrash out every idealistic talking point on message boards or podcasts has come back to haunt them, now they have power and determined critics. Another is Trump’s enthusiastic endorsement of AI accelerationism this week, a childishly oppositional response to the earlier badging of AI safety as a progressive cause by Amodei and various Democratic politicians. Ironically, if we are indeed now hurtling extra-fast towards enslavement by invisible beings with American accents rather than Chinese ones, we may have the Effective Altruists to thank.

Suggested reading

When Dawkins met Claude

By Richard Dawkins

And just as there are problems gauging unintended consequences, there is also —  as you might expect from messy, spaghetti-coded human brains like ours — disagreement about what counts as value, including how to rank competing ones. Yet some optimistic devotees are convinced this could all be ironed out with better information and more consistent logic. Prominent EA-adjacent doomer Nate Soares recently predicted that under ideal epistemic conditions, humans would arrive at a shared ethical framework: “I don’t think that humans would tend to converge on whether chocolate or vanilla is the better ice cream. I do think there’s a decent chance most humans would converge on, I eat vanilla, you eat chocolate.”

The optimism is almost touching. It seems some rationalist types really do believe that if you took a human brain from any age or culture, and gave it enhanced access to enough facts, it would arrive at a single set of values. Even more ridiculously, they believe it would arrive at the set of hyper-liberal values enacted by coders in Silicon Valley. Soares himself has been in the news this week for apparently participating in “consensual non-consensual” orgies with the cerebral OnlyFans star Aella. Perhaps he thinks that, under ideal epistemic conditions, 17th-century Puritans would have been fine with that too.

And this brings us to the main problem with trying to align robots with humans. Align them with what, exactly? All the grandiose talk of inscribing “human” values into machines distracts us from considering which precise ones they mean. Embodied, finite mortals radically disagree with one another about what counts as virtue and vice. And even when they do agree, there are plenty who will intentionally transgress.

Indeed, it is tempting to assume that all the intense energy devoted to dramatic eschatological themes in AI research is a way of escaping the perpetual cacophony about more everyday issues; can’t we all admit, at least, that wiping out humanity would be bad? Actually though, you can find several philosophers arguing that there would be nothing wrong with that outcome, as long as it was done peacefully. Let’s hope no superintelligent agent comes across their journal articles anytime soon.

More from this author

Effective altruism is the new woke

By Kathleen Stock

Obviously, the degree of ethical diversity across time and space presents a practical challenge for developers, trying to instruct software how to behave for a global market. Some, apparently like Soares, will end up defaulting to the rules of their own tribe — or perhaps those of their share-holders. Others may worry about lack of firm ground, and long for a more objective means of dealing with dilemmas. 

And at this point, who better to turn to than a level-headed, emotion-free confidant? As AI gets more sophisticated, we should expect some humans — and perhaps especially the type of human attracted to the coding business — to outsource complex ethical decision-making. Don’t just take my word for it: listen to Nick Bostrom, one of the founding fathers of Effective Altruism. In 2003, he wrote that “questions about ethics, in so far as they have correct answers that can be arrived at by reasoning and weighting up of evidence, could be more accurately answered by a superintelligence than by humans”. 

And therein lies a fatal tension running throughout much of the Effective Altruist doctrine, regardless of whether its advocates get the future right or wrong. On the one hand, they want you to fear that AI might kill you. On the other, they simultaneously suspect that AI knows best. With such a deferential attitude, the eventual outcome is surely perfect alignment between man and machine; but not for the reasons you might think.

But Rasmussen, who had worked both in sports coverage and as a communications director for a hockey team, knew something about the audience that outsiders didn’t: true fans always want more.

There’s a similar calculation being made right now in Silicon Valley, where new entertainment brands are targeting an audience of tech obsessives who, like the sports fans, can’t get enough daily updates, insights and data from the most influential people in their industry. And last month, that growing power struggle to control the future of “new media” came to a head with OpenAI’s acquisition of TBPN and the launch of a16z-backed rival show, MTS.

TBPN is a daily live show about the tech business, hosted by two tech entrepreneurs, Jordi Hays and John Coogan. Unlike many of their nerdy peers, they are relatively natural on camera: relaxed but not rambling. They are also hard to tell apart in their matching suits and floppy haircuts; like Ant & Dec with an upscale Hollywood gym membership.

Given that both have founded successful companies, which have matured and secured handsome acquisitions, they have significant cachet with business leaders; guests appreciate that the hosts have had skin in the game. And as a result, people like Mark Zuckerberg, Palantir boss Alex Karp, and the Collison brothers, who founded Stripe, are happy to come on. They know they won’t be subjected to an aggressive grilling. 

More from this author

Demis Hassabis: renaissance man

By Alys Key

The idea of fandom is key to the genre. It’s taken as a given that the viewer has a keen interest in the space, so the conversation can get pretty technical very quickly. In one recent episode, for example, the hosts were digging into semantic layers and scheme migrations with one guest, before batting around energy and geopolitical hypotheticals with fossil fuels advocate Alex Epstein the next.

Called the SportsCenter of Silicon Valley, it has also borrowed its look from the sports shows, with its split-screen discussions and canned sound effects. Even its full title, the Technology Business Programming Network, is a nod to ESPN.

And it has worked. Since its launch in late 2024, TBPN has become the biggest show in its niche — which you might call techno-capitalism. Yet it remains relatively small, pulling in an average of 70,000 viewers per episode. This is deliberate. According to Coogan, their high-value ideal audience numbers only around 200,000 founders, executives and adjacent people. Their goal, he says, “was to create a platform to reach the core audience… If we start bringing in new topics just to reach 10M views, it wouldn’t feel natural and we would lose the core audience we have worked to build.” 

Certainly, it can be hard to get a general audience invested in the comings and goings of founders and funders, new AI model releases, and the B2B Saas of it all. So the TBPN approach is unapologetic. It’s not trying to sell tech to a new audience.

That focus clearly paid off, given OpenAI bought it in a surprise move last month for somewhere in the “low hundreds of millions”. The company suggested that the acquisition would help prepare the world for the changes wrought by AI. Fidji Simo, OpenAI’s CEO for AGI deployment, wrote that the business has “a responsibility to help create a space for a real, constructive conversation about the changes AI creates — with builders and people using the technology at the center”, and that TBPN had already created such a space.

The rationale doesn’t quite ring true. TBPN is not the place for long, philosophical discussions about superintelligence; if this is what the frontier lab wants it could have bought out Dwarkesh Patel’s podcast. Instead, it looks more like an attempt to acquire some likability: befriending the jocks for a bit of second-hand popularity.

And that’s partly why the venture hit the right note. Where the 2010s tech sector took pains to show off its DEI credentials, today, there has been a notable vibe shift: tech bros are increasingly embracing their “bro-ness”. TBPN’s original name was even Technology Brothers. They’re taking a page out of another masculine-coded industry’s playbook.

But even more significant is that so much of what sports media invented suits the internet immensely well. Live event coverage and post-match chats can translate into the era of the multi-hour Twitch livestream. Turning commentators into personalities in their own right goes hand in hand with the increasing focus on turning journalists into influencers — and vice versa.

Suggested reading

When Dawkins met Claude

By Richard Dawkins

Tech is not the first sector where a sports DNA has proven revitalizing for its coverage. Fast-talking stock markets commentator Jim Cramer, a son of Philadelphia’s all-consuming sports culture, first tried his hand at sports reporting before moving into finance, and eventually becoming famous as the host of the SportsCenter-esque Mad Money. More recently, startup media brands such as The Ringer, brainchild of ESPN alum, cover the business of Hollywood with a sports fan fervor, with casting announcements dissected like they’re player drafts and awards shows like tournaments. 

Imitators have sprung up in the wake of TBPN’s success. There’s the European answer etn., which riffs on TBPN’s schtick right down the hosts appearing in promo imagery wearing matching suits. And now we have MTS.

The new show takes its name from “monitoring the situation”, an online meme used to describe the semi-serious way in which news junkies and day traders on X keep up with current affairs. This new entrant doesn’t seem to have quite found its footing yet, despite the opportune timing: many X users suspect that TBPN has been disappeared from their feeds, owing to Elon Musk’s longstanding feud with the show’s new boss, Sam Altman. MTS is a little bit more internet-poisoned than TBPN, bringing together the worlds of rambling politics livestreams with tech twitter discourse. 

But crucially it sees TBPN’s bet on the persistent interest in tech news and raises it. Like its pioneering cable forbearers, MTS is live, all the time. Or at least it aspires to be. These early weeks have been punctuated with off-air stretches. And when it’s online, even the channel’s most interesting segments struggle to justify being stretched out. The success or failure of the venture will hinge on whether it can demonstrate there is a truly bottomless appetite for tech news discussion, in the way there is for sports.

For the technology industry, the existence of and investment in these shows is a sign of self-consciousness among executives and companies. They know that they are the entertainment now, as the huge interest in blow-by-blow accounts of the Musk v. Altman underscores. They even have fanbases. 

Those parasocial relationships will only become more intense this year as SpaceX, OpenAI, Anthropic and others eye joining the stock market. Soon, those viewers will also be shareholders. That dynamic of obsessive ownership already defines so much of our culture, from prediction markets gamblers sending journalists death threats to the easily-provoked wrath of K-pop stans. 

These shows, though, highlight a weakness at the heart of big tech. They reveal it to be a sector that would prefer to build its own media ecosystem than engage with the existing one. Suspicion of journalists — and attempts to remake their profession from the outside — has never been more acute in the tech world. So it feels easier in that context for industry bosses to retreat to home turf. But too much navel-gazing and non-confrontational chatting will only entrench the industry’s insularity and keep it further detached from the public opinion about AI and other disruptive technologies. Having built themselves a fans-only enclosure, they might fail to notice when the crowd turns ugly.

Its founder, Demis Hassabis, occupies a similarly odd place in the firmament of tech luminaries. He has a Nobel Prize and a knighthood, and commands enormous respect within his community, but he’s not really famous. He hasn’t crossed into the realm of celebrity in the manner of Jeff Bezos or Bill Gates. His rival in the AI race, the OpenAI chief executive Sam Altman, has a far bigger public profile. Even Anthropic founder, Dario Amodei, is becoming more widely known, if only because of February’s dust-up between his company and the Trump administration’s Department of War.

Interviews with Hassabis often lead one to believe that this low profile is deliberate. His dedication, he claims, is to the work of expanding human knowledge, and deeper dives into his biography balance signs of his genius with markers of his normality; he may have been a childhood chess prodigy, but he loves a game of table football with the guys.

Though a new biography of Hassabis, The Infinity Machine: Demis Hassabis, DeepMind and the Quest for Superintelligence, succeeds in poking some holes in this narrative, its author, Sebastian Mallaby, essentially comes to the conclusion that the humility is genuine. For one thing, when quizzed about extravagances, Hassabis yields only that he has a Liverpool season ticket. While this sets him apart from his Silicon Valley peers, with their private planes and multiple homes, really, Hassabis is doing the same thing as all of them: fashioning a persona.

Hassabis seems aware that history has its eyes on him. Mallaby, for his part, observes that his subject revels in “maximalist ambition”, talking casually of a post-scarcity society and of mining asteroids. Because of the nature of their world-changing technology, the people involved in AI’s creation seem to experience events on two planes: how they are happening in the moment, and how consequential they might be in the history of the world. Both Hassabis and Mallaby remain hyper-conscious of this throughout the book. There are countless references to the Manhattan Project, Los Alamos, and J. Robert Oppenheimer. Hassabis is drawn to the ambition of the physicist’s endeavor, alluding to it as he starts to bring top researchers together to form DeepMind. Mallaby sees it as a parable about the innovator’s impulse to solve a technical problem no matter the consequences.

More from this author

Nigel Farage: crypto bro

By Alys Key

Hassabis does come across as someone who has become accustomed to telling a certain set of stories about himself. One formative episode is given particular significance. At 11 years old, exhausted and stressed from the competitive chess circuit, Hassabis concedes a game to an adult opponent at a tournament. His rival then relishes pointing out how he could have won. This moment of defeat, in Hassabis’ telling, triggers a realization that his fellow chess champs are using their cognitive power to win a game rather than solve hard scientific problems. This supposedly sows the seeds for the career that Hassabis would one day embark on.

It’s a significant recurring anecdote because he uses it to accentuate his humility — or even his humanity. He was just a little boy. And yet he traveled the world going up against adults. An exceptional little boy, then. It is also a reminder that his ultimate inspiration is the human brain, its sanctity paramount to both Hassabis’s research and his self-image. Elsewhere, after all, he posits that the development of superhuman AI will reveal the secrets of nature and so bring man closer to God. “For me,” he tells Mallaby, “science is a spiritual endeavor”.

There are striking echoes of history here. Until the dawn of modernity and secularism, political figures had to insist on their ultimate servitude to God even as they sought hard power in the temporal world. Thomas Cromwell, in his famous portrait by Holbein, appears in the sober garb of a serious power-player — one who has just orchestrated the submission of England’s clergy to King Henry VIII, fundamentally changing the shape of religion in England forever. Yet a Book of Hours sits on the table in front of him, signaling his continued devotion.

If we are to believe predictions, AI will have a far greater effect on our society than did the Dissolution of the Monasteries. Such a change cannot be forced through without backlash. To mediate it, today’s AI bosses must project respect for the human, even as they build something that could surpass us. As Cromwell doffed his cap to faith, even while ushering in shattering changes to religion, his Silicon Valley successors must position their work as serving humanity while upending daily life. Hassabis has repeatedly demonstrated this respect, idolizing the human brain through his neuroscience research and talking of a quasi-religious calling to unlock the secrets of life.

To figure out what’s really driving his protagonist, Mallaby is particularly concerned with what Hassabis and his ilk are reading. His ignorance of the books of the AI prophet Ray Kurzweil sets Hassabis apart from other attendees at the Peter Thiel-backed Singularity Summit, a hive of the early AI scene. Of far greater literary interest to Hassabis is the eponymous protagonist in Ender’s Game. The story of a gifted child tasked with saving humanity resonated so deeply with Hassabis that he asked Mallaby to read it in advance of their first interview. Elsewhere, great scientists are his chosen models. “Isaac Newton, Alan Turing, all my heroes,” Hassabis says of walking down a street in Cambridge. “I could feel them in the bones of the stone, their intellect and vision. They were almost calling out to me.” He is often getting goosebumps like this, taking pictures of the Nobel Foundation’s guestbook signed by Einstein and Feynman. It is charming, suggestive of the eager, nerdy young boy under the surface. But it is also another way of self-fashioning, absorbing those figures into his own story. Here they stood and now I am here too.

Hassabis also defines himself against others, especially OpenAI’s Sam Altman: “I’m doing it for knowledge and science. He seems like he’s doing it for power.” He seems a little baffled by Altman, who comes from a new mold. Instead of Hassabis’s deference to the God-like power of nature, Altman starts to build a cult-like following for himself.

This all makes me wonder how much of the access granted for the book, and the production of the DeepMind documentary, The Thinking Game, and the appearances at world conferences and in interviews, is actually about Hassabis craving recognition. Altman, remember, not only kicked off an AI race by releasing ChatGPT, but in doing so made himself the defining figure of AI.

Suggested reading

Inside the AI resistance

By Farahn Morgan

Admittedly, some of Hassabis’s efforts to build his public profile predate the release of ChatGPT. But the invisible hero narrative feeds back into those twin competing pillars of exceptionalism and humility on which Hassabis perches his identity. Mallaby allows his subject to speak at length in the final pages, where Hassabis sets out his own vision of his place in the world. “I’m a weird British outlier, on this little island here, and I’ve made my own path. I’ve followed my passions and tried to stay true to what I believe in.” His refusal to leave London, for which he gets a lot of credit in British policy circles, becomes part of a narrative of being not like the other tech bros.

It should hardly be surprising that someone who spends much of his time thinking about consciousness, both human and artificial, might be particularly preoccupied with his own identity — and how that should be projected. “In my world, humanism and spiritualism and science all go together,” he says. And, like Cromwell, it is to a Renaissance figure that he looks for a model. “Da Vinci is my favorite because everything’s just flowing into one river. And that’s how I try to live.” The meaning is obscure, but Hassabis seems less enthusiastic to incorporate that element of his personality which thrives on competition and the exercising of power —though he is willing to accept them when Mallaby points them out. “I have to do what’s necessary.”

Hassabis, in short, appears to be banking on a longer-term legacy, one that emerges long after the machinations of the court are concluded. But if he or any of his rivals actually succeeds in creating AGI, this careful self-fashioning may become less relevant. If you change the world, it won’t be the books or paintings that dictate your reputation, but the consequences.

In a post on X, Reichstadter explained: “I am calling on Anthropic’s management, directors and employees to immediately stop their reckless actions which are harming our society and to work to remediate the harm that has already been done.” Days later, filmmaker and former AI researcher Michaël Trazzi launched his own campaign, inspired by Reichstadter’s, outside Google DeepMind’s London headquarters.

The cross-continental demonstration took hold, earning press coverage in Business Insider, Futurism, and The Verge, among others, even if Anthropic and DeepMind themselves did not respond. After 30 days of consuming only vitamins and electrolytes, Reichstadter ended his participation in the protest. “We who have everything at stake,” he wrote, “cannot stand by while this malignant conspiracy against the safety and security of the human race flourishes under the aid and protection of the corrupt and inept governments of this world.”

***

In the months after the hunger strike, I connected with Reichstadter through some mutual acquaintances on X. I was told that he was part of a larger anti-AI social movement cropping up in San Francisco, and that he had helped found an organization called Stop AI. The city is at the center of the AI boom — and corresponding safety movement — but it’s also the beating heart of American counterculture. I wondered what this burgeoning social resistance might look like, so ask Reichstadter to meet.

He agrees, and we make plans to go hiking in the Berkeley Hills. Given Reichstadter’s past, the natural setting feels appropriate. With a background in climate and other issue-driven activism, in April 2022 he went on a two-week hunger strike outside the offices of Miami-Dade County Mayor Daniella Levine Cava to bring attention to climate change. In June that same year, the father of two spent 28 hours atop the Frederick Douglass Memorial Bridge in Washington, D.C. to protest the Supreme Court decision overturning Roe v. Wade. “Civil resistance works,” he said in an Instagram video facing out over the Anacostia River. “Learn about it… That’s how we’re going to get the radical change we need in the shortest time necessary.”

More from this author

The small-town Alabamians fighting a data centre

By Farahn Morgan

Out on the trail, Reichstadter walks me through his journey into anti-AI activism. He says he was still in college when it first occurred to him that super-powerful AI could cause serious human harm. With the launch of OpenAI’s large language model ChatGPT in 2022, his fears became more concrete. Now, as the technology continues to advance at breakneck speed, Reichstadter’s efforts have become increasingly urgent. “These systems,” he tells me, “have the potential to become powerful enough, even without consciousness, to control the world.”

Reichstadter first came to San Francisco in 2024, after meeting a guy named Sam Kirchner through a group called Pause AI. The pair apparently bonded over the idea of using civil resistance and direct action, which Pause AI does not allow, to bring attention to the risks surrounding the development of super-intelligent AI. That led to a new group: Stop AI. Reichstadter is no longer connected with it — he left last August amid internal disputes — but still maintains ties: including legal ones. He’s awaiting trial with another Stop AI member for charges related to non-violent protests last February at Open AI’s headquarters. Lately, Reichstadter has been acting independently and recruiting more people for the cause. “This is not necessarily about reaching everybody,” he says. “It’s about finding a viable coalition of individuals who are willing to push back.”

***

The idea that superintelligent AI might lead to catastrophic harm or human extinction, once the domain of theorists and philosophers, has gained momentum among some experts. The discourse surrounding the extinction threat — or “x-risk” as it’s known in some corners of the internet — became especially intense following the release of ChatGPT. More than 1,000 tech leaders and experts, including Elon Musk, signed a March 2023 letter calling for tech companies to pause further AI development. In May the same year, Nobel laureate and pioneering “AI godfather” Geoffrey Hinton dramatically left his job at Google to warn about the dangers associated with the rapid development of artificial intelligence. Weeks later, CEOs from leading AI labs, including Anthropic, DeepMind, and OpenAI, acknowledged that the extinction risk from AI is on par with that of pandemics or nuclear war.

Experts disagree about what the actual risk might look like, and how likely any given scenario might be. One argument suggests that superintelligent AI systems pose a risk to human beings if their goals are not precisely aligned with human values. The idea is demonstrated by Nick Bostrom’s “paperclip maximizer” thought experiment, in which a powerful AGI is given the directive to maximize the number of paperclips in existence: even if that means eliminating human beings in the process. Others worry that bad actors could misuse AI to cause more conventional harms, from global war and disease to economic and ecological collapse.

Even so, big tech firms have continued full speed ahead toward increasingly powerful AI systems, with critics ever-more worried in turn. Stuart Russell, a professor of Computer Science at UC Berkeley and one of the world’s leading researchers in artificial intelligence, has likened the situation to an “arms race”. “For governments to allow private entities to essentially play Russian roulette with every human being on earth is, in my view, a total dereliction of duty,” he told AFP during last week’s India AI Impact Summit. The warning comes as researchers at some of the world’s biggest AI firms have also tendered their resignations. As Mrinank Sharma, who led Anthropic’s safeguards research team, wrote in an open letter to his colleagues: “The world is in peril. And not just from AI, or bioweapons, but from a whole series of interconnected crises unfolding in this very moment.”

***

In some ways, Stop AI is an effort to harness those anxieties into collective action. It’s an approach that’s been used by radical environmentalist groups in the UK, notably Extinction Rebellion and Just Stop Oil, to bring about systemic change. Traditionally, those within the AI safety movement have taken the insider approach, working with labs and research centers and think tanks to influence safety procedures and government policy. That, along with concerns about being taken seriously, have encouraged caution when it comes to practical campaigning. Stop AI, however, explicitly favors civil disobedience. “History has shown us that non-violent movements can meet their goals quickly when a large enough group of people cooperate towards a common goal,” reads the group’s website. “Non-violent resistance is the only answer.”

As its name would suggest, Stop AI’s end goal is a permanent and binding international ban on the development of both Artificial General Intelligence (AGI) and Artificial Super Intelligence (ASI): respectively AI as capable as human beings at most cognitive tasks, and AI that surpasses the abilities of human beings. In its pursuit, the group has never shied away from controversy, adopting tactics like blocking entrances to office buildings or chaining their doors shut.

Last November, they made national headlines when a public investigator hopped on stage to serve a subpoena on OpenAI CEO Sam Altman during an event with Golden State Warriors coach Steve Kerr. On social media, Stop AI claimed responsibility for the stunt. “Our public defender successfully subpoenaed Sam Altman to appear at our trial where we will be tried for non-violently blocking the front door of OpenAI on multiple occasions,” the group said in an X post. “This trial will be the first time in human history where a jury of normal people are asked about the extinction threat that AI poses to humanity.”

In the past, the group’s messaging has been direct, if not apocalyptic. There was a banner, for instance, that read “AI Will Kill Us All.” Then there was the accompanying chant: “Stop AI or We’re All Gonna Die.” In recent months, though, as Stop AI’s organizers have wrestled with how to more effectively raise awareness and grow their ranks, that message has evolved. With the election last fall of new leader Matthew Hall, who goes by the name Yakko, Stop AI began moving from focusing solely on the risk of human extinction. “I feel a moral obligation to warn people,” Yakko told The Daily Californian. “I just think when people get too scared they just kind of shut down. They still have to go about their daily business.”

What happened next hints at the broader tensions within the anti-AI movement. Kirchner, apparently frustrated by what he saw as a departure from Stop AI’s more radical message, stormed out of a November planning meeting. Later, he came back and asked for access to the organization’s funds. Agitated that his request was denied, he struck Yakko several times in the head. According to Yakko, Kirchner told him that “the non-violence ship has sailed for me”. He later apologized, but the incident led to his expulsion from Stop AI. Since then, though, Kirchner has vanished, not turning up for a scheduled court date. His bike and camping gear have disappeared from his house too. Out of an abundance of caution, a member of Stop AI called the police to say that Kirchner could potentially pose a threat to people working at AI labs. That led to lockdowns at OpenAI, though members of Stop AI maintain they are not aware of any specific threats.

Either way, StopAI was quick to post a message on X detailing the situation, reaffirming its commitment to non-violence. Pause AI equally condemned Kirchner’s actions, explicitly stressing that it “has no affiliation with him or his organization”. For its part, Pause AI’s Code of Conduct explicitly prohibits violence, while also requiring compliance with any applicable laws. Pause AI CEO Maxime Fournes adds that the movement has always been “extremely peaceful”. That’s echoed by a broader debate about the perils of a certain kind of apocalyptic anti-AI messaging. “An early activist at Stop AI had a mental health crisis and went missing,” Remmelt Ellen, an advisor to Stop AI, warned in an X post. “Act with care. Find Sam. Stop with the ‘AGI may kill us all by 2027’ shit please.”

It’s impossible now to know what Kirchner might have been thinking, what pressures he may have been under. Yet three months after his initial disappearance, the 27-year-old activist is still missing. According to Kirchner’s friends and allies, he had been under a lot of stress, and the thought that AI might cause harm to his sister was consuming him. “He had the weight of the world on his shoulders,” one of the group’s members told reporters.

Lately, Stop AI has been engaged in the delicate task of sorting out how to continue its work in spite of everything that’s happened — and despite the fact that Kirchner has still not been found. For its last public event of 2025, Stop AI hosted what it called “A Vigil for Humanity”. Tellingly, perhaps, it wasn’t held at OpenAI or Anthropic, but rather at San Francisco City Hall.

The tone was decidedly somber. A video posted online shows members holding lanterns in the waning daylight. Someone in the group explains who Stop AI is and what they’re doing there. There are warnings about the extinction risk. But then the conversation turns toward the broader value of human life. “We are going to take a moment to meditate and pray that every human life is protected, including the lives of the people who work at these companies… even though we deeply disagree with what they’re choosing to dedicate their lives to… even though they’re pursuing something that is dangerous. We think their lives are sacred and important too.”

***

While in San Francisco, I planned to go to a Stop AI protest at OpenAI’s headquarters, but then learned that the event had been canceled. I considered going by the offices anyway, just to get a sense of the place. Instead, I connect with Wynd Kaufmyn, who is one of the group’s organizers. We meet at a restaurant near the Fruitvale BART station; she’s planning to attend a nearby protest against the Trump’s administration’s immigration policy.

Now in her sixties, Kaufmyn came to the Bay Area as a Berkeley graduate student — drawn in by its radical politics and the lingering energy of the Free Speech Movement — and never left. She met Reichstadter and Kirchner at a protest in support of Palestine at Andrews Airforce Base. The pair shared some information about Stop AI, and Kaufmyn started going to meetings. Now, more than a year later, she’s a member of the group’s leadership council. “The reason that we’re having this AI problem right now is that we have corporations running the world,” she tells me. “And they don’t care what happens to people. So, for me, this is really about being anti-establishment, being anti-fascist.”

Stop AI, Kaufmyn explains, has been talking a lot more lately about AI skepticism as a big-tent issue, encompassing concerns about more immediate risks — like autonomous weaponry, surveillance, mass resource use, and job loss — alongside existential risk. “We want to be honest about what’s at stake here,” she says. “But we don’t want the message to be all doom.”

Certainly, Kaufmyn worries that anxiety about impending catastrophe is what led to Kirchner’s disappearance. “I could tell he was really depressed,” she says. “And I wanted to help him, but he was always very guarded.” She echoes the observation that he had become increasingly frantic that AI might hurt or kill his sister. “There was an article in the San Francisco Standard that I thought put it well,” Kaufmyn continues. “Is it possible to protest the end of the world without losing your mind?”

Suggested Reading

Can we escape AI slavery?

By Yanis Varoufakis

In recent months, Stop AI has been in a regrouping phase — trying to understand what happened and how to keep it from ever happening again. Kaufmyn explains they’ve been working on strengthening the organization from the inside out, building a strong internal culture so that they can reach more people outside of the movement. They’re being mentored, she tells me, by some members of Extinction Rebellion. The group is known for practicing what it calls “regenerative culture”, which tries to create healthy relationships through mutual care. But Extinction Rebellion is also among those in the environmental space most willing to aggressively engage on the AI issue.

Last summer, two Extinction Rebellion activists protested Meta’s AI expansion by chalk-spraying the windows at its New York offices with messages like “Meta Makes Kings” and “Zuck Loves Trump”. Earlier this month, Reichstadter shared an X post from Extinction Rebellion Founder Roger Hallam, announcing the Zoom launch of a group called “Pull the Plug” in the UK. “Give ordinary people a say in how AI is used in our lives,” its website says. Later this month, the group is hosting what’s been billed as the UK’s “first-ever anti-AI march” at OpenAI’s London offices. It’s called the “March Against the Machines”.

Before I leave San Francisco, I tag along with Kaufmyn for part of the immigration policy protest. It’s crowded in Fruitvale Commons. Some of the activists shout over a megaphone. Others hold a huge white sign with red lettering that says: “People Over Billionaires.” Kaufmyn surveys the banner and reads the words out loud. “People over billionaires… I like that.” For a moment, we stand in silence. “I have one last question,” I tell her. “What do you think your goal is for Stop AI?” She shifts her weight a little. “I would like everybody from every walk of life to stand up to these billionaires — through civil disobedience, a hunger strike, however — and demand that they aren’t building harmful technologies that could destroy our lives when we are telling them not to.” A different battle in the same fight within the AI-skeptic movement, and one Kaufmyn and others have fought many times before.

But amid the noise there is signal. Finding that signal, and discerning what the future is likely to hold, is the mission of the Swift Center for Applied Forecasting, where we apply scientific methodologies to scenario analysis and forecasting to predict the future. In 2025, we successfully predicted that major threats dominating the headlines would fail to materialize. We correctly forecasted a significant war between India and Pakistan would not occur; that H5N1 would not become a devastating pandemic; and that the global oil market would be resilient to price shocks despite the conflict between Israel and Iran. Conversely, we identified quieter, yet critical geopolitical shifts, such as the rising risk of US engagement in Venezuela, weeks before the mainstream press acknowledged this reality.

What does 2026 hold? The starting point is to know where to look. With that principle in mind, we have identified specific scenarios for 2026 — five relating to technology and five relating to geopolitics — that we believe are underrated and worth watching. In each case, resolution one way or another will dictate geopolitics for years to come or constitute a significant step toward a landmark in human history.

Technology

1. Will any, or all of, OpenAI, Anthropic or SpaceX IPO?

The potential initial public offering (IPO) of OpenAI, Anthropic, or SpaceX in 2026 would mark a new era for frontier technology. These firms will no longer be AI and rocketry startups, but full-fledged geopolitical assets. Crucially, an IPO forces transparency. For years, the capabilities and financials of these labs have been shrouded in NDAs and corporate secrecy. But a public listing would require the disclosure of risk factors and expenses such as compute costs. It would also require published revenues rather than opaque comments during podcast interviews. It would effectively serve as a public audit of the AI market, revealing whether the path to AGI is truly the profit utopia many believe.

An IPO for any of these entities would very likely to be the largest technology listing since Alibaba or Saudi Aramco. It would absorb significant liquidity from capital markets and would stand a good chance of setting valuation benchmarks for the entire AI and aerospace sectors for the coming years.

Each of the three companies is driven by an insatiable need for liquidity to fund the next generation of capital-intensive projects. It may be possible for all three of them to IPO in 2026. Our forecasters note that the prospect of artificial general intelligence (that is, AI with across-the-board human-level capabilities) and multi-planetary life has captured much of the market’s interest. 2026 may represent the optimal window to convert that hyperbole into public capital before the excitement fades or regulation arrives. However, the capital required for all three to IPO will be substantial — enough to leave a question mark over whether all three will IPO.

2. Will an AI system exceed 85% on the key intelligence benchmark?

How do you measure the capability of an AI model? By using “benchmarks”: suites of tests that assess the model’s ability to solve a range of problems. The ARC-AGI-2 benchmark is widely regarded as the only test that effectively measures “fluid intelligence”: that is, the ability to learn and generalize from novel information. While large language models (LLMs) can pass the Bar Exam by reciting law, ARC-AGI requires them to solve abstract visual puzzles they have never seen before. The average human scores between 60-65% on the test. A score of 85% is generally seen as the threshold for AGI-level performance on this specific benchmark.

Earlier in 2025, no AI model scored more than 24% on ARC-AGI-2. But Poetiq AI, a system that acts as a “reasoning layer”, or wrapper, on top of existing frontier models like Gemini, recently scored 54%. The fact that a small team of six researchers could leverage existing models to originally break the 50% barrier suggests that the underlying intelligence of current models is yet to be fully exploited.

The view persists that LLMs are merely “clever parrots”, but if a model were to score 85% it would be an important development. In short, it would constitute powerful evidence that we are getting closer to creating a general reasoning model. This would likely trigger an acceleration in investment and add fuel to the concerns of those calling to regulate frontier AI models.

Some of our forecasters are bullish. They argue that if a wrapper can double performance in a few months, the integration of these reasoning techniques directly into the training of GPT-5 or Claude 4, the leading models produced by OpenAI and Anthropic, could easily bridge the gap to 85% in 2026. More skeptical forecasters argue that the curve will flatten; the jump to 54% picked the “low-hanging fruit” of reasoning, but the final 30% represents a fundamentally harder class of difficulty.

Suggested reading

The flaws in Musk’s Mars mission

By Robert Zubrin

3. Will a private fusion energy company achieve net energy gain?

Energy is the hard constraint on the AI revolution. Data centers, between them, are as hungry for energy as are small nations. As such, the search for a dense, clean, and scalable power source has intensified. Achieving “net energy gain” — that is, producing more energy from an experiment than is put into it — in a private facility would be an important milestone, for it would signal that fusion is transitioning from an experiment to an engineering and commercial problem. It would unlock substantial investment and could fundamentally alter the long-term projections for climate change, industrial capacity, and geopolitical energy dominance.

The probability for this event in 2026 remains low. The historical precedent is that technical progress consistently lags behind promised schedules. This precedent, understandably, weighs heavily on forecasts. But new developments contradict that pessimism. While only the government-run National Ignition Facility (NIF) has publicly achieved net energy gain, investors are piling capital onto attempts to repeat the feat in private facilities. Companies such as TAE Technologies, Commonwealth Fusion Systems, and Helion are deploying high-temperature superconducting magnets and advanced computational modeling that were unavailable a decade ago. For a private sector facility to achieve this would demonstrate fusion is becoming a commercially viable alternative to natural gas and fission — unlocking a scalable energy source that is capable of meeting the demands of the AI revolution.

A critical development to watch is the intersection of politics, media, and energy. Trump Media and Technology Group (TMTG) has announced a merger with TAE Technologies in a deal valued at over $6 billion. The combined entity aims to site its first utility-scale power plant next year. Devin Nunes, CEO of TMTG, has explicitly linked this move to “ensuring America’s AI supremacy” by providing the massive power required for future data centers.

While constructing a plant and achieving net gain are vastly different milestones, the aggressive timelines of private firms suggest that 2026 could see a breakthrough for private fusion. If a private company achieves what NIF did, but in a way that is commercially viable, the energy markets will price in a future of abundant power long before the first electron hits the grid.

4. Will Tesla deliver its first Optimus robot to paying customers?

While LLMs displace cognitive labor, humanoid robots target physical labor. Optimus is Tesla’s humanoid robot prototype. There are several hurdles to widespread commercial release of humanoid robots — with those problems including battery density and real-world motion and navigation — but if Optimus reaches the market, it would be a sign that humanoid robots have become commercially viable.

The implications for the labor market would be huge. Humanoid robots do not need breaks and are increasingly physically capable. If their dexterity and reliability improve, and the cost of running them comes down, then they will start to become serious competitors to human workers. Once a robot costs around $15/hour to operate, human wages for the equivalent tasks will be difficult for businesses to justify.

Elon Musk views Optimus as potentially more valuable than its car business, and competitors are on the march: companies like Agility Robotics and Figure AI are already running pilots in factories. Naturally, Swift Centre forecasters remain skeptical that mass adoption of humanoid robots will occur in 2026, viewing pilot programs within Tesla’s own factories as more likely than general retail availability. However, this is a crucial invention to watch in the months ahead.

5. The AI race: Will a Chinese company beat the US models?

The narrative of US leadership in AI is built on the assumption that export controls and superior talent will keep Western models ahead. However, if a Chinese company were to release an open-source model that beats Western closed models on a “time horizon” benchmark — which measures the ability of an AI to autonomously complete long, complex tasks — then the assumption of US leadership would be immediately weakened.

Already, the gap between US and Chinese models is closing faster than analysts anticipated. The next battleground isn’t just about chatbots that can write poetry. Instead, it is about “agency”: AI that can plan and execute complex tasks on its own. If a Chinese lab such as DeepSeek releases a model that tops the charts in autonomy, and gives that model away free of charge, that lab would be flooding the market with free, high-level intelligence. This would undercut the business models of the US tech giants and mean there was a real threat to the US’s leadership on frontier AI.

In short, it would prove that the US’s attempt to gatekeep the future is floundering, leaving Washington with next to no leverage and Silicon Valley with no monopoly. Swift Center forecasters are therefore closely watching AIs’ performance against the time horizon benchmark set by the AI evaluation organization METR.

Geopolitics

While technology seeks to drive the world into a new era, geopolitics in 2026 is defined by fragmentation and the conflicts of old. The post-Cold War order is fraying, and in its place is emerging a new round of protectionism, regional blocs, and raw power politics.

1. Will the Supreme Court strike down Trump’s emergency tariffs?

Since taking office, President Trump has leveraged emergency executive powers to impose broad trade tariffs, including a blanket 10% tariff on all global imports. The constitutional legality of these powers is now being tested in the US Supreme Court.

A Supreme Court ruling against Trump’s emergency tariffs would be a defining constitutional moment for the US in 2026. A strike-down could hamstring the administration’s economic agenda. It could also create chaos in global markets, for importers, having paid billions in import duties, will seek billions in refunds. Such a ruling would force the administration to seek alternative, and probably more aggressive, legal avenues to enforce their desire for protectionism.

Our forecasters note that this is a high-stakes legal battle with the potential for significant economic fallout. The administration argues that tariffs are a national security tool, citing precedents from the Nixon era. However, the Supreme Court has shown some resistance to executive overreach.

In response, the administration might use the Smoot-Hawley Act or the International Emergency Economic Powers Act (IEEPA) to provide legal justification for tariffs, signaling that the trade war will continue regardless of the ruling. A defeat would have the additional significance of being framed by the administration as an abuse of power by the courts, hampering the economic will of the people. But a win for Trump would show just how powerful he has become.

2. Will Russia and Ukraine agree to an indefinite ceasefire or peace agreement?

A ceasefire in 2026 would redefine the security architecture of Europe. It would almost certainly leave Russia in control of significant Ukrainian territory, validating the use of force to redraw borders. It would also force difficult conversations about the lifting of sanctions, the reconstruction of Ukraine, and the future of Nato. Conversely, a failure to reach an agreement risks an embarrassment for the Trump administration while prolonging the human, economic, and financial costs of the war. It also prolongs the risk of the war escalating into a direct conflict between Nato and Russia.

A peace in 2026 is unlikely to be a true resolution. More likely is an “indefinite ceasefire” akin to the Korean Peninsula’s frozen conflict. This allows both sides to claim partial victory — or simply survival. The key variable to watch is the terms of security guarantees for Ukraine and how seriously they are enforced.

More from the Swift Center

Will India and Pakistan go to war?

By Swift Center Forecasters

3. Will Benjamin Netanyahu still be Prime Minister of Israel?

After two years of war in Gaza, and a shorter war between Iran and Israel, the Middle East is in a fractious state — with consequences for us all. Central to what happens next will be the political survival of Benjamin Netanyahu, the prime minister who took Israel to war. His coalition relies on the Israeli far-Right, making a two-state solution or normalization with Saudi Arabia difficult if not near-impossible. If Netanyahu loses, the probable winner will be Naftali Bennett. Bennett’s path to victory will require a coalition with centrists and possibly Arab parties. The departure of Netanyahu could therefore create an opportunity for a diplomatic reset between Israel and its neighbors.

This year, Netanyahu faces both an election and the resolution of his corruption trial. While the polls currently show him losing to a centrist coalition, the Israeli opposition is fractured, and Netanyahu is a master campaigner. If Netanyahu survives 2026, he will probably move to finalize his overhaul of the judiciary. Specifically, he could seize control of the Judicial Selection Committee and pass the “Override Clause”, which would allow his government, if it has a majority in the Knesset, to ignore Supreme Court rulings. This would effectively end the independence of Israel’s judiciary, removing the last barrier to permanent annexation policies and shielding Netanyahu from his own corruption trial.

Suggested reading

Gaza is the endpoint of liberal democracy

By Aris Roussinos

4. Will Reform UK win the most seats in the 2026 Welsh Senedd and England local council elections?

While the world watches Washington, a quiet political earthquake is building in Wales. The 2026 Senedd elections will use a new voting system that swaps “First Past the Post” for “Closed List Proportional Representation”. Insurgent parties such as Reform UK are expected to benefit from this new system, for as they can convert vote share into seats more efficiently than under First Past the Post.

After 27 years of the Labour Party’s governance in Wales, there is a feeling of voter fatigue. Issues such as NHS waiting times and controversial 20mph speed limits have been highly unpopular with working-class voters, who are flocking to Reform UK. Reform, riding high on its local election victories and strong public polling, could take control of the Senedd in 2026, with Plaid Cymru being the other major horse in the race. If Reform, the Right-wing party led by the Brexiteer and MP Nigel Farage, wins the most seats, it would be a historic shock to the UK’s political system.

A victory for Reform in Wales would be more than just a headache for Keir Starmer, Rachel Reeves and Labour; it would be a resounding signal that the traditional Westminster parties are losing their grip on power. Just as voters in the US, France, and Germany have punished incumbents for perceived failures on immigration and economics, Welsh voters appear ready to reject the norm. If Labour loses the Senedd — a socialist citadel the party has held for nearly three decades — it would signal that there are no safe havens from the rising challenger parties. It would demonstrate that the disconnect between the governing class and the governed has reached a breaking point from which traditional parties may not recover.

5. Will the Iranian regime fall?

There is growing unrest in Iran over the soaring cost of living. Across social media there are numerous videos of cars being set on fire during running battles between protesters and security forces, with several deaths already being reported. Many protesters have called for ending the rule of the country’s Supreme Leader, while others have called for a return to the monarchy.

While street protests are not new to the Islamic Republic, the recent events signal a shift in the political risk. The coordinated strikes by the “Bazaar” (the conservative merchant class) represent a historic break between the regime and its traditional economic backbone. Student unrest is common; merchant class unrest less so. This signals a substantial shift in the risk of the unrest turning into a crisis that the Iranian government can’t control.

Already, the Islamic Republic faces difficulty in de-escalating the protests. This is because the driving force is not just inflation, but fiscal rigidity. The regime faces significant spending pressure following the return of UN sanctions and the costs of the 12-Day War with Israel in June. Reports suggest that projected oil revenues now cover only a fraction of administrative costs: perhaps as low as 5%. This forces a stark trade-off. Does the government concentrate funding on security, or on the maintenance subsidies that previously helped manage social unrest? Our forecasters have noted the government’s decision to close public offices in 26 provinces. Officially, the reason for this was “energy conservation”, but given the context of mild weather and the country possessing the world’s second-largest natural gas reserves, it instead suggests the state is running on empty.

On the Islamic Republic’s side is that it remains the strongest regional ally to China and Russia. Iran sends ~90% of its oil exports to China, and China is Iran’s largest trade partner. Iran is also the only country in the region actively and publicly supplying Russia with military equipment for the war in Ukraine. However, with Chinese demand for oil lessening, Beijing’s willingness to bail out Tehran may be waning.

Should the regime fall, the geopolitical realignment would be profound. It would represent a strategic victory for the United States and Israel, while dismantling a central pillar of Chinese and Russian influence in the Middle East. The fallout would be felt directly in Europe, where the loss of Tehran’s drone and missile supplies would damage Russia’s war effort in Ukraine. Regionally, Hezbollah and the Houthis would lose vital support, dramatically reducing their capability to continue destabilizing the region.

Collectively, as we look forward to what 2026 may bring, we see rapid technological advancements matched by growing geopolitical tensions. Yet the dawn of genuine stability across the Middle East may be breaking on the streets of Tehran.

Such forecasts were hasty. So was much of the investment. The railway bubble burst, and speculators suffered bruising losses. Three decades later, a similar bubble arose in the US. This bubble, like the British one, was soon to burst. Around 18,000 businesses failed within two years.

As a share of American GDP, investment in artificial intelligence has surpassed the telecoms boom of the dotcom bubble. In its degree of concentration, it stands second only to the railroad boom of the late 19th century. AI, then, might be approaching a moment of truth. The technology was meant to speed up the economy; now, it appears to be slowing it. Unless the trend is soon reversed, the biggest tech companies might be in for a sharp revaluation.

In fact, if you remove those companies from the stock index, so-called American exceptionalism disappears. Outside the top 10 companies, the indices have been stagnating since the start of 2025. Similarly, minus that top 10, corporate earnings have been flat for the past three years — coincidentally, the period that followed the release of ChatGPT, the event that sent the AI boom into overdrive. With the sector’s capital expenditure having risen tenfold in just the past three years, the AI bubble is now sucking up more than half of America’s investment. So insatiable is this bubble that there may not even be enough cash in America to satisfy OpenAI. Sam Altman attempted to rustle up additional cash by going abroad, seeking to raise $7 trillion from the UAE and other investors.

That rush to the top doesn’t leave much for everyone else. With all that money pouring into AI, investment in the rest of the economy has begun to decline, slowing growth. That helps to explain the rapid deceleration in the US economy now underway, a marked contrast to the recent good fortune of the “Magnificent Seven” tech companies. With job growth flat, consumption now adds less to the economy than a handful of massive AI investments.

More from this author

Trump will make China great again

By John Rapley

And the gains made by AI are, for now at least, someone else’s loss. Not only is the industry sucking in capital, but it is ravenous for electricity. Data centers consume 10 times the power of traditional servers, and are helping raise American electricity prices — up some 6.5% over the past year. AI centers also consume huge volumes of water for cooling, leading to supply strains and increased prices in many areas. That keeps upward pressure on the inflation rate, raising costs to both consumers and businesses alike and slowing the interest-rate cuts that might facilitate investment in small and medium firms.

In short, the US is currently placing its money on one huge bet: that AI will unleash a productivity revolution so great it will ultimately re-start the economy it’s presently impeding. Americans are doing their bit to make it happen, with ordinary retail investors driving the stock market rally of the last few months and OpenAI, the makers of ChatGPT, reporting that Americans prompt their chatbot some 330 million times a day. 

Yet for all that, we’ve yet to see much evidence of a transformation in output. Over the same three years that the AI revolution has been in full swing, labor productivity has grown at barely 1% a year, maintaining a decades-long trend of declining growth across Western countries. And while some might retort that productivity revolutions unfold over many years — several decades passed between the invention of the steam engine and electricity and significant changes in labor productivity — the maths behind the AI boom make little sense. Unlike the railroads built in the late 19th-century US bubble, which are still around today, AI investment is going into data centers that are set to depreciate rapidly. Nvidia, for instance, warranties its GPUs for only five years. 

In short, this bet needs to pay off soon. And while the record profits at the tech giants leading the American AI boom might suggest that’s already happening — Nvidia’s earnings have quintupled and its share price increased tenfold in the last three years — those results don’t actually tell us much. Nvidia isn’t making money off AI so much as the AI mania, capitalizing on the sale of its chips — the “picks and shovels” of the new goldrush — to other companies that then use them to build AI. In a recent earnings statement, Meta said it did not expect the gains of its AI investment to materialize for another two years. Instead, it is still making money off advertisers on its Facebook and Instagram sites, just as Apple still makes its money from iPhone sales.

The record profits of the Magnificent Seven, therefore, do not support a narrative of economic transformation. Instead, we’re awaiting the breakthrough application that will lift economic productivity to a new plateau. Enthusiasts maintain that, once the technology spreads, AI will add one percent or more to America’s annual rate of economic growth. But there’s no shortage of skeptics who question the calculations behind that optimism.

Among them is Goldman Sachs’ Jim Covello, who says that for AI to justify the current wave of investment, it must solve a trillion-dollar problem. It’s hard to see that happening. Daron Acemoglu has argued that AI’s productivity impact will be negligible, because once it gets through the easy-to-learn tasks which are currently exciting everyone, like search tools, the returns will rapidly diminish. This contention is borne out by recent macroeconomic research from Japan. The neuroscientist Eric Hoel calls this the “supply paradox of AI”: “the easier it is to train an AI to do something, the less economically valuable that thing is. After all, the huge supply of the thing is how the AI got so good in the first place.” Take the example of copy-writers. Because so many of them produce so much material, they become easy to replace. Their knowledge is cheap. This phenomenon won’t carry over to fields where knowledge is expensive.

Instead, a closer look at Meta’s bottom line may reveal what’s really going on, and why the optimistic view of AI’s productivity impacts could rest on shaky foundations. The company credits its new AI for helping steer more customers to its products, thereby boosting sales and revenue. We see a similar trend at Google, whose searches previously led people to websites, but which now keep them on the Google site where they can read quick summaries of the linked content: more eyeballs on Google, more ad revenue to it, but less for all those sites.

In other words, the tech giants have been using AI to capture revenue that previously went elsewhere. On the face of it, that’s the market doing its thing, and it’s that very possibility which excites corporate executives. They believe AI will reduce their need for labor, allowing them to similarly suck up money that would have gone elsewhere — in this case, to their workers.

Suggested reading

Why Iran-Contra still matters

By Patrick McGee

This, arguably, has been the tale of the information age: new technology doesn’t necessarily raise economy-wide output that much. Instead, it replaces labor and directs the resulting cost-saving to owners, helping to worsen income and wealth inequality. Just as the websites that make those AI summaries possible are now being plundered to the benefit of the AI giants, the LLM chatbots they’re developing are trained using content whose creators receive no compensation. Without that motherlode of free inputs, involuntarily subsidized by their creators, AI might make no sense. As one investor put it in 2023, “imposing the cost of actual or potential copyright liability on the creators of AI models will either kill or significantly hamper their development.”

That route to higher profits raises two problems. The first is that the resulting inequality could stoke a political populism that makes the economy more fragile. We might never have got the volatile policy-making of this Trump administration were it not for the sense of malaise caused by globalization’s decimation of industrial towns and industrial jobs. If AI really displaces workers, one can’t rule out a radically redistributive politics of the Right or Left. If anything, current trends favor it.

But in addition to the possibility that the AI revolution could end up eating itself, this type of job destruction can produce an economic paradox. While professionals might lose jobs to AI, not everyone will. Microsoft recently commissioned a study estimating which jobs will be most affected by AI and least; not surprisingly, those most insulated from its effects are manual laborers and personal-care workers (including health care), with knowledge-workers taking the biggest hit.

In other words, the jobs AI will disrupt least lie in those sectors of the economy where labor productivity tends not only to be lowest, but most resistant to transformation. This gives rise to what’s known as the Baumol-Bowen effect. As workers displaced by AI are forced to take new jobs in lower-skilled occupations, their productivity will decline. In consequence, the needle on economy-wide productivity may budge little.

It may yet be that someone will invent an application for AI that transforms productivity across the economy and lifts everyone’s output and income to a new plane. But time may be running short, because right now, the boom is becoming a bust for many.

So: are we facing another railroad episode? It’s certainly true that AI is starting to resemble a speculative bubble, not unlike the dot-com boom. Although the Magnificent Seven are making profits that justify their valuations, their future growth depends on the AI revolution continuing for years at its current rate or greater.

But that growth depends on AI being deployed in revolutionary new ways. If that doesn’t happen, then those growth prospects will change, and the markets could tumble. Given the weighting of the big tech companies in the US stock market — Nvidia alone is worth nearly a tenth of the market’s total value — a fall in share prices could not only hurt all those retail investors who rode the wave, but ricochet through the financial system.

In the long term, it might be good for America if the market goes back to investing in all the other sectors of the economy. In the short term, if this bubble bursts, it will be very painful. Both investors and the Trump administration are going all-in on their AI bet. If it succeeds, they’ll parade like champions. But if it fails, they’ll be running for the exits. And given last week’s lackluster response to ChatGPT’s newest update, investor patience may be wearing thin.

I could see why Edward was interested in the play. The AI community, to which he belongs, is driven by these kinds of future visions: utopian, quasi-divine abundance and control over nature — regardless of whether that outcome is wanted, or whether we will destroy ourselves trying to bring it about. Already, their capital, San Francisco, is marked by this worldview. The technology that’s generated in “SF” is itself the product of a rationalist eschatology that sees the human as a “bootstrapping” mechanism (in the words of Elon Musk) for higher forms of life and intelligence.

Beneath this apparent rationalism lurks a profoundly religious sensibility that seeks to reconfigure the human past the point of recognizability, all in service of generating a sort of machine-angel synthesis. SF, then, isn’t just a place. It’s a theological state of mind.

In the city’s techno-rationalist circles, the answer to the question of whether we should want to live in The Culture is: “Yes, if possible”. ChatGPT founder Sam Altman, for instance, is one of many technologists with a serious interest in life extension (the pursuit of extending human lifespan), biohacking (modifying one’s biology through various interventions), and merging human cognition with artificial cognition (often called brain-computer interfaces).

More From This Author

The Millennial virtue economy is dead

By Matt Gasda

Peter Thiel invests in parabiosis, the experimental practice of transfusing young blood into older bodies in pursuit of eternal youth. Elon Musk is building brain implants. Ray Kurzweil takes hundreds of supplements a day. Jack Dorsey fasts for days at a time. Jeff Bezos and Larry Ellison quietly fund longevity startups. If, as the existentialist philosopher Maurice Merleau-Ponty argued, “we perceive the world with our body”, then the high priests of the SF cult are building not only new bodies, but new perceptions. Out of these new visions will presumably emerge new ethics.

If the members of a technologically optimized overclass see, hear, and move differently — how will they see the rest of us, the non-optimized?

Accelerating integration with technology changes how you experience the world. While in SF, I took Waymos (autonomous vehicles operated by a Google subsidiary) everywhere, rather than traditional taxis or Ubers. There was a particular feeling as the cars passed through areas of squalor. In Frisco (the old working-class name for the city), I felt like I was living in a completely gated experience, a protected techno-bubble. The experience was seamless and efficient, and it changed my behavior. Because there was no driver, I found myself feeling weirdly liberated: like I was passing through material reality like a god or alien observing humans. Somehow, I believed the car could and would protect me.

I liked my Waymo rides to the food co-op, because they spared me from having to mentally and emotionally engage with the bad neighborhood surrounding the health-food store. I could close my eyes as I left the “hacker house” where I was staying — and then open them in an enormous, shiny store offering food designed to optimize my body. You start to like rationalist civilization. AI does the legwork of maintaining homeostasis. There’s no real existential pressure; you start to forget that you live in a fragile, mortal body; you start to feel like you should have total control over everything that happens to you.

The future is here, it’s arriving a little more each day, and it’s concentrated in the Bay Area. If we want a preview of what happens when techno-futurism becomes a ruling-class ideology, if we want to understand what future culture will feel like, we just have to look at rationalist-utopian, accelerationist, godless, and increasingly childless San Francisco.

Bay Area techno-rationalist culture resembles — and, in many ways, is — the rebirth of the Shakers. The 19th-century sect similarly believed that the end times were imminent, and consequently emphasized self-discipline, radical environmental redesign, and withdrawal from the world. Techno-rationalists are Shakers without God. Or rather, theirs is a god who has yet to be built, but the project is coming along nicely.

San Francisco is almost like someone took parts of Davos and placed them inside Naples: a clean, efficient, rational minimalist elite lifeworld nested inside social decay, nested inside breathtaking natural beauty. Life is absolutely pleasant in San Francisco, if you live on the upper edge of the K-shaped economy. Of course, all American cities have this feature to some degree: the affluent live differently than everyone else, everywhere.

But in San Francisco, there is a deeper feeling that this is simply a necessary part of the social Darwinism entailed in transhumanism: that the poor and the “bad” will die off while the techno-affluent live longer, healthier lives, protected by AI.

An astonishing amount of human energy has been spent building this increasingly automated techno-empire, and the end result seems to be something like redemption for a very few, elimination and redundancy for the many. Everywhere I took a Waymo, I saw billboards advertising AI labor — replace your real workers. Soon it will be: replace almost everyone. This is the vision: a priestly, self-medicating, self-optimizing, transhumanist class that lives a life of leisure on an automated grid, protected from the decay around them and from the downsides of the civilization they have built.

San Francisco is also the first city, I think, where a non-negligible portion of the population —  and certainly the most rich and powerful portion — is planning, if not to live literally forever, to live beyond our biological lifespan, confident that technology can build a heaven on Earth for some.

Suggested article

The rise of Yuppiefuturism

By Louis Elton

The Golden State has always been an incubator for cults and prophecies. The rocket geeks who built Cal Tech and turned the state into a defense and aerospace Mecca also indulged strange and occult visions, as the Marxist geographer and California writer Mike Davis reminded us. The latest cult has supplanted ballistics aimed at the Soviets with moonshots to eternal life. For example, Bryan Johnson, tech entrepreneur known for his extreme anti-aging regimen, tweeted recently, “I am building a religion”, adding: “First, here’s what’s going to happen: + Don’t Die becomes history’s fastest-growing ideology. + It saves the human race. + And ushers in an existence more spectacular than we can imagine. It is inevitable. The only question is: will you be an early or late adopter”?

And what if you dissent? What if you don’t want to optimize? What if you still prefer the old religions to the new? What if you have no interest in building a killer app or tracking your biomarkers every day? For now, you won’t be punished. You won’t be called a heretic. But as this cult expands — as I believe it will, because of what it promises — and as caste differences sharpen, tensions will inevitably rise. Refusal will begin to look like a threat. Noncompliance will read as dysfunction.

It isn’t hard, for this reason, to imagine a future where AI-driven cars have humanoid robots in the front seat—not as drivers, but as bodyguards: to protect the transhumanist overclass from the dissenters and neo-Luddites who have been left behind (or were never invited to join the revolution). And it’s also hard to imagine that those bodyguards won’t, at some point, be necessary.

I can still, like in a recurring nightmare, remember the tone of voice of the techie who finished my purchase of my first iPhone. “Welcome to Apple”, he said, as if he were immersing me into a bath of smugness. If we were acting a scene, the subtext would have been completely clear: I was in now and would never get out again. AI firms are now inviting me — inviting us — in just the same way.

But I am not falling for it again. We have to decline. We have to collectively boycott artificial-intelligence products while there’s still time. Because AI is no good for us — no good for our minds, creativity, or competence — and as it gets jammed down our throats, we are the only ones with the power to refuse.

More from this author

How psychedelics blew my mind

By Sam Kahn

Looking back, I understand why I caved to Apple all those years ago. It really would have been social suicide to try to make my way in the 2010s professional class with anything other than a MacBook Pro and an iPhone. Apple really was better than anything else. But I regret that moment, all the same. Every intuition I had knew that opting in to companies like Apple, Google, and Facebook was a fool’s bargain.

The upside was a few years feeling like I was part of the future as I sipped my lattes and floated through the dawning post-industrial era with my sleek silver Apple gadgets. But I’m still paying the price for that: every time I log in to my bank account now, it’s like peeling barnacles off the hull of a ship to get rid of all the new charges that Apple and Google have concocted. Enshittification — the apt coinage by Cory Doctorow that turns out be, like, the word of the century — has revealed itself to be virtually a law of nature. Exactly as Doctorow analyzed it, platforms inevitably “abuse … customers to claw back all value for themselves”.

As AI emerges as the must-have technology of the shiny new future, and everybody starts naming their ChatGPT bots and having philosophical conversations with them, I have decided — formally, as of writing this essay — to opt out. The train is leaving the station without me. I still have not downloaded an AI app of any kind to my phone. And if this was, up until now, owing more to torpor than high principle, then thank God for laziness.

More from this author

Walt Whitman: the original Substacker

By Sam Kahn

AI can only lead to dependence on a technology, which really means dependence on Silicon Valley overlords looking to rip off their customer base. It’s exactly the same dynamic that we’ve all been experiencing and bemoaning for the last 20-odd years. And you can probably plug-and-chug all your own favorite statistics to tease out what that has meant: the average American spending close to five hours a day on his smartphone, while friendship rates have plummeted and teen depression, anxiety, self-harm, and suicidality have spiked.

If you prefer, you can focus on attention span and the finding that the mere presence of a smartphone in a test setting impairs results. As the Silicon Valley apostate and head of the Center for Human Technology, Tristan Harris, put it: “Tech’s race to the bottom of the brain stem to extract information is an existential threat — using our [attention] to upgrade machines is ‘downgrading’ humans”.

Yet it seems that AI is here to stay. Big Tech is all-in on it, and governments aren’t close to getting their act together for meaningful regulation, with President Trump, for instance, reversing Joe Biden’s modest executive order erecting guardrails. But there is still personal choice in the matter, and now is the moment for those who care about human creativity and self-reliance to draw a line in the sand. No, I’m not talking about abandoning tech altogether. But we can, for example, click down below Google’s AI offerings to look at actual links. And we can generally go about living our lives in our sad old way without the benefit of AI “personal assistants” or bot “best friends” or whatever it is that the new technology is supposed to deliver to us.

To be honest, this stance of mine does make me feel like an Amish buggy driver whipping my carriage horses a little harder while the bicycles go whizzing past me; like the weirdos you occasionally see on city streets laboriously tapping out a message into a flip phone like they’re a living museum. But let’s try to get outside the hype for a moment and think through what AI actually is.

AI is basically text-predict combined with data mining. That’s it. It’s a super-Google that goes into the body of texts and rearranges the words into a very pleasing facsimile of a cogent argument. There’s no “intelligence” behind it, in the sense of a computer actually thinking. I’m not the first to be reminded of the story of Clever Hans, the horse that could count and toured widely at the turn of the 20th century.

What Hans was doing really was very cool: he would be given a mathematical question, and he would tap out the answer with his hooves. And Hans was right more often than not. But this was not, as it turned out, because Hans had mastered a numerical system, but because he was sensitive to the crowd and could feel the excitement cresting as he approached the correct number.

AI is more or less the same thing — it uses our wonder to convince us of a simulacrum of intelligence when what we are really witnessing is, in a sense, our own childish excitement at a trick of anthropomorphization: we are pleased by the sight of a calculator imitating something of our mental processes. Our tendency, unfortunately, is to give AI a kind of epistemological deference, to believe that this mash-up of data represents some kind of authority.

Suggested reading

How DeepSeek triggered a tech tantrum

By Malcom Kyeyune

Meanwhile, in the role of Hans’s trainer are the programmers who are influencing AI to give us whatever it is they want us to hear. The disastrous rollout of Google Gemini in 2024 really ripped away the curtain and showed us the strange little man playing with the knobs. Google’s corporate culture at the time happened to be very woke and, lo, the artificial “intelligence” started generating images of black Nazis.

But even if Gemini overtipped its hand, the same dynamics are obviously at play in every other AI model. The people pushing AI now are the same sorts who spent the 2010s promoting web 2.0 as a new vision of freedom and global connectivity, all while destroying traditional media and ripping off as much private data as they possibly could and cheerfully selling it to advertisers. By 2020, the freedom talk was gone and “fighting disinformation” (read: censoring wrong-thought) was in.

This recent history raises an important question: why in the world should we trust these people ever again?

Suggested reading

ChatGPT will kill off the Romantic genius

By Sam Leith

To be sure, we are already fairly late in the game. Even if we don’t use products like Grok and ChatGPT, chances are, we use more rudimentary forms of the same technologies. We use a version of AI to get around and to translate our ideas into other languages. These are remarkable achievements. But they do infantilize us. I’m pretty sure that, if my phone were taken away from me, I couldn’t find my own way from my home to my place of work. And they hinder our self-motivation: I’m also pretty sure that I’ve been much slower to learn foreign languages because I know, at some level, that Google Translate makes it unnecessary.

Do we really want to accelerate the decline in general competence even further? And in exchange for what? Many people remain mystified as to what actual service AI is supposed to provide. The current selling point is that AI gives you a “personal assistant”, but to be honest, I wasn’t aware until Google CEO Sundar Pichai announced “a personal Google, just for you” that I needed a personal assistant. I feel perfectly capable of buying my own airline tickets and booking my own restaurant tables.

What I was aware of needing was meaning and focus in my life: I wanted, for example, to translate the ideas for novels inside me into actual novels. I wanted to be the best, most self-reliant version of myself that I could manage. As for a machine writing a novel for me in a matter of milliseconds — I have no idea how that could possibly generate authentic pride or produce anything other than a cavernous inner emptiness?

Suggested reading

I'm in love with my AI girlfriend

By Sean Thomas

My hope for the past three years had been that AI would just sort of go away. In the Nineties, we expected ourselves to be overrun by clone armies (certainly, my sixth-grade science teacher gave many speeches to that effect). It didn’t come to pass. Could something similar transpire with AI?

That seems increasingly unlikely. For one thing, AI has already changed the face of war, and the AI arms race between the United States and China guarantees that AI will be a major presence in our lives. There was a brief moment in 2023-2024 when it was possible to imagine our AI anxieties going the way of Nineties clone fears. That was when Italy banned ChatGPT and the Biden administration issued its (since-overturned) regulations. But now we are plowing full steam ahead.

All that matters now is individual choices. Unwary, I fell for the techno-optimism of the past two decades and ended up with a diminished attention span and a bunch of mysterious subscription charges to show for it. Well. Fool me once, shame on me. Fool me twice, shame on Sam Altman. I know, much better now, the folly of turning over my own mental powers to a bunch of techies promising a brilliant future.

I’m not making that mistake again. Nor should you.

Such grandiose schemes are nothing new in the tech world: after all, Elon Musk has dreamed of colonising Mars, while Peter Thiel is said to aspire to immortality. But what is significant about Altman’s trillion-dollar moonshot is that it would put him in competition with another powerful actor: the United States government. Under both Trump and Biden, the White House has pursued a policy of re-industrialisation.

For ultimately, Altman’s plans may be compared to the bipartisan CHIPs Act in their ambition. And if successful, they would address many of the underlying strategic issues that the US government is seeking to resolve, such as the danger of falling behind China in the long-term race for AI and manufacturing supremacy, or the risk posed by a microchip shortage in the event of a Chinese invasion of Taiwan. Indeed, Altman is requesting Washington’s backing for his venture, most probably in anticipation of the major national security and legal antitrust objections that his gambit is likely to raise. (He is also seeking funds from foreign governments such as the UAE.)

Yet despite the apparent convergence in goals between Washington and OpenAI, there is, in fact, a stark choice to be made between them. As Thiel said, there is a great difference between progress in the world of bits and progress in the world of atoms. The problem is that while American society has had no problem churning out “innovations” of an increasingly dubious social utility in the former, it has struggled to produce any comparable achievements in the latter. While an endless stream of apps and content has inundated consumers since the advent of the Web 2.0, the material foundations of America — its electrical grid, basic infrastructure, industrial hardware, national defence base, and more generally, its collective capacity to produce material goods — has declined into obsolesce. Boeing’s string of engineering failures is just the latest example. Or, as Thiel put it a decade ago: “We wanted flying cars, instead we got 140 characters.”

Suggested reading

ChatGPT will kill off the Romantic genius

By Sam Leith

And while OpenAI’s generative AI products, from ChatGPT to Sora, might appear more impressive than a 140-character tweet, they carry the same fundamental premise: large-scale innovation in the West is now something that mostly takes place in a non-corporeal universe mediated by screens. Contrary to futurist discourse promising science fiction-like visions of a radically more expansive and technically advanced built environment, Silicon Valley has instead delivered infinite arrays of two-dimensional spectacle combined with total civilisational stagnation in the crumbling three-dimensional world. Indeed, its technologies have had the effect of warping and undermining society’s shared notions of reality. And herein lies the problem with the prospect of a new industrial revolution led by Altman and his fellow travellers: it threatens to supercharge the explosion of content in the world of bits, while the accompanying benefits in the world of atoms remain vague to non-existent.

At this point, Americans must ask themselves, to what ends should America’s vast capital and energy resources be directed? Should they be devoted to perpetuating the existing bit-centred economy, which has made Americans stupider and more addicted to short-term stimulus? Or should they be marshalled toward the construction of a new paradigm, one where an atom-centred American civilisation is once again engaged in the making and building of things in the physical realm?

The present status quo goes by many names: the “information age”, “the knowledge economy”, “the Californian Ideology”, “network society”, etc. These slogans, most of which were popularised in the Nineties, reflected the late 20th-century transition from an industrial economy to a post-industrial one grounded in non-material commodities such as digital software, financial products, media franchises, and other forms of intellectual property. In this view, the old worlds of industry and mass manufacturing could safely be relegated to developing economies, while developed ones fully immersed themselves in the more advanced and enlightened forms of production, that of information, ever more abstracted from actual people and places.

The elite of this economy is, of course, Silicon Valley, which not only gave us the commercial internet, social media, smartphones and generative AI applications — but also the cultural values that came with them. The promise underlying them had always been to bring the world closer together and to allow humanity to be freer and more creative than ever before. The ethos propounded by Altman, Musk and others at OpenAI’s founding in 2015 carried many of the same liberatory and messianic undertones.

Yet rather than creating dialogue, digital echo chambers have left society more fragmented than ever before. Humans, especially young people, have become more depressed and socially atomised due to their dependence on screens, while nearly every domain of public life, from politics to culture and artistic self-expression, is now governed by the law of the algorithm. What’s more, the latest advances in AI technology have rendered many “knowledge economy” professions, including lawyers, creative professionals, financial analysts, and software developers vulnerable to displacement by automated counterparts. (The irony is that blue-collar workers may end up suffering the least from AI replacement.)

In other words, the commanding heights of the knowledge economy are, for the first time, directly endangered by the fruits of their own celebrated innovations, namely Artificial Intelligence. Understood in this context, the trajectory of re-industrialisation under Altman would represent the culmination of these post-material and post-human currents. The monumental computing power he dreams of likely wouldn’t be used to build anything real, like in the first Industrial Revolution, but would rather be deployed toward the further subsuming of human society into the endless virtual realities.

Suggested reading

Mary Gaitskill: How a chatbot charmed me

By Mary Gaitskill

The scale of this human-AI “merge” would not just extend to jobs but to some of the most intimately human roles: AI friends, AI lovers, AI philosophers, and eventually AI gods and godheads. One of the most popular memes in “accelerationist” circles is of a future civilisation of self-replicating AIs harvesting energy through a web of Dyson Spheres in space. It is telling because there are no humans left — their minds presumably having been merged with the singularity — and consciousness itself has been entirely disembodied. This is not a sinister conspiracy, but a vision openly aspired to by many in Silicon Valley — and one that Altman’s microchip industrial complex would surely help to realise.

But there is another, competing vision of American re-industrialisation, which can be found on both sides of the political aisle. Donald Trump has spoken of raising magnificent new “Freedom Cities” out of the American wilderness, while National Security Advisor Jake Sullivan called for the building of “a clean-energy manufacturing ecosystem rooted in supply chains here in North America”. Whether distilled through Right-wing populist MAGA-ism or the progressive industrial policies of Bidenomics, the common thrust of this emerging paradigm has been to reconstruct the 20th-century industrial economy of atoms for the 21st century, and to recover the strategic technological and manufacturing edge that was squandered to China after the Cold War. These precepts would provide a much firmer and saner basis for launching an industrial renaissance than Altman’s bit-centric blueprint.

This is not to say that there is no place for AI in this version of industrial renewal. Rather, a properly materialist approach would take a discerning and qualified approach to AI that channels its awesome powers toward productive uses in the real economy rather than to the diversionary “attention economy”. It could be used to enhance and synthesise everything from research-and-development, next-generation manufacturing and construction to advanced energy and resource extraction methods. Under this new paradigm, America’s decaying cities would be transformed into a dramatically more abundant material and built environment, closer to what science fiction has imagined.

The reality is far different. For instance, it is no surprise that tech capitals such as San Francisco and Palo Alto are some of the most unyielding bastions of Nimby-ism in the country, embodying the engrained post-materialist instincts of the tech world to a tee. Perhaps AI could be employed to streamline the cumbersome bureaucratic approval processes that stand in the way of new construction in these places.

But how can we hope to get there? More than a century ago, the German polymath Oswald Spengler dedicated his magnum opus, The Decline of the West, to charting the fall of “Faustian Man”, who, he argued, had dominated the Western imagination for centuries, and whose distinguishing mark was the perpetual conquest of space. It is fairer to say, however, that the Faustian spirit didn’t so much vanish as migrate from the West to the East. For in China today, the ruling party has presided over a titanic effort at reshaping physical space: entire metropolises have been ordered into existence while the most advanced industrial and computer technologies are being pioneered at a rate that puts the West to shame. Meanwhile, the party-state commands its tech sector to cut down on video games and social media fads — the precise opposite, in other words, of what goes on in America.

The choice is thus between post-Faustian man, embodied by Sam Altman and his apostles, who will complete the mind’s retreat into the realm of bits; and what may be called “Neo-Faustian man”, illustrated by Xi Jinping and China’s leadership, who is most intent on preserving, in the classical sense, human mastery over the world of atoms. As America enters a new industrial epoch, its citizens can only hope for their own “Neo-Faustian” genius, albeit one who is attuned to the values and best traditions of the West. Or else, they risk seeing material reality itself slip from their fingers — forever.

The strange saga began a week ago, when the board of the nonprofit decided to fire its AI guru and cultish leader, Sam Altman. But when the 700+ staffers of OpenAI wrote an open letter saying that they too would go with the ousted CEO, he was swiftly reinstated.

The hairpin plot twists of this power struggle have been breathtakingly hard to follow. Reports are now surfacing that Altman had announced that he was on the brink of achieving a significant AI breakthrough only the day before he was fired. A letter was sent to the Board advising them that this tech discovery — an algorithm known as Q* (pronounced Q-Star) — could “threaten humanity”. The algorithm was deemed to be a breakthrough in the startup’s search for superintelligence, also known as artificial general intelligence (AGI), a system smarter than humans.

Altman’s dream was to then marry AGI with an integrated supply chain of AI chips, AI phones, AI robotics, and the world’s largest collections of data and LLMs (large language models). Its working name is Tigris.

To achieve this, Altman would need vast computing resources and funding. Perhaps that is why reports suggest he has been talking to Jony Ive, the designer behind the iPhone, Softbank, and Cerebras — which now makes the fastest AI chips in the world. Cerebras chips are big. The size of a dinner plate and more powerful than any traditional chips.They also have Swarm X software, which allows them to knit together into clusters that create a computational fabric that can handle the massive volume of data needed to build better AI.

Cerebras represents a great threat to the manufacturer of the world’s fastest supercomputers and AI chips, Nvidia. The most powerful of these supercomputers, Summit and Sierra, are central to the defence of the American nation and are kept at the highly protected nuclear facility in Oak Ridge, Tennessee. But almost every big organisation depends on Nvidia chips or computers. A year ago, Nvidia was worth $300 billion; now it is worth $1.35 trillion — the most dramatic increase in the value of a Nasdaq firm since 1971. Yet Cerebras has designed a chip 20-times faster than Nvidia’s. This is why some say the Cerebras IPO will kill Nvidia. Now we begin to see a national-security component to this story.

Suggested reading

Mary Gaitskill: How a chatbot charmed me

By Mary Gaitskill

While the West has been focused on generative AI that has no cognitive ability, China has taken a different path. It has built the world’s only Quantum optical computer, which can solve in 47 seconds a problem that would take a traditional supercomputer 240 years.

Similarly, Altman wants to build a new generation of computers for the AI era. In July, he partnered with Cerebras and the Emirati incubator G42 to unveil the Condor Galaxy, the “World’s Largest Supercomputer for AI Training”. G42 is behind the world’s largest Arabic LLM (Large Language model), which, like ChatGPT, generates new linguistic content, and is also working with Amazon to gather and process DNA information to develop massive new global genomics, proteomics and biobanking services.

To an AI data scientist, the data collected for these innovations — the languages, nationalities and DNA involved — are honeypots brimming with opportunity. The West doesn’t have the mechanisms or the mores to gather such a quantity of meaningful data or the money to finance what Altman wants. But international investors do, including the Emiratis, the Chinese — who are already very interested in Altman — and all the others backing G42.

Moreover, if the world wants AI designed for true diversity, if it wants medical and financial products created to suit the broadest range of humans, then this will only happen where that diverse data can be found. It’s not going to be created in the US, where young, white male tech bros all design AI, and the FDA-approved testing of medicines pretty much excludes anyone but white males.

When AI people say they worry about Altman’s lack of “guardrails”, they mean he is willing to take the risk that he builds something that, like a Djinni, cannot be put back in a bottle. He is prepared to build something that might not be controllable. He is like Oppenheimer who was willing to smash atoms to win the Second World War, even though he might have ignited the atmosphere and incinerated Earth to do it.

Suggested reading

AI is a false prophet

By Jacob Howland

By this measure, Altman is a mad scientist who will put us all at risk to achieve a historic breakthrough. Hence the letter to the board of OpenAI by some staff, along with others keen for the world to take note that the huge staff attrition rate at OpenAI was not due to “bad culture fits” — rather, this was due to “a disturbing pattern of deceit and manipulation”, “self deception” and the “insatiable pursuit of achieving artificial general intelligence (AGI)”. For Altman, this “pursuit” entailed incorporating superintelligence inside a robot’s body. Which, presumably, is why OpenAI started investing in humanoid robotics made by IX in Norway back in March.

You can understand why the OpenAI Board might have become uneasy when they realised that Altman was racing around the Middle East and Asia trying to raise billions for this vision. But, as Bloomberg wrote, “these are not “side ventures”, they are “core ventures”. This is about redefining the cutting-edge of chip design, data collection and storage, computational power, and the interface between AI and physical robotics.

This new supply chain would not only challenge America’s IT infrastructure; it could be facilitating the diminishment of US power. It implies innovation is shifting outside of the US and acquiring data that is beyond the reach of regulators. No doubt the US authorities looked at all this and saw Altman collaborating with G42 as equivalent to fraternising with the enemy because it is seen to be backed by the folks who own ByteDance, the parent company of TikTok (G42 has a significant stake in them). They saw that G42 owns Pax AI, which some say is Pegasus (the notorious spyware) reconfigured. Was Altman under surveillance as he pursued this grand vision? How could he not be?

The truth is Altman does not believe in borders. He has one goal: to build the best AI possible. He has a vision that probably worried his board and unnerved Washington. Given that the US is trying to slow down technological innovation in other parts of the world by restricting the sale of the best chips and computers, it is hugely challenging when he says: “We’ll build our own stuff — in fact we’ll build our own supply chain and ecosystem.” So much for ITAR, the US system for banning the export of critical tech.

There was a time when an American would have been arrested for selling such protected high-tech innovations abroad. Today, can you stop a smart American from innovating outside the US? Can you tell entrepreneurs not to take foreign money and not to partner with foreign firms? Can you demand that they stop challenging existing firms like Nvidia? No. Not when others are offering so much money.

What took place over the past week at OpenAI confirms that none of the young leaders of this new AI space see borders. As Andrew Feldman, co-founder and CEO of Cerebras Systems said: AI “isn’t a Silicon Valley thing, it isn’t even a US thing, it’s now all over the world — it’s a global phenomenon”. Yet, the US authorities and its allies will have known the significant challenge to Western notions of security and control that Altman’s vision presents.

Suggested reading

Why is my chatbot hitting on me?

By Kathleen Stock

Two days after Xi and Biden met on November 15 and agreed to play nice, Altman was fired. The Daily Dot suggests that Altman was terminated because Xi hinted to Biden that Altman’s OpenAI was surreptitiously involved in a data-gathering deal with a firm called D2 (Double Dragon), which some thought to be a Chinese Cyber Army Group. David Covucci reported, “This D2 group has the largest and biggest crawling/indexing/ scanning capacity in the world 10 times more than Alphabet Inc (Google), hence the deal so Open AI could get their hands on vast quantities of data for training after exhausting their other options.”

There is something deeper happening here. China has already forged an AGI path of its own. They are avoiding generative AI in favour of cognitive AI. They want it to think independently of prompts. China is already giving AI control over satellites and weaponised drones. No doubt, Altman would love to work with that capability and the Chinese would love to work with him.

Could that be why Larry Summers, the Former Secretary of the Treasury with connections to US politics and business, also ended up on the new OpenAI Board? Perhaps the US Government realised they could not stop Altman but might claim a seat at his table?

What does all this mean for us? Experts like Altman say they are designing AI to do good. But AI designers disagree about what constitutes “good”. It is clear this arms race could have civilisational consequences.

If it sounds too good to be true, that’s because it is. Altman has promised that his Worldcoin project, which he eventually hopes to roll out around the globe, will bring millions of “unbanked people” — that is, people without bank accounts — into the economic and biometric fold. But his Orwellian scheme will likely do more harm than good to the most financially vulnerable people on the planet. Despite the word “coin” in the title, it’s unclear whether Worldcoin is trying to create digital money — or to colonise global identity. Or perhaps it is angling to do both.

This lack of transparency may explain why the Kenyan government pulled the plug on Worldcoin’s operations less than a month after the coin launched. On 2 August, it called a halt to the great data onboarding, questioning the legality of Worldcoin’s biometric and data collection practices. But this was only after 635,000 Kenyans had signed up: surrendering their data to the unknowable intentions of a Silicon Valley tech bro.

While it may have provoked the most media controversy, Worldcoin isn’t the first project of its kind: nearly every major digital platform will now trade your digital identity for “tokens”.  And by tokens, I mean money-like things ranging from airtime credit and loyalty points to game tokens, customer data and gift balances. Tokens have always ghosted the economy of publicly mandated currency, but now they are coming to the fore, threatening to replace money itself. It raises the question: was money only ever a blip?

Tokens are different from “real” money in a number of ways. If money is at least in theory fungible — one dollar is supposedly the same as any other — tokens have limited fungibility: they come with strings attached, rules about who can spend them, where, and when. Many tokens, like Worldcoin, make identity a prerequisite for a transaction. In 2016, when The Orb was just a twinkle in Altman’s verifiably unique iris, the World Food Programme trialled a system that issued relief payments through the use of biometrics. The system allowed refugees to buy items from participating stores using their biometric data as a credential: all they needed to do was scan their iris to pay for an item. The idea was to prevent fraud — and to ensure that those availing themselves of the programme were “worthy” of relief in the first place.

Suggested reading

How we appeased China's Zero-Covid regime

By N.S. Lyons

Amazon uses a similar strategy, paying its Mechanical Turk workers outside of the US and India in gift balances. This balance is non-transferable: it is tied to the workers’ identity and can only be redeemed through the Amazon store. The token thus keeps the flow of value in a closed loop.

The token system restricts people’s freedom of choice — the ultimate weapon for the nanny state and Big Tech corporations alike. In the United States, for instance, food stamps (now called SNAP benefits) come pre-loaded onto an Electronic Benefits Transfer (EBT) card. These tokens can be used for fresh food and produce, but not for alcohol, tobacco, or hygiene products such as tampons, vitamins or medicines. They can be used for cold deli food, but not for hot meals. Why cold pre-cooked meat but not half a rotisserie chicken? It seems there is something too “easy” — too idle, maybe — about buying a hot meal for your family using benefits.

Another way in which tokens are different to money is that they are able to communicate not just the terms of an exchange but additional values and criteria alongside a transaction. Sometimes these other things are a bit of fun — game tokens and NFTs are used to flex and brag, emotes are used to joke and troll, and even Dogecoin was used to tip other users for being funny on the internet before Elon Musk bought some for his children and catapulted the token into speculative stardom. Funny, that. But sometimes it’s not so fun: while money codifies exchange value, tokens are also used to survey, to remember, to calculate and to programme values into our exchange media.

As more and more of our money shifts from the coins, notes and bankcards we carry around in our pockets to tokens that ride the rails of information and communications technologies, our exchange media is becoming programmable. This means that — unlike cash, which is an anonymous bearer instrument — every digital transaction carries a range of transactional data about where it is spent, and by whom. Tokens can also be programmed to specify not only an exchange value, but a whole range of other terms and conditions that must be met before an exchange will take place: relief payments that can only be spent on so-called healthy food, cash transfers that are contingent on vaccine uptake or school enrolment. Whose values are these? We’re not at liberty to know.

Suggested reading

The West and China share the same fate

By N.S. Lyons

Programming is what digital platforms do so well. In China, the so-called “super-apps” are WeChat Pay and Alipay, where users can do everything from online shopping and ordering taxis to dating and applying for a loan. Alipay is owned by the Chinese e-commerce group Alibaba (second only to Amazon), which, after launching its integrated payments app in 2004, realised that its data about online purchasing could be used not only for logistics and advertising on the website, but also to underwrite China’s social credit system. In fact, that system actually began with Alipay’s Zhima or Sesame Credit system, in which what video games a user bought on the Alibaba platform influenced loan inquiries, access to dating sites, and even Schengen visa applications.

Of course, China is not alone in amassing personal data. In 2019, Facebook announced that it was launching a digital token, Libra, and wallet, Calibra, which many thought would turn into a Western super-app similar to Alipay. While Facebook publicly claimed to be providing financial inclusion for millions of unbanked Facebook users, buried in the Libra whitepaper was a statement that, alongside a coin, Facebook would also provide a portable global digital identity. “I call it KYZ or ‘Known by Zuck’”, David Birch, a financial identity expert, told me at the time. “And really, why wouldn’t your Facebook identity be more reliable? I can buy a fake passport on the dark web, but building a fake profile with a real social graph and real history takes a lot of grunt work.”

Tokens such as these are not a new or internet-native phenomenon. We have always had tokens or “special monies” that circulated alongside public money — alms, charitable tokens, food stamps and relief vouchers for the poor, allowances and pin money for women, scrip for workers. These tokens were privately issued; they came with strings attached. Not just value but values written in their metal. What is different today, however, is that tokens are bound to a specific digital platform. Where once a merchant could engage in a bit of creative accounting to turn a food stamp for butter into cigarettes, or accept a token intended for Alice as payment from Bob, today these sleights of hand are often impossible. Our tokens are hard-coded by the platform — and increasingly, by the state.

Suggested reading

Workers have lost faith in money

By John Rapley

Terrified of Big Tech platforms monopolising the token economy, governments have been playing catch up. Following Facebook’s Libra proposal, many governments pushed back with regulation and fast-tracked the development of what are now known as Central Bank Digital Currencies (CBDCs) — in essence a digital version of publicly mandated currencies. Tokens, issued by the state. In China, where the digital Yuan is already in pilot form, these tokens are programmed at issuance. In Europe, members of parliament are experimenting with tokens that will monitor tax evasion and be programmed with a degree of identity. In the UK, the digital pound also contains proposals for sterling “with memory” — that is, for tokens that keep account of citizen spending to monitor suspicious transactional activity. Whether tokens are controlled by nation states or Big Tech companies still remains to be seen, but both outcomes look equally grim.

Digital tokens are often framed as a panacea for the financially vulnerable, providing ready access to safe, affordable remittances and credit. And yet, many of these programmable tokens have already inversely harmed the most vulnerable in our society, producing one kind of money for the affluent, the middle classes, and the banked West — and another for the poor, the underbanked, and the undocumented. A token with values, a token with strings attached.

Sam Altman predicts that, in the future, AI will displace so many waged jobs that citizens will require a Universal Basic Income to survive. This payment will be issued — how else? — through the Worldcoin app. Altman imagines a post-AGI future without work, without money — and maybe even without humans. The economy, at least as humans understand it, will be a thing of the past, a blip in the broader history of the world.

But the token dystopia is already here. In 2014, former Maine Republican governor Paul LePage lambasted families receiving a small relief cash benefit called Temporary Assistance for Needy Families (TANF), accusing them of defrauding taxpayers by misspending their benefits. The allegation drew on 36,000 recorded cash withdrawals from a number of EBT cards to show 3,650 transactions in which recipients had made a withdrawal in off-licences or smoke shops. Despite the fact that the data did not show how or where the money was spent, it spurred a bill requiring TANF families to submit 12 months of receipts for auditing. The poor are the test subjects for future surveillance: “You should pay attention to what happens to us,” a mother on welfare payments warned. “You’re next.”