Only modernity could have produced such a form of total dependency, or coercion. (Bruno Vincent/Getty)
In Tarkovsky’s 1972 film Solaris, the cosmonauts orbiting a cold and sentient ocean planet have filled their futuristic space station with a clutter of homely, earthly things. A copy, or perhaps the original, of Breugel’s The Hunters in the Snow hangs in their wood-paneled, candlelit library, with its dogs and birds and trees, and peasants skating on the ice and woodsmoke curling from the brick chimneys. In the Armenian cosmonaut’s cabin, Caucasian rugs and daggers and photographs of medieval monasteries outcompete his scientific instruments for space. Tarkovsky’s vision of the future was of a desperate grasping onto a rooted human past. Like the Soviet Union’s rearguard defense of 19th-century bourgeois European high culture, the effort proves futile: though knowing it is not real, the film’s protagonist Kelvin surrenders himself to the simulacrum of his family dacha in the forest, and of his dead wife and father, produced by the alien intelligence.
The founders of frontier AI labs present us with a similar, perhaps equally futile choice. In his 2024 essay Machines of Loving Grace, the Anthropic CEO, Dario Amodei, considers, almost as an aside, the economic and social implications of the world he is rushing into being. If AI will shortly surpass humans at almost any given task, then at least, he observes, “One area humans are likely to maintain a relative (or even absolute) advantage for a significant time is the physical world.” While good news for builders, or plumbers, or mountain shepherds, even here there is a warning “I do think in the long run AI will become so broadly effective and so cheap that this will no longer apply. At that point our current economic setup will no longer make sense, and there will be a need for a broader societal conversation about how the economy should be organized,” adding that “some new and stranger thing will be needed”.
Perhaps, he considers, some form of universal income could be dispersed (by whom is not made clear), skimming off the labor of the AIs for the benefit of economically useless humans, although “I suspect that will only be a small part of a solution”. Perhaps, he argues, “It could be a capitalist economy of AI systems, which then give out resources (huge amounts of them, since the overall economic pie will be gigantic) to humans based on some secondary economy of what the AI systems think makes sense to reward in humans (based on some judgment ultimately derived from human values).” Having sketched in hazy fashion a vision of the future he is rushing us towards, Amodei notes that “we will likely have to fight to get a good outcome here: exploitative or dystopian directions are clearly also possible and have to be prevented.”
Indeed, Amodei is no doubt aware that the question of how societies, and economies, should rightly be ordered has been a matter of some dispute in history, not always to the benefit of human flourishing. But how much time remains for us to do so is worryingly unclear: the suspicion grows that the desperate rush towards Artificial Superintelligence, or ASI, is timed more to coincide with the lifespan of the ailing Trump administration, and its laissez-faire approach to Frontier AI advances, than to any pressing consumer need or desire.
The questions raised are so vast in scope that they are daunting to even enumerate, let alone grapple with; perhaps that is why the AI CEOs wave them away so airily. What are the “human values” that a super-intelligence will apparently choose to defer to? Confucianism? Widow burning? The morality of Radio 4 afternoon plays, or of San Francisco polycules? Is the belief that a post-human super-intelligence will converge on the single objectively correct, optimum form of governance and economy for all human societies? If so, the idea that we are in an urgent race to grant AI unlimited power in the belief that it will, in turn, churn out a blueprint for Utopia seems to be, literally, magical thinking. Or will the intelligence produced by different AI labs settle upon different solutions, and if so, what would this tell us about the nature of the solutions themselves? Is choosing between, or rooting for, the different labs now a form of politics in itself?
Amodei writes, without developing the idea, that it is vital for Western liberal democracies to acquire ASI first, to “be able to parlay their AI superiority into a durable advantage”, which “could optimistically lead to an ‘eternal 1991’ — a world where democracies have the upper hand and Fukuyama’s dreams are realized”. Leaving aside the misreading of Fukuyama, is the implication here that a Chinese superintelligence would adhere to different values, and if so, who is to say that those values would be wrong? Perhaps the latent computing power to be brought to bear by the Chinese Communist Party will think up an even more efficient world order than that of America’s scattered and competing commercial frontier labs, and we mere humans should simply comply.
Or is the argument that human oversight, bearing the imprint of specific cultural beliefs, is necessary to guide ASI to the correct outcomes, in which case what claims are we actually making about superintelligence? If the race for ASI is a component part of the struggle between ideologies, or empires, then it is difficult to see why China, or Russia, or anyone, will resignedly submit to the eternal hegemony of an American “God in a box”, which is after all composed of a certain amount of real-world infrastructure, in a few overwhelmingly American locations, overseen for now by distinct persons who are all-too-human. Perhaps the AI kill switch already exists, and not in California or Washington. It is not comfortable to find oneself hoping so.
Talk of “human values” is most jarring when coming from Californian tech CEOs and their minions, whose own expressed worldview is barely perceptible as human, even as they seem in a cult-like rush to supersede humanity itself. As one Anthropic researcher, Tao Lin, blankly remarks, “It’s very likely that pretty much all humans on earth will become permanently unemployable, and will need to strictly rely on government welfare to sustain themselves,” as “Human employment will go away”. If humanity’s future is to depend on handouts from some higher power, then who or what determines who is paid, and how much, and what will be required in turn? Who is Tao Lin, or Dario Amodei, to foist this on us? It is mistaken to term this state of affairs, as many have, as “technofeudalism” because feudalism, in its Western European form, described a series of binding rights and obligations up and down the social scale: what is being proposed is a form of total dependency, or coercion, that only modernity could have ever produced, with nothing but the most thinly sketched out rationale for why any of it is desirable.
Perhaps it is a marker of my inveterate conservatism that I think this “broader societal conversation” would be better had before the advent of total unemployment. What’s being offered is, after all, a total leap into a revolutionary unknown, whose own advocates openly, if off-handedly, admit the looming prospect of great social and economic harm to the vast majority of humanity — all in the service of a form of technological determinism that has evolved into a religion. This revolutionary change is the unwanted vibe shift forced on us through the lax oversight of a distracted, and clearly incompetent, but notionally conservative American presidential administration. In so far as there is any political logic to the Trump administration’s stance, it must be that the promised GDP boom will cement his legacy, and reassert America’s status in the world. Given the already widespread unpopularity of AI advances among American voters, and their anger at modest fuel price inflation, it is difficult to judge how a road to Utopia that leads through total unemployment, and dependence on perpetual Covid stimulus payments, will prove a popular electoral platform. It is now the Democrats, through figures such as the Georgia senator Jon Ossoff, who have taken up the populist mantle Trump has frittered away, with reining in AI at the very center of their pitch.
It seems strange, perhaps, that it is the political Left which has come out most stridently against unfettered AI research: the vision offered is, after all, one of a revolutionary shakeup of society in the name of progress, while the desire to rein AI advances in is by its nature a conservative one. Like the implicit conservatism of Tarkovsky’s Soviet parable, the tension was perhaps there from the beginning. Marx’s vision of life in a communist society, where one could “hunt in the morning, fish in the afternoon, rear cattle in the evening, criticize after dinner”, sounds like the idealized life of a Victorian country parson: concerted labor, as opposed to quasi-aristocratic hobbyism, in itself evidently held little dignity or value for communism’s intellectual father. In a scattered series of notes, later called The Fragment on Machines, Marx seemed to predict that technological advance would in itself render labor unnecessary, undoing the basis of capitalist society: “As the system of bourgeois economy has developed for us only by degrees, so too its negation, which is its ultimate result.” Writing during the Industrial Revolution, Marx perceived already that “it is the machine which possesses skill and strength in place of the worker, is itself the virtuoso, with a soul of its own in the mechanical laws acting through it”, and that “the science which compels the inanimate limbs of the machinery, by their construction, to act purposefully, as an automaton, does not exist in the worker’s consciousness, but rather acts upon him through the machine as an alien power, as the power of the machine itself”. The tendency of capital to invest in efficiency, Marx believed, would ultimately become capitalism’s undoing, as “capital here — quite unintentionally — reduces human labor, expenditure of energy, to a minimum. This will redound to the benefit of emancipated labor, and is the condition of its emancipation.”
This, elaborated with robot slaves and asteroid mining, is the rosy vision of the “techno-optimist Left” rejected by the Marxist writer Gavin Mueller in his 2021 ode to Luddism, Breaking Things at Work. Instead, Mueller observes, “the radical left can and should put forth a decelerationist politics: a politics of slowing down change, undermining technological progress, and limiting capital’s rapacity, while developing organization and cultivating militancy”. Mueller’s argument against automation is not an aesthetic or ethical choice, but a political one, rooted in radical Left goals. “As pleasing as slow aesthetics might be,” he states, “I am not content to ground my argument in claims that a particular pace of life is ‘a more natural, human’ one’”. “The argument for deceleration is not based on satisfying nature, human or otherwise, but in recognizing the challenges facing strategies for organizing the working class.” Rather than a skulking retreat from technological advance, it is an explicit, open revolt against it: “Decelerationism is not a withdrawal to a slower pace of life, but the manifestation of an antagonism toward the progress of elites at the expense of the rest of us. It is Walter Benjamin’s emergency brake. It is a wrench in the gears.” But if his ends are radical, Mueller’s means, and perhaps motivations, are by their nature conservative: the desire to be cautious, to slow down, to protect what is good about what already exists before casting everything into the furnace of unknowable change. In the future politics of AI, instinctive conservatives may find themselves joining strange political bedfellows.
But the same is true of the political Left. It is not throne-and-altar Catholic postliberals most vocal in their support for Pope Leo XIV’s recent pronouncements on AI, most clearly outlined in last May’s encyclical Magnifica Humanitas, but Leftists not otherwise positively engaged with the doctrine of the Roman Catholic Church. For Catholics, this ambiguity is simply natural in a moral philosophy that predates and surpasses the modern divisions of Right and Left: the Pope, as is his role, acts as the defender of the human soul, and its bodily flourishing, at what appears to be a time of unprecedented threat. Like Mueller, but from a more exalted stage, the Pope urges the same politics of deceleration, “a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.” Making the same warnings many non-Catholics make against the ethical, political, and economic risks of unrestrained AI research, Pope Leo also emphasizes, as Marx seems to reject, the inherent value of human labor itself, for as he observes, “work is not simply an instrument; it expresses and enhances the dignity of our lives. It is a requirement of the human condition, a normal path toward maturity, development and personal fulfillment.”
Against the vision of total unemployment, and total dependency on financial handouts from our AI overlords, the Pope insists that “financial assistance to the poor may at times be necessary in emergencies, but it cannot become the sole response, since the goal is to enable each person to live with dignity through his or her own work”. Of what value is the leisurely lifestyle envisioned by Marx when the intellectual pursuits also enjoyed by Victorian curates will be far outclassed by AI? Like the threatened extinction of mathematics, there is a fundamental opposition between those who see the world as a series of abstract problems to be solved by the most efficient means available, and those who see inherent value in man’s pursuit of intellectual and physical excellence. These are moral questions those leading us towards the AI future seem unequipped to answer: at least the Church has many centuries of experience in dealing with the threat posed by soul-less imitators of the human.
Just as Marx observed, with gothic relish, that capitalism “is like the sorcerer who is no longer able to control the powers of the nether world whom he has called up by his spells”, our futures are currently in the hands of those who believe they are ushering in a Utopia outside human control. For intellectual advocates, cautious or otherwise, of a Trumpian restoration of common-sense conservative values, this reckless leap into a post-human future shows us up as fools. What was the gain of all the lofty tomes and tweedy philosophizing on post-liberalism and the moral value of the human and small-scale when this is the result? Conservative voices have so far been silent on this most conservative of causes: perhaps, like the cosmonaut Kelvin in Solaris, the empty simulacrum of the rightly-ordered, human life has already proved the more seductive prison.



