Massively dangerous technologies that could destroy us all: How about no?

Here’s something you can say for humanity: despite having the ability to nuke each ourselves back to the Stone Age, we haven’t yet done so. That’s not to say that we won’t at some point in the future, but we’ve resisted the temptation for decades now, so well done us!

Of course, the logic of Mutually Assured Destruction does rather put one off the idea of a cheeky pre-emptive strike, which is the main reason why nuclear peace prevails. This is what game theorists call a Nash equilibrium – as Mark Buchanan explains in an article for Bloomberg:

“…named after the mathematician and economist John Nash… a Nash equilibrium is a set of strategies that, once discovered by a set of players, provides a stable fixed point at which no one has an incentive to depart from their current strategy.”

Not all Nash equilibria are as dramatic as Mutually Assured Destruction, but they are useful:

“…from local regulations to international law, negotiations can be seen as a gradual exploration of all possible moves to find a stable framework of rules acceptable to everyone, and giving no one an incentive to cheat – because doing so would leave them worse off.”

Unless one or both sides lose their minds, the terms of Brexit will be determined by a Nash equilibrium. Much is made of the ‘complexity’ of the Brexit negotiations, but most trade negotiations are bounded by knowable scenarios (which in this case might range from continuing membership of the Single Market to WTO rules only). Moreover, trade agreements tend to be legally binding, limiting room for manoeuvre on the part of the signatories.

But what about negotiations where the possibilities are unbounded and unknowable? This is true complexity (as opposed to mere detail). Buchanan cites a paper by Dimitri Kusnezov and Wendell B. Jones, who argue there are circumstances under which it is impossible to reach a Nash equilibrium:

“Kusnezov and Jones derive insight from recent mathematical studies of games with many players and many possible choices of action. One basic finding is a sharp division into two types, stable and unstable. Below a certain level of complexity, the Nash equilibrium is useful in describing the likely outcomes. Beyond that lies a chaotic zone where players never manage to find stable and reliable strategies, but cope only by perpetually shifting their behaviors in a highly irregular way. What happens is essentially random and unpredictable.

“The authors argue that emerging technologies – especially computing, software and biotechnology such as gene editing – are much more likely to fall into the unstable category. In these areas, disruptions are becoming bigger and more frequent as costs fall and sharing platforms enable open innovation. Hence, such technologies will evolve faster than regulatory frameworks – at least as traditionally conceived – can respond.”

The implication is that the future will be full of rapidly evolving and potentially dangerous technologies that we have no known means of regulating.

Perhaps we need to replace the Nash equilibrium with an ‘Amish equilibrium’ – that is a deliberate decision on the part of the (international) community not to use certain technologies. The Amish – an ethno-religious group mostly to be found in the United States – are famous for restricting their use of modern technology. They do so not because they believe these conveniences to be intrinsically wrong, but because of the potentially disruptive and unpredictable impact of, say, motorised transport on the community’s chosen way of life. By taking the option entirely off the table, there is no need to negotiate a regulatory regime (which may, in any case, be an impossibly complex task).

Whether formally or informally, the international community has already taken the Amish option on some technologies. For instance, the various nuclear test ban treaties have been effective since the 1960s (though not completely). An example of an informal Amish option is the global taboo against eugenics – which, though far from comprehensive, has discouraged once common and accepted practices like forced sterilisation.

The pushback against eugenics shows that technology is not destiny. If we are unable to moderate our use of a dangerous technology then we must, if we can, ban it all together.

---

Peter Franklin is Associate Editor of UnHerd. He was previously a policy advisor and speechwriter on environmental and social issues.

peterfranklin_