June 29, 2021

The convenience of the smart home may be worth the price; that’s for each of us to decide. But to do so with open eyes, one has to understand what the price is. After all, you don’t pay a monthly fee for Alexa, or Google Home. The cost, then, is a subtle one: a slight psychological adjustment in which we are tipped a bit further into passivity and dependence. 

The Sleep Number Bed is typical of smart home devices, as Harvard business school Professor Shoshana Zuboff describes in The Age of Surveillance CapitalismIt comes with an app, of course, which you’ll need to install to get the full benefits. Benefits for whom? Well, to know that you would need to spend some time with the 16-page privacy policy that comes with the bed. There you’ll read about third-party sharing, analytics partners, targeted advertising, and much else. Meanwhile, the User Agreement specifies that the company can share or exploit your personal information even “after you deactivate or cancel … your Sleep Number account.” You are unilaterally informed that the firm does not honor “Do Not Track” notifications. By the way, the bed also transmits the audio signals in your bedroom. (I am not making this up.)

The business rationale for the smart home is to bring the intimate patterns of life into the fold of the surveillance economy, which has a one-way mirror quality. Increasingly, every aspect of our lives — our voices, our facial expressions, our political affiliations and intellectual predilections — are laid bare as a data to be collected by companies who, for their own part, guard with military-grade secrecy the algorithms by which to use this information to determine the world that is presented to us, for example when we enter a search term, or in our news feeds. They are also in a position to determine our standing in the reputational economy. The credit rating agencies and insurance companies would like to know us more intimately; I suppose Alexa can help with that.

Allow me to offer a point of reference that comes from outside the tech debates, but can be brought to bear on them. Conservative legal scholars have long criticized a shift of power from Congress to the administrative state, which seeks to bypass legislation and rule by executive fiat, through administrative rulings. The appeal of this move is that it saves one the effort of persuading others, that is, the inconvenience of democratic politics. 

All of the arguments that conservatives make about the administrative state apply as well to this new thing, call it algorithmic governance, that operates through artificial intelligence developed in the private sector. It too is a form of power that is not required to give an account of itself, and is therefore insulated from democratic pressures. 

In machine learning, an array of variables are fed into deeply layered “neural nets” that simulate the binary, fire/don’t-fire synaptic connections of an animal brain. Vast amounts of data are used in a massively iterated (and, in some versions, unsupervised) training regimen. Because the strength of connections between logical nodes is highly plastic, just like neural pathways, the machine gets trained by trial and error and is able to arrive at something resembling knowledge of the world. The logic by which an AI reaches its conclusions is impossible to reconstruct even for those who built the underlying algorithms. We need to consider the significance of this in the light of our political traditions.

When a court issues a decision, the judge writes an opinion in which he explains his reasoning. He grounds the decision in law, precedent, common sense, and principles that he feels obliged to articulate and defend. This is what transforms the decision from mere fiat into something that is politically legitimate, capable of securing the assent of a free people. It makes the difference between simple power and authority. One distinguishing feature of a modern, liberal society is that authority is supposed to have this rational quality to it — rather than appealing to, say, a special talent for priestly divination. This is our Enlightenment inheritance. It appears to be in a fragile state. With the inscrutable arcana of data science, a new priesthood peers into a hidden layer of reality that is revealed only by a self-taught AI program — the logic of which is beyond human knowing.

The feeling that one is ruled by a class of experts who cannot be addressed, who cannot be held to account, has surely contributed to populist anger. From the perspective of ordinary citizens, the usual distinction between government and “the private sector” starts to sound like a joke, given how the tech firms order our lives in far-reaching ways.

Google, Facebook, Twitter and Amazon have established portals that people feel they have to pass through to conduct the business of life, and to participate in the common life of the nation. Such bottlenecks are a natural consequence of “the network effect.” It was early innovations that allowed these firms to take up their positions. But it is not innovation, it is these established positions, and the ongoing control of the data it allows them to gather, that accounts for the unprecedented rents they are able to collect, as in a classic infrastructure monopoly. If those profits measure anything at all, it is the reach of a grid of surveillance that continues to spread and deepen. It is this grid’s basic lack of intelligibility that renders it politically unaccountable. Yet accountability is the very essence of representative government. 

Mark Zuckerberg has said frankly that “In a lot of ways Facebook is more like a government than a traditional company.” If we take the man at his word, it would seem to raise the question: can the United States government tolerate the existence of a rival government within its territory? 

In 1776, Americans answered that question with a resounding ”No!” and then fought a revolutionary war to make it so. The slogan of that war was “don’t tread on me.” This spirited insistence on self-rule expresses the psychic core of republicanism. As Senator Amy Klobuchar points out in her book Antitrust, the slogan was directed in particular at the British Crown’s grant of monopoly charters to corporations that controlled trade with the colonies. 

Today, the platform firms appear to many as an imperial power. The fundamental question “who rules?” is pressed upon America once again.

This is an edited version of Matthew B. Crawford’s testimony before the US Senate Judiciary Committee on June 15th 2021.