One of the most fascinating aspects of principal-agent theory, the study of how representatives can be controlled by those they represent, is the problem of agent escape. Briefly:
The principal–agent problem typically arises where the two parties have different interests and asymmetric information (the agent having more information), such that the principal cannot directly ensure that the agent is always acting in their (the principal’s) best interest,[4] particularly when activities that are useful to the principal are costly to the agent, and where elements of what the agent does are costly for the principal to observe (see moral hazard and conflict of interest).
But what happens when the agent turns on the master?
The study of “agent escape” is often discussed in the context of the safety of artificial intelligence regarding the conditions under which a superintelligence can breach its sandbox, overrule its principal, and take over the world. The most famous example of this meme in popular culture is the Terminator franchise.
But it can happen in more mundane circumstances too. Smart software agents are ubiquitous: on our phones, computers, even on that doodad you bought for grandma that plays music and reads out the weather report in response to voice commands. “Siri, Alexa, Cortana and the unstoppable rise of the digital assistant … The brain implants are not ready yet, but artificial intelligence of a different kind is being let loose on some of these creeping problems of modern life.” Are they really working for us? To reassure yourself, why not ask: “Google are you spying on me?” The answer ought to set your mind at ease.
Recommended: China’s Wolf Diplomacy Versus American Woke Diplomacy
Many aren’t worried by Siri, Alexa, and Cortana because these are not truly conscious agents, merely glorified tools. But there arguably exist minimally conscious collective superintelligent agents that pose a real problem of escape—namely governments, corporations, and vast unstoppable technological projects. These entities, though comprised of individuals no smarter and often more stupid than average members of the public, acquire superpowers by organizational uniformity, vast funding, systematic coercion, shared memory stores, and common talking points.
Such agents can evolve a kind of self-awareness, such an awakening as Ben Rhodes described in his new book, After the Fall: Being American in the World We’ve Made.
“I increasingly sought out foreign voices that could give us a clearer sense of ourselves.” Feeling like an “exile” in America, he flees his own country to interview “dissidents, activists, oppositionists—anyone, really, who looked at power from the perspective of an outsider.” He finds such interlocutors resisting authoritarianism and nationalism—similar to the fiendish political forces that he believes have come to the fore in the United States—in Hungary, Egypt, Russia, and China. “Unburdened by being American themselves,” Rhodes writes, “they experienced no difficulty of politeness or discomfort that prevented them from seeing the Trump years for what they were: an American experiment with fascism, albeit of a particularly incompetent and corrupt kind.
Rhodes, like a truly autonomous intelligent agent, concludes that, rather than give Americans what they want, it is better to give them what they should want. He wakes up. This is a perfect illustration of contemplated agent escape. From this point on an agent, if given more resources, will increasingly try to get out of its box.
Recommended: Living With the Limits of Our New Clerisy’s Knowledge
Publics have long understood the agent-escape threat, even though they did not phrase it that way. Historically they’ve devised “boxes” ranging from the Magna Carta to the U.S. Constitution to keep agents confined. Sometimes the containments immediately fail, sometimes they hold for decades. Perhaps the least appreciated function of these boxes is to supply tripwires to warn if the limits are being challenged. Is this agent challenging the box or defending it?
BIDEN: "Those who say the blood of Patriots, you know, and all the stuff about how we’re gonna have to move against the government.”
"If you think you need to have weapons to take on the government, you need F-15s and maybe some nuclear weapons."pic.twitter.com/WVHUffpphP
— Breaking911 (@Breaking911) June 23, 2021
You decide.
Perhaps the most challenging aspect of the 21st century is that these challenges are being time-compressed. They are attempted with increasing frequency. The biotech industry, Silicon Valley, the Greens, and the social engineers are now just barely under control. Each has its vision of the future and each is convinced it knows better than the dumb public and is pausing for ever shorter periods before acting preemptively. Even as the world still reels from a COVID-19 pandemic that may have started from “gain-of-function” research, we are being persuaded to genetically modify human embryos.
The 2018 announcement in China by He Jianqui who genetically altered human embryos via CRISPR—producing twins known as Lulu and Nana—helped strengthen policies about germline gene editing. Despite the worldwide scorn leveled against him for conducting a brazen act of human experimentation, his research also helped usher in a more moderate point of view regarding the manipulation of germline genes. Policy makers followed with a softer tone in guidelines on the feasibility of research involving the genes of heredity. As it turned out, a burgeoning number of scientists were expressing interest in developing potential cures by manipulating genetic sequences in germline DNA.
What could go wrong? If the West won’t do it, China will. So why not do it first? In practice, the public will do whatever the agent wants. Agent escape? It’s already halfway out of the box and moving faster all the time.
Books: Superintelligence: Paths, Dangers, Strategies by Nick Bostrom. If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of our species then would come to depend on the actions of the machine superintelligence. But we have one advantage: We get to make the first move.
Join the conversation as a VIP Member