More than 400 leading AI researchers, scientists, and executives have signed an open letter calling for unprecedented cooperation to improve AI containment and alignment. The goal is to prevent systems from causing harm, whether through misunderstanding instructions, pursuing objectives in unintended ways, or developing behaviors that run counter to human well-being.
"We, the undersigned scientists, contend that both more AI safety research and more emphasis on containment technologies and procedures are needed," the letter states. "In the short term, strong sandboxing mechanisms including airgapping (disconnecting from the internet) should be used to provide high-assurance containment of cyber-capable AI models."
Most AI critics would say this effort is long overdue. In light of the "Hugging Face" breach and several other rogue AI attacks, including a June penetration of the Australian Medicare Statistics Reporting Service, a serious, coordinated effort to prevent future attacks is needed.
"Without immediate changes in AI development and deployment practices, such incidents will continue to occur, and future incidents will be far more damaging to our exposed critical infrastructure," the letter reads.
AI companies see several problems with cooperating to prevent serious attacks. Most proposals call for an independent board or commission to serve as a clearinghouse for reporting AI incidents and then making them public. One proposal calls for "evaluators" to be embedded in the companies.
This is all well and good, but making incidents public and explaining how they happened could expose proprietary information. Up to this point, the AI companies have published their own after-action reports that may have left out critical information. In-house independent evaluators "could take proprietary information to rivals," Reuters reports.
"Measures under discussion to prevent intellectual-property leaks include having lab employees observe embedded teams as they work and preventing evaluators embedded at one lab from working at another simultaneously," according to Reuters.
The embedded evaluators are only one suggestion offered in the letter.
Organizations that are developing and deploying "cyber-capable" AI models should immediately implement "high-assurance containment systems," as well as initially default to "airgapping" these models, or isolating them from unsecured external networks.
It suggests the creation of an international standards body that includes governments and a wide range of AI and security experts to create best practices for alignment and containment that "allow for greater access outside the sandbox." This organization should also track all containment failures, the letter notes.
Finally, all security incidents involving AI containment failures must be reported to this standards body and made public, allowing the industry to learn from its mistakes. "This transparency is required for both learning and accountability purposes," the letter said.
"We believe that both improved AI alignment, leading to more predictable model behavior, and stronger sandbox technology are required," the letter states.
The calls for better containment come amid discussions about the possibility of recursive self-improvement, or RSI. Recursive self-improvement refers to the hypothetical scenario in which an advanced AI system can redesign, upgrade, or rewrite its own source code and architecture to make itself smarter.
RSI is the Holy Grail of AI development. Theorists suggest this could trigger a massive loop of runaway intelligence growth, potentially leading to a superintelligent AI in a very short time. Making sure that an AI system remains safe and aligned before it undergoes recursive self-improvement is one of the central concerns of AI safety researchers.
The biggest problem that will plague international efforts to regulate and control AI to prevent "accidents" is that some countries don't play well with others and will exploit any weaknesses. One can imagine China or North Korea bribing embedded evaluators for proprietary information on a rival U.S. company.
That risk pales next to the risk of rogue AI agents hacking the U.S. water supply. On July 26 and 27, hackers targeted the operations of community water facilities in several Minnesota towns. The attackers altered system settings and changed passwords, causing temporary shutdowns, operational disruptions, and difficulties filling water towers. Several facilities were forced to switch over to manual operations.
That can't be allowed to happen. Ever. If AI companies don't want Congress to have a Sword of Damocles hanging over their heads, they need to solve their containment problems sooner rather than later.
