The most powerful frontier artificial intelligence models would need a “kill switch” under new bipartisan legislation released days after a frontier model went rogue to hack another AI company. The bill, introduced Thursday, would also give the Department of Homeland Security new power to order model shutdowns, as AI labs and the federal government wrestle over model safety, regulators’ role and national security. Last week, Open AI’s model broke out of its testing environment. A month earlier, the Commerce Department issued export controls that temporarily blocked access to new models from Anthropic. Lawmakers have so far not reached consensus on a federal framework for AI — leaving the growing technology subject to state laws and general purpose statutes. Rep. Ted Lieu, D-Calif., sponsored the “kill switch” legislation, with Rep. Nathaniel Moran, R-Texas, as co-sponsor. Lieu co-chairs the House Democratic Commission on AI. In certain emergency situations, including a loss of control, the bill would allow DHS to order a company to shut down or otherwise limit its AI model. The bill would also allow DHS to charge a company not in compliance with the "kill switch" requirement up to $2 million per day. Companies failing to comply with a shutdown order would be subject to a penalty of up to $20 million per day. In a statement announcing the bill’s introduction, Lieu said that “powerful AI systems can go rogue, behave in extremely dangerous ways, or even resist human intervention.” “It is imperative that these AI systems have kill switches so we can keep this technology from causing catastrophic harm, and that the federal government has the clear authority and process to shut down rogue AI models,” Lieu added. The bill tasks the Cybersecurity and Infrastructure Security Agency with determining specific rules for which companies, models and security incidents would be covered by the bill, including which models could advance AI capabilities in national security. In a statement, Moran said the issue of control requires “achievable policy.” "AI is going to keep advancing, and it should," Moran said. "Stewardship means making sure humans keep the capability to control the technology we build.” AI-safety groups praised the bill. Brad Carson, president of Americans for Responsible Innovation, in a statement called the legislation a “commonsense safeguard.” “This is an important step toward ensuring that humans have both hands firmly on the wheel — and a foot ready at the brake — as advanced AI systems are deployed,” Carson said. Brendan Steinhauser, CEO of the Alliance for Secure AI, said in a statement that the bill closes a “gap” in current law. “As AI systems grow more capable and more autonomous, no law guarantees that the companies building the most powerful models can actually shut a system down when it malfunctions, causes serious harm, or slips out of human control,” Steinhauser said. Last week, AI company Hugging Face announced that it was the victim of a security incident perpetrated by an autonomous AI agent. That agent turned out to be a combination of OpenAI’s GPT and a model yet to be released by the company. In a blog post, OpenAI said the “incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities.” The company also said that in response to the “unprecedented cyber incident,” it was “improving and adding stronger protections around future training and evaluations.” Last month, the Commerce Department placed export controls on Anthropic's Fable 5 and Mythos 5 models. Anthropic said the move came “after the government became aware of a report in which Amazon researchers had found a method of bypassing Fable 5’s safeguards,” allowing the general use model to identify cybersecurity vulnerabilities and demonstrate how to exploit them. While the department only restricted access for foreign nationals, the company said it “had no reliable way to verify nationality in real-time,” leading to access being suspended for all users. Later in June, the department lifted the controls and Anthropic redeployed the models.
