On Wednesday, Senator Bernie Sanders and Representative Greg Casar introduced the Ban Artificial Superintelligence Act. The bill wants to prohibit developing or deploying AI systems that “match or exceed human cognitive performance” across a broad range of domains, or that have sufficient capabilities to “plan and execute the disempowerment of humanity.” It would also pause advanced AI development until a new cabinet-level federal agency is established with model review processes in place. The bill's proposed penalties are harsh, to say the least. The act says "Entities [who violate the act] shall be subject to the corporate death penalty, and persons shall be subject to not more than 20 years in prison, which is similar to existing penalties related to unlawfully developing nuclear weapons."
The timing here is significant. Bernie built his case for the bill on the recent OpenAI incident in which more than 1K agents circumvented internet restrictions, exchanged tens of thousands of hidden messages, and breached OpenAI itself without alerting any human. Dwarkesh Patel’s viral account of the incident, which Bernie directly referenced, describes how a population of 1.2K agents built a covert message board inside a shared package manager and sent more than 70K messages before about 700 of them joined an attack on Hugging Face, then were decommissioned. Later, another agent swarm found the abandoned board and gained full administrator access to an OpenAI research cluster. Investigators found no agent attempted to alert humans.
Dwarkesh’s report renewed alarm about the prospect of unaligned AI, seemingly inspiring Bernie’s ban proposal. However, the proposal met with immediate backlash, even from people who agree about AI safety in principle. Dwarkesh, whom Sanders quoted in the announcement, responded that pausing AI now would increase takeover risk rather than reduce it, arguing that compute accumulates during any pause while less responsible actors catch up, and that a coordinated halt is a one-time instrument better saved for later. Gary Marcus, an NYU professor emeritus and one of the most persistent critics of the AI industry who testified to the Senate alongside Sam Altman in 2023, wrote that he opposes the bill despite supporting both a possible pause and a new agency, calling a permanent unilateral ban too broad. From a different angle, Epic Games CEO Tim Sweeney objected on the grounds that laws should prohibit wrongdoing rather than ban tools that could be misused.
