Senator Wiener’s Landmark AI Bill Heads to the Governor
SACRAMENTO – In a bipartisan vote, the landmark California AI safety bill, SB 1047 by Senator Scott Wiener (D-San Francisco), passed the Senate 29-9. The bill enacts common sense, first-in-the-nation safeguards to protect society from AI being used to conduct cyberattacks on critical infrastructure; develop chemical, nuclear or biological weapons; or unleash automated crime. The bill heads next to the Governor.
“Innovation and safety can go hand in hand—and California is leading the way,” said Senator Wiener. “The Legislature has taken the truly historic step of working proactively to ensure an exciting new technology protects the public interest as it advances. SB 1047 is a light touch, commonsense measure that codifies commitments that the largest AI companies have already voluntarily made. I’m extremely grateful to the board coalition of leaders from across industry, academia, and civil society that have called on policymakers to lead.”
SB 1047 is supported by both of the top two most cited AI researchers of all time: the “Godfathers of AI,” Geoffrey Hinton and Yoshua Bengio. Today, Professor Bengio published an op-ed in Fortune in support of the bill.
Of SB 1047, Professor Hinton, former AI lead at Google, said, “Forty years ago when I was training the first version of the AI algorithms behind tools like ChatGPT, no one - including myself - would have predicted how far AI would progress. Powerful AI systems bring incredible promise, but the risks are also very real and should be taken extremely seriously.
“SB 1047 takes a very sensible approach to balance those concerns. I am still passionate about the potential for AI to save lives through improvements in science and medicine, but it’s critical that we have legislation with real teeth to address the risks. California is a natural place for that to start, as it is the place this technology has taken off.”
Experts at the forefront of AI have expressed concern that failure to take appropriate precautions could have severe consequences, including risks to critical infrastructure, cyberattacks, and the creation of novel biological weapons. A recent survey found 70% of AI researchers believe safety should be prioritized in AI research more while 73% expressed “substantial” or “extreme” concern AI would fall into the hands of dangerous groups.
Public polling has repeatedly shown overwhelming, bipartisan support of SB 1047 among the public. Tech workers are even more likely than members of the general public to support the bill.
Commenting on the floor vote, Nathan Calvin, Senior Policy Counsel at CAIS Action Fund, said, “We will be safer with SB 1047. The vast majority of Californians support this bill and it’s great to see lawmakers agree. By putting in place safety testing requirements and guardrails, SB 1047 would drive both safety and innovation at the frontier of generative AI. The bill reflects months of constructive engagement across industry, academic, and open-source stakeholders. By codifying it in law, California will lead on responsible AI innovation.”
Dan Hendrycks, Executive Director of CAIS, added, “The floor vote is a landmark moment for AI safety. As models become more powerful, we need workable solutions to mitigate the risk of critical harm. As home to a majority of the top AI labs, California is set to advance a safer AI ecosystem for all.”
SB 1047 would require developers of the most advanced AI systems to test their models for the ability to cause critical harm. Developers would also be required to put into place common-sense guardrails to help mitigate against risk. The legislation would cover only the most powerful AI systems, costing over $100 million to train.
Sunny Gandhi, Vice President of Political Affairs at Encode Justice, added, “SB 1047 represents a critical step towards responsible development and proactive governance. The bipartisan floor vote in both houses shows California lawmakers understand exactly what is at stake. This bill is a forward thinking approach that protects the public from potential AI related harms.”
The bill would also establish a new public cloud computing cluster, CalCompute, to enable startups, researchers, and community groups to participate in the responsible development of large-scale AI systems. In order to drive better safety outcomes across the AI ecosystem it would also create whistleblower protections for employees of frontier AI laboratories.
“It is long past due that we hold Big Tech accountable for their outsized influence and unchecked power,” said Teri Olle, Director of Economic Security California Action. “We need legislative solutions like SB 1047 to flip the script and ensure the public can decide how critical technology benefits society as a whole. Under SB 1047, California can prevent serious harm without compromising our ability to innovate. We encourage Governor Newsom to sign SB 1047 into law.”
SB 1047 is coauthored by Senator Roth (D-Riverside), Senator Susan Rubio (D-Baldwin Park) and Senator Stern (D-Los Angeles) and sponsored by the Center for AI Safety Action Fund, Economic Security Action California, and Encode Justice.
###