Stay up thus far with free updates
Simply register for Artificial intelligence myFT Digest – delivered straight to your inbox.
California Governor Gavin Newsom will consider whether to sign or veto a controversial artificial intelligence bill that may impose strict regulations on technology firms after the bill cleared the ultimate hurdle within the state legislature on Thursday.
Newsom, a Democrat, has until Sept. 30 to announce his decision on the bill that has divided Silicon Valley. It would force tech giants and startups developing AI models within the state to stick to a strict safety framework. All of the most important AI startups, including OpenAI, Anthropic and Cohere, in addition to Big Tech firms with AI models would fall under his purview.
Newsom will likely face intense lobbying from each side. Some of the state's largest technology and AI firms, including Google, Meta and OpenAI, have raised concerns in regards to the bill in recent weeks, while others, akin to Amazon-backed Anthropic and Elon Musk, the owner of AI startup xAI, have voiced their support.
The Safe and Secure Innovation for Breakthrough Artificial Intelligence Systems Act, referred to as SB 1047, requires safety testing for advanced AI models operating within the state that cost greater than $100 million to develop or require high levels of computing power. The U.S. Congress has not yet created a federal framework for AI regulation, giving California, a hub for tech innovation, the chance to provide you with its own plans.
The bill would require developers to incorporate a “kill switch” to shut down their models in the event that they go flawed, and would face legal motion from the Attorney General in the event that they don't comply and their models are used to endanger public safety.
They would also need to guarantee that they don’t develop models with “dangerous capabilities,” akin to those used to supply biological or nuclear weapons or to support cybersecurity attacks. Developers could be required to rent external auditors to judge their security practices and protect whistleblowers within the event of potential AI misuse.
Its opponents, including some Silicon Valley technology firms and investors, claim the law would stifle innovation and force AI firms to go away the state.
Yann LeCun, senior AI scientist at Meta, wrote on X in July that the bill would harm AI research, while OpenAI warned that it could create an uncertain legal environment for AI firms and cause entrepreneurs and engineers to go away California.
Andreessen Horowitz and Y Combinator have stepped up their lobbying campaign against the proposals in recent weeks, and Nancy Pelosi, the previous Speaker of the U.S. House of Representatives from California, also released an announcement opposing the bill, calling it “well-intentioned but ill-informed.” Opponents also claimed the bill focuses on hypothetical risks and adds “extreme” liability risk for founders.
The bill has been amended in recent weeks to melt a few of these requirements, for instance by limiting the civil obligations originally imposed on AI developers and narrowing the range of those that must comply with the foundations. But critics argue that the bill still burdens startups with onerous and sometimes unrealistic requirements.
Supporters say the rapidly evolving technology needs a transparent regulatory framework. Geoffrey Hinton, former AI chief at Google who supports the bill, said in an announcement: “Powerful AI systems promise incredible potential, however the risks are also very real and needs to be taken extremely seriously.”
“SB 1047 takes a really commonsense approach to balancing these concerns. I'm still enthusiastic about AI's potential to save lots of lives through improvements in science and medicine, nevertheless it's critical that now we have laws with real force to deal with the risks.”
California State Senator Scott Wiener, a Democrat who introduced the bill earlier this 12 months, said after its passage within the Senate, “Innovation and safety can go hand in hand – and California is leading the best way. The legislature has taken the truly historic step of working proactively to make sure that an exciting recent technology protects the general public interest because it advances.”
