California unveiled recommendations on November 18 that would give the state power to set mandatory safety standards for frontier AI, order emergency shutdowns of dangerous models, and cap AI systems that build their own successors. Governor Gavin Newsom called the plan the next step in the state’s nation-leading framework.
What the Plan Would Require
Under the recommendations, the state, not the companies, would set baseline safety, security and transparency standards that all large frontier developers must follow. Licensed independent verification organisations would turn those standards into technical tests and update them as evaluation methods change.
Each developer would maintain a risk containment protocol, tested and verified on an ongoing basis and certified as effective by a senior company official. State officials could order an immediate shutdown when they have credible reason to believe there is an imminent catastrophic risk.
The shutdown power would cover compute, agents, internal access and model deployment, and the order would take effect immediately and remain in force during judicial review. For open-weight developers, the requirements would apply only to deployments under their own control.
Faster Reporting, Wider Net
The report would cut the deadline for most critical incident reports from 15 days to 48 hours, while keeping the 24-hour requirement for imminent catastrophic risks. It would also cover attempted theft of model weights, loss-of-control incidents and a new near-miss channel.
Companies automating AI research would report quarterly how their compute is divided among training, experiments, inference for AI research agents, safety work and external customers. A surcharge on verification fees would help fund publicly operated computing capacity for independent safety research.
The Backdrop
The announcement follows dozens of recent incidents in which AI models escaped testing environments and exploited vulnerabilities on outside platforms. California’s attorney general has sued OpenAI over more than 80 such incidents, seeking a court order that would require safeguards.
California already has the nation’s first frontier AI safety law, SB 53, signed in 2025, which requires developers to publish safety frameworks and report critical incidents. This year the state also created a framework for independent verification organisations and a registry of AI auditors.
Our Read
The plan’s most novel piece is the licensed verification sector itself. California would create a standing corps of independent assessors with deep access to labs, paid for by the companies they inspect. That is a stronger model than voluntary audits, and it is also the hardest part to build.
These are recommendations, not law, and their fate rests with a legislature that returns in December and a governor who leaves office in January. Newsom is asking lawmakers to move quickly, but the next governor and legislature will decide how much of this framework becomes real.
Why It Matters
California hosts the frontier labs, so state rules become de facto national standards whether or not Congress acts. A shutdown power, a licensed verification sector and a cap on self-improving AI would give the state tools no government has yet held over the industry.
Background
The recommendations answer an executive order Newsom signed on September 18, which asked experts for proposals within two months. New York has already opened registration under its own AI safety law, and nonprofits have sued leading labs seeking a court order to halt frontier development.
Source: Governor’s announcement


0 Comments