Anthropic has told California Governor Gavin Newsom that government officials should be able to ask a court to block the release of an AI model that poses a serious risk to the public. In a letter dated 15 October, the company backed a targeted court power while opposing blanket requirements to build kill switches that could shut down entire models.
What Anthropic asked for
The letter’s central sentence is direct: when a review finds a model poses a serious risk, the government should be able to ask a court to block it. An Anthropic spokesperson confirmed the letter is authentic, while the governor’s office did not immediately respond to a request for comment.
Anthropic submitted the recommendations after Newsom convened a panel of artificial intelligence experts in September to develop proposals for AI safety requirements. The letter therefore lands in an active state process, not as an abstract essay, and it follows the company’s earlier policy proposals on the same theme.
The company also said it opposes requirements that developers build kill switches capable of shutting down entire models if a company loses control over their actions. Instead, it supports more targeted interventions, such as disabling a problematic AI agent or cutting off a specific customer’s access, which it says would be more effective.
The wider fight over who can stop a model
Anthropic itself has faced government restrictions on its AI systems. The Trump administration in June ordered Anthropic to immediately restrict foreign access to its cyber-capable Mythos 5 and Fable 5 models, citing national security concerns, before the Commerce Department later lifted the order.
A bipartisan group of United States senators debated legislation in September that would give the federal government power to block the release of certain models deemed unsafe, with companies able to challenge that decision in federal court. Congress left for its pre-election break shortly after, and lawmakers have not moved forward on that proposal.
United States House lawmakers proposed an AI Kill Switch Act in July, which would give officials power to order firms to shut down models that put human life or the economy at risk. Anthropic’s letter draws its line against that approach, preferring a court order aimed at a specific release over a standing power to switch a model off.
No model is named as a target in the letter, and this is a proposal, not a law, a bill or a court order. That distinction matters, because the debate is about which powers should exist before a dangerous release, not about any single model currently before a judge.
Why It Matters
A frontier lab asking a governor for a court power over its own industry is unusual, and it reframes the regulation debate. Instead of the familiar split between companies resisting rules and officials imposing them, Anthropic is arguing for one specific power while arguing against a broader one, on the grounds that targeted action works better than a blanket switch.
The timing is also worth noting, without asserting a motive. Anthropic is preparing for a United States initial public offering, as covered in our Anthropic IPO report, and its public positions on safety now sit alongside investor scrutiny of its models, its testing, and its government relationships.
Model safety testing is already central to Anthropic’s public record, including its work on training pauses and model escapes, as covered in our S-1 model safety report. A court-block power would give those test findings a legal consequence, turning a safety review into a possible bar on release.
Background
Anthropic’s government dealings run beyond safety letters, including a discounted deal for United States agencies, as covered in our OneGov discount report. Whether California takes up the court-block idea, and how Congress answers the same question, will shape where the next contested model release is decided.
Source: Reuters report
0 Comments