Anthropic has told California Governor Gavin Newsom that government officials should be able to seek court orders blocking the release of AI models that pose a “serious risk” to the public, according to an October 15 letter whose details were reported by Reuters on October 28, 2026. The company, which is preparing for a US initial public offering, said a court, not an automatic shutdown switch, should be the mechanism for stopping a dangerous release. It also opposed requirements that developers build “kill switches” capable of shutting down entire models, arguing that more targeted interventions, such as disabling a problematic AI agent or cutting off a specific customer’s access, would be more effective.
What The Letter Proposes
The core proposal is judicial rather than administrative. When a review finds that a model poses a serious risk, Anthropic’s letter says, the government should be able to ask a court to block it. That framing matters because it puts a judge between a regulator’s concern and a company’s product, with evidence, argument and appeal, rather than giving an agency a unilateral off switch. It echoes Anthropic’s earlier policy proposals, and it arrives after Governor Newsom convened a panel of AI experts in September to develop proposals for AI safety requirements in California.
An Anthropic spokesperson confirmed to Reuters that the letter is authentic, while the governor’s office did not immediately respond to a request for comment. Politico first reported the details of the letter. The timing is notable: California is the home state of most leading AI companies, and its rules tend to travel, so a position taken by Anthropic in Sacramento is effectively a position taken for the national debate as well.
Why A Company Would Ask For Regulation
It is unusual, though not unprecedented, for a leading company to ask for a power that could one day be used against its own products. Anthropic’s calculation appears to be that regulation is coming regardless, and that shaping it early is better than fighting it later. A court-ordered block, with a serious-risk threshold and judicial oversight, is a narrower and more predictable regime than an open-ended agency power, and it may also be more defensible in court if a company ever challenges a block.
Anthropic itself has faced government restrictions, which gives the letter a particular edge. The Trump administration in June ordered the company to immediately restrict foreign access to its cyber-capable Mythos 5 and Fable 5 models, citing national security concerns, before the Commerce Department later lifted the order. That experience may explain the preference for courts: an executive order can arrive quickly and disappear just as quickly, while a court order at least requires a case to be made and tested.
The Wider Legislative Picture
The California letter does not exist in isolation. A bipartisan group of US senators debated legislation in September that would give the federal government the power to block the release of certain AI models deemed unsafe, with companies able to challenge the decision in federal court, but Congress left for its pre-election break shortly afterwards and lawmakers have not moved forward on that proposal. The Senate idea and Anthropic’s California proposal share a judicial instinct: block, if at all, through a process a company can contest, rather than through an unreviewable switch.
What Happens Next
The key test will be the threshold itself. “Serious risk” sounds clear in a letter and becomes contested the moment it is applied to a real model, where the company will argue the risk is manageable and critics will argue it is already realised. Courts are good at resolving such disputes slowly and carefully; they are less good at resolving them quickly, which is precisely when a dangerous release matters most. Anthropic’s proposal therefore trades speed for legitimacy, and its success will be measured by whether that trade still looks wise when the first block is actually sought.
Why It Matters
This matters because it reframes the AI safety debate from whether governments should be able to stop a model to how they should do it. A kill switch is a power exercised in secret and in haste; a court order is a power exercised in public and on evidence, and Anthropic is betting that the second kind of power is both more legitimate and more durable. If California adopts that model, it could become the template for federal rules and for other jurisdictions, much as government adoption of Claude is setting procurement norms, while research into how experts defer to AI agents shows why oversight matters. The sceptical view is that courts are too slow for the risks they are meant to contain, and that a company proposing its own regulator is choosing the gentlest available leash; the optimistic view is that judicial oversight is the only form of control that survives contact with both politics and the courts. Either way, the question is no longer whether AI will be regulated, but who gets to pull the lever, and under what rules.
Background
Anthropic is an AI company preparing for a US initial public offering, and it submitted its recommendations after California Governor Gavin Newsom convened a panel of AI experts in September to develop proposals for AI safety requirements.
Source: Reuters report
0 Comments