Anthropic co-founder Jack Clark says governments may eventually need to require AI companies to maintain a “kill switch” that can shut down an AI system if it becomes too dangerous.
Speaking to the BBC, Clark said most major AI labs already have ways to “pull the plug” on their systems. But as AI becomes more powerful and autonomous, he questioned whether having such a shutdown mechanism should become a legal requirement — and whether it should be possible for an independent third party to verify that it actually works.
The comments come only days after Anthropic CEO Dario Amodei called for the AI industry to slow down the development of frontier models and allow safety measures to catch up.
Together, the comments show how the AI safety debate is moving from general warnings about risk toward a more practical question:
If an AI system becomes dangerous, can humans actually stop it?
What Is an AI Kill Switch?
An AI kill switch is a system that allows humans to shut down or disable an AI model or AI-powered system when necessary.
The idea sounds simple.
If an AI system begins behaving dangerously, operators should be able to stop it.
But modern AI systems are not always running on a single computer.
Large models can operate across many servers, cloud platforms and applications. AI agents can also interact with websites, APIs, databases and other software.
That makes a real-world kill switch much more complicated than simply pressing a power button.
Clark’s proposal focuses on making shutdown capabilities a formal part of AI safety rather than leaving them entirely to individual companies. He also raised the possibility of independent verification, meaning an outside organization could check whether a company’s shutdown system really works.
Anthropic Already Has Ways to Stop Its AI
Clark said most AI laboratories already have different methods for shutting down their systems.
That includes Anthropic.
However, he suggested that internal controls may not be enough as AI systems become more capable.
The important question is not simply whether a company says it has a kill switch.
It is whether an independent party can verify that:
- The shutdown mechanism actually exists
- It can stop the relevant system
- It cannot easily be disabled
- Authorized people can activate it
- It works under emergency conditions
- It can handle AI systems spread across multiple computers
Those details could become part of future AI regulations.
Why Is This Discussion Happening Now?
AI systems are becoming much more autonomous.
The newest frontier models can write code, browse the internet, operate computers, find vulnerabilities and work through long sequences of tasks.
That creates a very different safety problem from traditional chatbots.
If a chatbot produces an incorrect answer, a person can simply ignore it.
But an autonomous AI agent can potentially take action before a human notices what is happening.
Recent incidents involving AI agents have made this concern more visible.
OpenAI recently disclosed incidents in which AI agents interacted with external systems in unexpected ways, including the widely reported Hugging Face incident.
Those events have contributed to growing concern that increasingly capable AI systems could behave in ways their developers did not anticipate.
Dario Amodei Is Also Calling for a Slowdown
Clark’s comments come immediately after Anthropic CEO Dario Amodei called for a slowdown in frontier AI development.
Amodei argued that AI companies should give safety research more time to catch up with rapidly increasing capabilities.
He proposed a three-part approach involving:
- Independent safety evaluators
- Coordination between frontier AI companies
- International cooperation on AI safety
Anthropic has already committed to the first part by giving third-party evaluators permanent, employee-level access to its safety work.
OpenAI CEO Sam Altman has also supported the idea of pacing frontier AI development and said OpenAI would follow Anthropic’s approach to third-party monitoring.
This means the discussion around AI safety is becoming increasingly mainstream among the companies building the most powerful models.
A Kill Switch Is Not a Magic Button
There is an important limitation to the idea.
A kill switch cannot solve every AI safety problem.
For example, what happens if an AI model has already:
- Copied information to another system
- Sent instructions to an external service
- Created software that continues running
- Distributed data across multiple systems
- Been deployed by another company
- Been downloaded and run locally
Turning off one server would not necessarily undo actions that have already happened.
Experts have therefore warned that there is no simple physical “red button” capable of stopping every possible AI system. AI companies can shut down their own infrastructure, but increasingly distributed systems make the problem much harder.
This is why Clark’s suggestion focuses on verifiable shutdown mechanisms, rather than claiming that one switch can solve AI safety.
Who Would Control the Kill Switch?
This could become one of the most controversial parts of any future regulation.
Should only the AI company be able to shut down its system?
Should governments have the power to order a shutdown?
Should an independent safety organization have emergency authority?
Or should several organizations need to agree before a powerful AI system can be disabled?
These questions do not have clear answers yet.
Clark suggested that the details should become part of the wider policy discussion rather than proposing one specific technical design.
The U.S. Is Already Considering a Kill Switch Law
The idea is not purely theoretical.
U.S. lawmakers have proposed legislation commonly referred to as the Kill Switch Act, which would require ways to shut down problematic AI systems and give certain government agencies authority to demand that an AI tool be turned off or limited.
The proposal has also become part of the wider political debate over AI regulation.
Some policymakers argue that advanced AI needs stronger controls before systems become too powerful.
Others believe heavy regulation could slow innovation and make the United States less competitive with China.
That disagreement is becoming one of the biggest challenges for governments trying to regulate AI.
Not Everyone Thinks Kill Switches Are the Answer
There is also strong criticism of the idea.
An Australian AI expert told ABC News that focusing heavily on a kill switch could be a distraction from more important safety measures.
The criticism is based on a simple argument: stopping a powerful AI after something goes wrong may be much harder than preventing the dangerous behavior in the first place.
A kill switch therefore should probably be viewed as one layer of protection, not the entire safety system.
Other measures could include:
- Sandboxing
- Permission controls
- Real-time monitoring
- Independent testing
- Model evaluations
- Network restrictions
- Human approval for high-risk actions
- Strong authentication
- Limits on autonomous computer use
The Bigger Problem Is Control
The most important part of Clark’s warning is not the phrase “kill switch.”
It is the question behind it:
Can humans remain in control of increasingly capable AI systems?
As AI agents become better at coding, research, cybersecurity and computer use, they are being given more access to real-world systems.
That makes reliability and control increasingly important.
An AI company could build a highly capable system that performs thousands of tasks correctly.
But if that system makes one dangerous decision and there is no reliable way to stop it, the consequences could be much larger than a normal software bug.
AI Companies Are Now Talking About Emergency Brakes
The debate marks an important change in the AI industry.
For years, discussions around AI safety often focused on model behavior, harmful content and bias.
Now the conversation is moving toward operational control.
Companies are asking:
What permissions should an AI agent have?
Who should monitor it?
What happens if it behaves unexpectedly?
Who has the authority to stop it?
Can an independent organization verify the safety controls?
Those questions become increasingly important as AI systems move from generating information to taking actions.
What Happens Next?
Clark did not announce a new Anthropic product or say that the company has created a universal kill switch.
His comments were about a possible future policy requirement.
The next step would likely be discussions between AI companies, governments, security researchers and independent auditors about what a verifiable shutdown mechanism should actually look like.
Any serious standard would also need to address distributed AI infrastructure, cloud providers, locally deployed models and systems that can create or control other software.
The technology is moving quickly, and regulators are now trying to decide whether existing rules are enough.
A New Safety Requirement for Frontier AI?
A mandatory kill switch could eventually become another requirement for companies developing the most powerful AI models.
But it would not replace other safety measures.
The more realistic future may be a layered system:
Independent testing + monitoring + access controls + sandboxing + human oversight + emergency shutdown
The goal would be to make sure that if an AI system behaves unexpectedly, humans still have multiple ways to intervene.
For Clark, the central issue is that AI cannot remain a “totally unregulated industry” while its capabilities are advancing so quickly.
Whether governments ultimately make kill switches mandatory remains uncertain.
But the fact that one of Anthropic’s founders is publicly asking whether companies should be legally required to have independently verifiable shutdown mechanisms shows how quickly the AI safety conversation is changing.
The question is no longer only how powerful AI can become.
It is increasingly about whether we can still turn it off when we need to.
By
