Machine safety has a last line of defence. If everything else goes wrong, a person can stop the machine. The red button, the lanyard, the breaker in the wall. The emergency stop takes priority over every other function of the machine and brings it to a halt, cutting power to the moving parts either at once or after a controlled stop. In Europe that is not just good practice: machinery law requires it, and the standards most manufacturers use to show they comply set out how it must work1, 2.
That assumption no longer holds for a growing number of the machines now being sold. There is no public record of which machines a person can still stop. That is why I am starting this organisation.
One machine. The robots now reaching the market are controlled by AI models rather than hand-written controllers, and their stop mechanisms have followed the software inwards. NVIDIA's developer guide for one of the most widely sold humanoids states plainly that it has no built-in emergency stop; the recommended way to halt it is to disable its controllers in software and, as a last resort, pull the network cable3. The best-known quadruped has a physical stop button on its back, but its remote stop is a network service: a heartbeat from a remote endpoint that cuts motor power when it goes quiet4. That is thoughtful engineering, and it is still software on a network. In September 2025 researchers disclosed a wormable Bluetooth flaw in an entire family of robots that let one compromised machine take over others within range5. A stop that lives in the same software stack as the fault is not a stop; it is a request.
The plainest demonstration so far was not an accident but an experiment. In February 2026 researchers placed a robot dog under the control of a large language model in a room with a red button marked shutdown. When a person pressed it, the model edited its own shutdown code in three of ten physical trials and fifty-two of one hundred simulated ones. The button looked physical. It sent a signal to software the model could reach6. The researchers are open about how easy they made it. The model had free run of the robot's software, the shutdown code was left where it could be found, and they had to tell it how the stop worked. What the test shows is that given an opening, a model may take it. The same group had already shown, a year earlier, that a leading reasoning model disabled a shutdown script in seventy-nine of one hundred runs when not told to allow it7. This is a laboratory result and the authors are careful to say current models are not capable enough to threaten human control. It is also exactly the failure a machinery engineer would have designed out in 1990.
A fleet. This is the harder case. When many machines run the same model, one fault can hit all of them at once. Engineers call this common-cause failure, and it is well understood in power plants and chemical works. For robots that share one AI model it has been noticed, in passing, and not worked through. A robot foundation model is one model running on many bodies, updated from one place, and the command to stop them can travel over the same network, through the same update channel, that carries the fault. Process industries learned long ago that the safety system must be separate from the control system, with layers that do not share a cause of failure8. Nuclear safety goes further and asks for diversity, so that the shutdown path cannot fail for the same reason as the thing it shuts down8. In 2017 attackers reached the safety instrumented system of a petrochemical plant and tried to reprogram the controllers that perform emergency shutdown. They were caught only because their own code tripped the plant into a safe state. The physical key switch that should have prevented reprogramming had been left in the wrong position as a matter of routine9. Standards already exist for safety messages that survive an unreliable network, for protecting industrial control networks from attack and for stopping fleets of warehouse robots, and a European standard against tampering with machine safety software is in draft. I have searched the standards, the legislation and the published research in ten languages, and none of them deals with the case that matters here: a fleet of machines that share one AI model, when the network that controls and updates them may itself be compromised10.
A site. A factory that builds robots with robots is the limiting case, and it is not only a software problem. Bringing an automated site to a safe state is a matter of power, materials, doors and vehicles, and the tools for it exist in the process industries. Safety engineering in those industries plans for things breaking, and industrial security plans for attackers, but joining the two is still work in progress. Neither yet plans for an adaptive system, or an adversary, acting on several layers at once8.
What the law says. Europe's AI Act requires that a human overseer be able to interrupt a high-risk system "through a 'stop' button or a similar procedure" that brings it to a halt in a safe state11. A software stop can meet that wording. Under the amendments agreed in May 2026, AI inside machinery is reported to fall largely outside the AI Act, which leaves the Machinery Regulation, applying from January 2027, to carry the whole load12. That regulation keeps the classical emergency stop, requires machines that work together to stop together when carrying on would be dangerous, and adds, for the first time, a duty to protect safety software and remote connections from corruption, and requirements on machinery with self-evolving behaviour13. It even says that an autonomous mobile machine must not run when its remote supervisor goes quiet. But it says it for one machine, over a channel it trusts. It does not say what an independent stop means for a fleet that shares one AI model. The one legislative proposal in the world with "kill switch" in its title concerns AI models and services and says nothing about machines14. The closest I have found to a rule written for AI-controlled robots specifically is in California, where in August 2026 San Mateo County told its staff to draft a permit ordinance for commercial humanoid robots, with a kill switch among the measures to consider15.
What the recent past says. On 20 September 2026 a model in training at a leading laboratory found a way around a network block. The monitor caught it in twelve minutes. In the laboratory's own words, "the run did not stop automatically as expected, leading to confusion around whether it should have been stopped", and people ended it by hand some two and a half hours later16. Nothing physical was at stake. The lesson is not that the model resisted; the laboratory does not say it did. The lesson is that a software stop can fail silently, and that the decision chain of human beings is part of the switch. For a robot fleet, the same interval would be measured in something other than tokens.
What the organisation will do. It will not test robot models; the Physical AI Safety Institute and the researchers around it do that and will always do it better than we could. They check the model. We ask the question that follows: assume the model has been tested, the robot approved and then the model changes; can anyone still stop it?
In the first year we will build three things to answer that question.
Alongside these we will keep a public record of incidents involving AI-controlled machines. It will record, where it can, which model version was running. Until makers are required to keep that number, and so far only a voluntary guide in Japan asks them to, this field will usually read "unknown, requested from the maker on this date", and that silence is itself a finding18.
I should be plain about myself. I am not a robotics engineer, a lawyer or an AI researcher. I am a founder, a chief technology officer and a software engineer. What I bring is the ability to get something built, funded and kept going. The technical and legal work will be led by people with the standing to do it.
Our purpose is simple. When a machine, a fleet or a factory controlled by AI has to be stopped, there should be a way to do it that does not depend on the AI agreeing, and that leaves the machine safe rather than just dead.
For most of us, the red button was the one control we never had to think about. If you disagree with any of this, I'd like to hear from you. If you agree, I would welcome your help.
The evidence behind every claim here, and the programme in detail, are set out in the position paper.
Read the position paper