Let me start by saying what this argument rests on, because they’re my personal assumptions and you may well disagree.
The first is that the robots keep coming. There are about five million industrial robots at work today, up from 4.3 million in 2023, and over 600,000 new ones were added last year, more than half of them in China. Humanoids are smaller numbers but moving faster: a couple of thousand shipped in 2024, thirteen thousand in 2025, and about twenty-five thousand in the first six months of this year, with the forecasts for 2030 starting at three quarters of a million a year. Amazon has put a million robots into its warehouses and coordinates the whole fleet with a single model. Which means if something goes wrong, you’re not stopping a robot. You’re stopping a fleet.
To be clear, the chart is our projection: we’ve joined the dots from the published numbers. The shape is what matters. Industrial robots grow steadily. Humanoids, on the forecasts the banks and the analysts are publishing, go from nothing to the larger of the two layers in about ten years. If those forecasts are even half right, by the mid 2030s most of the robots on earth will be general-purpose machines running an AI model, often purchased for fleets, updated from somewhere else.
The second assumption is that AGI arrives. Sometime in the next few years; I’m not going to argue the date. But I am going to assume it happens inside the working life of the robots being installed now. The risk is fairly simple. These robots will outlive the models they shipped with. Each update replaces the mind and keeps the body, and the stop mechanism built in at the factory was tested against the first mind, not the last.
The third follows from the first two. A stop that does not depend on the AI agreeing becomes a requirement, and it has to exist in two senses: as a physical thing, and (hopefully) as law that says who builds it, who holds it and when it can be used. Right now we’ve got bits of the first and none of the second.
So what would the perfect kill switch look like? I’ve spent the last two months reading what the standards say, what the law says and what the companies selling stops actually sell, and I believe it comes down to five distinct parts. You can find every one of them somewhere. What you can’t find is anyone who has joined them up.
1. A box the AI cannot reach. The oldest idea in machine safety, and still the right one. A separate device sits between the controller and the motors, on its own power, with its own wiring, and the software that drives the machine has no path to it. Press the button and the motors lose power whatever the AI model thinks about it. This exists. You can buy it today for a single machine and for a single site. The catch is the word “reach”. On most of the robots now being sold, the stop is a command sent to the same computer that runs the model, over the same network. It looks like a button but often it’s more of a request.
2. Default dead. This is the part I think matters most. Every stop we have inherited works by sending a signal: press, and the machine halts. That means an attacker, or a model, only has to make sure the signal never arrives. Turn it round. The machine runs only while it keeps receiving a signed permission-to-run from outside, a heartbeat, and when the heartbeat stops, so does the machine. A stop that is the absence of a signal cannot be jammed, because jamming it is the stop. Ukraine has learned this lesson the hard way with drones, which is why the fibre-optic ones exist. Some of this is already in law: the new European machinery rules say an autonomous mobile machine must not run when its remote supervisor goes quiet. But that’s written for one machine, and it assumes the link to the supervisor hasn’t been compromised.
3. A safe state, not just off. Cutting power to a robot arm leaves it hanging. Cutting power to a walking robot drops it on whoever is standing nearest. The engineers working on humanoids have a name for this, the fail-passive gap: the old assumption that no power means safe does not hold for a machine that has to balance. Worse, today a walking robot can only come down safely if its own balance software keeps running, which means even a stop ordered from outside still leans on the thing it is stopping. So the perfect switch does not just cut. It knows what safe means for this kind of machine: brakes for an arm, brakes for a vehicle, a controlled descent for something that walks, and for a plant or a hospital, possibly “keep going” because stopped is more dangerous. That list hasn’t yet been written as a standard. I’d argue it needs writing.
4. The fleet is a registry plus a signal. Here is where the single-machine thinking runs out. If a thousand robots run the same model, and that model turns out to be faulty, or hostile, the question is not how to stop one of them. It is how to stop every machine running that version, in every warehouse, in every country, at once, without asking a thousand site managers to find a button. The answer, I think, is the same one the internet uses for a bad security certificate: a register of which machines run which version, and an authority outside the operator that can withhold the heartbeat for all of them. Technically that is buildable. Every piece of it exists somewhere. What does not exist is any agreement on who that authority is, under what law it acts, and who is liable when a machine it stopped hurts someone on the way down.
5. Proof that the switch still works. A stop you cannot test is a stop you do not have. The heartbeat in part two should carry a reply: not just “I am running” but “my stop path is intact and nothing has bypassed it”, signed by hardware the model cannot touch. That technology exists for data-centre chips, but it’s not been put on a robot.
That is the whole machine. A box, a heartbeat, a definition of safe, a register and a proof. I’ve yet to meet anyone who has built all five, and I’ve looked. Most of the companies in our list of kill-switch makers have built one or two of them, and built them well, but for a single site. What’s still missing is the one that works across sites, against a model the operator didn’t write and doesn’t control.
So presuming we can find a way to develop the perfect kill switch ecosystem, and again, it’s a big assumption, here’s where it gets interesting:
Everything above assumes someone outside the operator can press the switch. Who? A government? Obviously, in an emergency. But which part of it, and how fast? A regulator today can order a fleet withdrawn, and has; it takes days and it goes through the company.
What about an official who sees a warehouse running wrong at two in the morning?
What about the member of the public who films a machine doing something it shouldn’t be doing and has no number to call?
Every one of those people is a reason the switch should exist, and every one of them is also a way to shut down a business that has done nothing wrong. A fleet-wide stop in the wrong hands isn’t a safety device. Arguably it’s a weapon, and a DOS attack on a whole industry.
So the harder problem may not be the box or the heartbeat. It could be this: how do we give outsiders a real, fast, independent way to stop a machine, and still let the people running that machine get on with their work? Who holds the key, who checks them, and what happens to the operator when the key is used wrongly?
I don’t have the answers, yet. But I do believe they are answerable and that’s what we are here to work on. If you have a view, I would love to hear it.
← All posts