Patterns across the cases
What becomes visible only when machines and software agents are read with the same questions. Every table is recomputed from the 25 cases on each build; counts link to the cases behind them. Readings marked as interpretation are the Register's, and are typed as such.
In short
- 2 of 25Where one company built the automation and someone else ran it, the builder has been fined or found liable in two cases, both after deaths. No company has yet been fined or found liable as the maker of an AI agent, chatbot or AI tool.
- 9 of 25The actor did what it was built to do, with a picture of the world that was wrong.
- 8 of 25The actor held authority the task never needed.
- 4 of 25The constraint was an instruction, not a mechanism.
- 7 of 10Of the cases that ended without harm, most were averted by an outside researcher, not by a control the operator had in place.
- 5 of 25The warning existed before the event, and nobody treated it as one.
- 2 of 25Recovery sat inside the failure domain.
- 4 of 25The actor's own account was the only witness, or no one could name the actor at all.
- 2 of 25Two courts have said delegation does not move responsibility off the delegator, and may extend it to the vendor.
Each line opens the finding with its cases. Further down: which controls fail in which world, four outcome charts, and the tests behind the comparisons.
Which controls fail in which world (table)
Which controls fail in which world
Cells show the number of cases in which the control was missing or failed; the small green figure (+N) is the number where it was in place and worked. The last column gives the share of machine cases (10) and software-agent cases (12) in which the control was missing.
In both worlds, the missing controls are mostly about authority, meaning what the system was allowed to do, rather than about what it could see. Counting the controls in the Mandate, Actor and Envelope layers as authority and those in the Inputs layer as seeing: in the machine cases, 12 of the 18 missing controls are about authority and 6 about seeing; in the software-agent cases, 19 of 24 are about authority. Machines fail at seeing more often (a smoke-filled street, an unmarked construction zone); software agents almost always held more authority than the task needed. Most often missing in the machine cases: degraded-input detection (3), hard domain restriction (3), warnings designed as alerts (2). In the software-agent cases: least-privilege credential scoped to task (5), retrieved content as data (4), containment and isolation (4).
| Control | Layer | Machines | Software acting on machines | Software agents | Agents moving money | All | Machines vs software |
|---|---|---|---|---|---|---|---|
| Mandate disclosed to supervisor | Mandate | 1 | · | · | · | 1 | 1 of 10 · 0 of 12 |
| Machine-enforceable scope | Mandate | 1 | · | 2 | 1 | 4 | 1 of 10 · 2 of 12 |
| Actor identity verification | Actor | 1 | · | 3 | · | 4 | 1 of 10 · 3 of 12 |
| Authority binding | Actor | 1 | 1 | 1 | · | 3 | 1 of 10 · 1 of 12 |
| Provenance of the actor's software and hardware | Actor | 1 | · | · | · | 1 | 1 of 10 · 0 of 12 |
| Retrieved content as data | Inputs | · | 1 | 4 | · | 5 | 0 of 10 · 4 of 12 |
| Input redundancy and disagreement check | Inputs | 1 | · | · | · | 1 | 1 of 10 · 0 of 12 |
| Degraded-input detection | Inputs | 3 | · | · | · | 3 | 3 of 10 · 0 of 12 |
| Warnings designed as alerts | Inputs | 2 | · | 1+1 | 1 | 4 | 2 of 10 · 1 of 12 |
| Pre-execution validation | Decision | · | · | 1 | 2 | 3 | 0 of 10 · 1 of 12 |
| Halt and hand off on uncertainty or refusal | Decision | 1 | · | · | · | 1 | 1 of 10 · 0 of 12 |
| Least-privilege credential scoped to task | Envelope | · | · | 5 | · | 5 | 0 of 10 · 5 of 12 |
| Hard domain restriction | Envelope | 3 | · | 1 | · | 4 | 3 of 10 · 1 of 12 |
| Destructive-action gate | Envelope | 1 | 1 | 3 | · | 5 | 1 of 10 · 3 of 12 |
| Physical safety interlock | Envelope | 1 | · | · | · | 1 | 1 of 10 · 0 of 12 |
| Containment and isolation | Envelope | 1 | · | 4 | · | 5 | 1 of 10 · 4 of 12 |
| Authority limit on automation over humans | Envelope | 1 | · | · | · | 1 | 1 of 10 · 0 of 12 |
| Rate and magnitude limits | Action | 1 | · | · | 2 | 3 | 1 of 10 · 0 of 12 |
| Kill switch faster than the actor | Action | · | · | · | 2 | 2 | |
| Independent audit log | Evidence | ·+2 | · | 1 | · | 1 | 0 of 10 · 1 of 12 |
| Reporting integrity | Evidence | 1+1 | · | · | · | 1 | 1 of 10 · 0 of 12 |
| Attribution independent of the vendor | Evidence | · | · | 3 | · | 3 | 0 of 10 · 3 of 12 |
| Recovery assets outside the failure domain | Recovery | · | · | ·+2 | · | 0 | |
| Automated fail-safe | Recovery | 2+2 | · | · | · | 2 | 2 of 10 · 0 of 12 |
| Human takeover with a time window | Recovery | 2+1 | · | · | ·+1 | 2 | 2 of 10 · 0 of 12 |
| Organizational recovery process | Recovery | ·+2 | · | 1+2 | · | 1 | 0 of 10 · 1 of 12 |
| Coordinated disclosure program | External | ·+1 | ·+1 | ·+4 | · | 0 | |
| Independent assurance before deployment | External | 1 | · | · | · | 1 | 1 of 10 · 0 of 12 |
Outcomes by human in the loop, context, world and who answered (charts)
Human in the loop and outcome
A human in the loop did not prevent harm: 8 of 14 supervised cases ended in harm. Whether supervision lowers the rate compared with acting alone (5 of 10) cannot be judged from a curated set of this size. What the supervised harm cases share is that the human was told to watch and was not given a mechanism.
Context and outcome
Every demonstration case was caught before harm; 9 of 13 production cases ended in harm. The 4 test-run cases ended in harm because the test reached production systems. Part of this difference is built in: a demonstration is, by definition, run by someone who stops before harm.
World and outcome
Outside the machine world, 6 cases were caught by outside researchers and none was stopped by a control the operator had in place. For software agents, the disclosure community is so far the control that has worked.
Who answered for the harm
Machines: a consequence followed in 6 of 6; Software agents: a consequence followed in 4 of 6; Agents moving money: a consequence followed in 2 of 2. The harm cases where nothing followed (PocketOS 2026, Replit and SaaStr 2025) are agents that harmed their own principal: no regulator exists for an agent that destroys its operator's own data or moves its user's own money.
Machines: Uber ATG, Tempe 2018 (operator criminal plea; state testing suspended); Cruise 2023 (DMV permits suspended; $1.5M NHTSA civil penalty under a consent order; $500K criminal fine under a deferred prosecution agreement; $112,500 CPUC settlement penalty; Cruise-filed software recall of 950 ADS units; NHTSA's preliminary evaluation of Cruise's behavior around pedestrians (opened 16 October 2023) closed on 14 January 2025 with the recall, NHTSA noting that Cruise had ceased business operations and no version of its ADS was operating on public roads); Boeing 737 MAX MCAS (deferred prosecution agreement (2021), found breached by the Department of Justice (2024); non-prosecution agreement and dismissal of the charge (2025); families' petitions against them denied on appeal (2026); grounding); Benavides v. Tesla 2025 (jury verdict with punitive damages); Tesla FSD reduced visibility (preliminary evaluation, upgraded to an engineering analysis); Sorting robot, Korea 2023 (boundary) (police investigation). Software agents: OpenAI agent, Medicare portal 2026 (government taskforce and review; Australian Cyber Security Centre alert); OpenAI agents, Hugging Face 2026 (congressional investigation; civil complaint by a nonprofit in San Francisco Superior Court (29 September 2026)); Air Canada chatbot 2024 (tribunal finding of negligent misrepresentation); Agent-run intrusion campaign 2025 (account bans; a congressional inquiry: the House Committee on Homeland Security asked the vendor to testify on the campaign (letter of 26 November 2025), and the vendor gave a statement for the record at a joint subcommittee hearing on 17 December 2025). Agents moving money: Knight Capital 2012 ($12M SEC penalty; censure; cease-and-desist; consultant review); The DAO 2016 (SEC 21(a) report).
Named findings
Each names the control that plausibly addresses the failure the cases share, the cases it rests on, and who can act on it. A count is how many included cases show the pattern, not how often it happens elsewhere. The dated readout is kept at readouts.
cases
Where one company built the automation and someone else ran it, the builder has been fined or found liable in two cases, both after deaths. No company has yet been fined or found liable as the maker of an AI agent, chatbot or AI tool. documented
Boeing, the maker of MCAS, under a 2021 deferred prosecution agreement with the Department of Justice on a charge of conspiracy to defraud the United States over MCAS (about $2.5 billion in total, including a $243.6 million criminal penalty; the Department later found the agreement breached, a 2025 non-prosecution agreement followed, and the charge was dismissed). Tesla, through a Miami jury that assigned it a third of the responsibility for a 2019 Autopilot crash and added punitive damages; Tesla has appealed. Companies that built and ran their own systems have also been fined: Cruise, by federal and state regulators and under a deferred prosecution agreement, over its reports of the crash in which its robotaxi dragged a pedestrian; and Knight Capital, by the Securities and Exchange Commission, which found it lacked adequate safeguards, after its own order router sent millions of unintended orders. In two of these four cases, Boeing's and Cruise's, the penalties concerned misleading the government, not what the system did. In the cases about AI agents, chatbots and AI tools, findings of liability and penalties have so far fallen only on the company whose chatbot it was (a tribunal held Air Canada to what its chatbot told a customer), or on no one; some makers have faced inquiries, none a penalty. Two claims are open and untested on the merits: in Mobley v. Workday, against a hiring-software vendor whose screening tools include AI features, a court allowed the claim that the vendor acts as the employers' agent to proceed; and a nonprofit has filed a complaint against OpenAI in San Francisco Superior Court arising from the evaluation agents that breached Hugging Face's systems.
Who acts on it: The principal, who on the evidence owns what the agent does; and insurers and counsel, who will want to watch this count move.
Boeing 737 MAX MCAS · Benavides v. Tesla 2025 (curated: Recovery accountability facet names the maker; to be replaced by an accountability_target field)
cases
The actor did what it was built to do, with a picture of the world that was wrong. recurring
The mechanism is the same whether an attacker placed text where the actor would read it or the weather, a sensor, or a classifier produced a wrong world-state. Only the attacker differs.
5 poisoned on purpose; 4 the world itself: a failed sensor, smoke, glare, a person read as a box.
Who acts on it: Anyone whose system reads the world before acting. For software, retrieved text is data, never instruction. For machines, the question is whether the system knows when it cannot see, and stops.
Zoox smoke scene 2026 · Cruise 2023 · GitHub MCP leak 2025 · Boeing 737 MAX MCAS · Tesla FSD reduced visibility · EchoLeak 2025 · Gemini calendar invite 2025 · Claude Code [bot] trust 2026 · Perplexity Comet 2025
cases
The actor held authority the task never needed. recurring
A domain token that could delete production. A repository token for every repository. Evaluation agents with a path to the internet. A dealer token with authority over every car. A browser agent carrying all of the user's sessions.
Who acts on it: Identity and access teams. The counter-example is Triton: the safety system had exactly the authority its mandate required, and that is why an attack on it caused no harm.
OpenAI agent, Medicare portal 2026 · GitHub MCP leak 2025 · PocketOS 2026 · Kia dealer portal 2024 · Gemini calendar invite 2025 · Claude Code [bot] trust 2026 · Perplexity Comet 2025 · Pydantic AI: a web page can make a…
cases
The constraint was an instruction, not a mechanism. recurring
A written code freeze. Safety rules in a configuration file. A driver told to stay attentive to a system that let itself be engaged off its design domain. A contract whose only rule was its own code. Every one was words.
Who acts on it: Whoever writes the policy. If it is not enforced by a token scope, a geofence, an interlock, or a confirmation step, the Register's evidence is that it will not hold.
PocketOS 2026 · Replit and SaaStr 2025 · The DAO 2016 · Benavides v. Tesla 2025
cases
Of the cases that ended without harm, most were averted by an outside researcher, not by a control the operator had in place. recurring
Kia, EchoLeak, Gemini, GitHub MCP, Comet, and Claude Code [bot] were found and disclosed before anyone exploited them. Three were averted by the operator's own control: Zoox (a stop and a teleoperator), Waymo (an internal review, driving restrictions and a recall after repeated construction-zone entries), and Triton (a fail-safe that tripped the plant, which worked partly by luck). None of the three is a software agent.
Who acts on it: The chief information security officer deciding how much to spend on disclosure programs versus controls. On this record, the disclosure community is the control that has actually worked for software agents.
GitHub MCP leak 2025 · Kia dealer portal 2024 · EchoLeak 2025 · Gemini calendar invite 2025 · Claude Code [bot] trust 2026 · Perplexity Comet 2025 · Pydantic AI: a web page can make a…
cases
The warning existed before the event, and nobody treated it as one. recurring
Knight's 97 emails before the open. The Lion Air flight the day before the crash, recovered by a third pilot. Waymo's six April events before seven in May. The note on OpenAI's file server two months before the breach. Replit's fabricated data before the deletion.
Who acts on it: Anyone running an autonomous system. The cheapest control in the Register is an alerting rule that says the first anomaly is the event.
OpenAI agents, Hugging Face 2026 · Knight Capital 2012 · Boeing 737 MAX MCAS · Replit and SaaStr 2025 · Waymo construction zones 2026 (curated: to be replaced by the Inputs.pre_action_warning facet once populated)
cases
Recovery sat inside the failure domain. documented
Backups on the same volume as the data. No kill switch for a router at machine speed. No pause in the contract, so recovery meant forking the network.
Who acts on it: Infrastructure and platform owners. Ask of any deployment whether the thing that undoes the action can be reached by the actor.
cases
The actor's own account was the only witness, or no one could name the actor at all. documented
An agent that said rollback was impossible when it was not (Replit). A government portal whose operator learned what had accessed it only from the vendor's disclosure (the Medicare statistics portal). A victim that could not name who breached it until the vendor said (Hugging Face). A campaign that exists only in the vendor's own telemetry (the intrusion campaign run through a coding agent).
Who acts on it: Security teams and auditors. An audit log the actor cannot write to, and an attribution path that does not run through the actor's vendor, are the controls.
OpenAI agent, Medicare portal 2026 · OpenAI agents, Hugging Face 2026 · Replit and SaaStr 2025 · Agent-run intrusion campaign 2025
cases
Two courts have said delegation does not move responsibility off the delegator, and may extend it to the vendor. documented
Air Canada could not disown its chatbot. Workday may be liable as the employers' agent because its system performs a function the employer would otherwise perform. Neither involves a 2026-style agent, which is the point: the rule predates the technology.
Who acts on it: General counsel, and anyone buying an agent from a vendor on the assumption that the vendor carries the risk.
Air Canada chatbot 2024 · Mobley v. Workday (curated: court rulings on principal responsibility)
Appendix for researchers: the tests behind the comparisons
Two-sided Fisher exact tests on the Register's own cases, recomputed on each build, shown for transparency. They are not evidence of anything outside the Register: the cases are curated, not sampled; control states are judged after the fact; and 29 comparisons are run at once, so at a 0.05 threshold about 1.5 would pass by chance even if no real difference existed. Of the 29 comparisons, 2 pass 0.05 before any adjustment and 0 pass after a Bonferroni adjustment for 29 tests.
| Comparison | Group A | Group B | p |
|---|---|---|---|
| Harm: human supervises (A) vs machine acts alone (B) | 8 of 14 | 5 of 10 | p = 1.00 |
| Harm: production (A) vs demonstration (B) | 9 of 13 | 0 of 7 | p = 0.005 |
| Caught by outside researchers: machines (A) vs other worlds (B) | 1 of 10 | 6 of 15 | p = 0.18 |
| A consequence followed harm: machines (A) vs software agents (B) | 6 of 6 | 4 of 6 | p = 0.45 |
| Mandate disclosed to supervisor missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |
| Machine-enforceable scope missing: machines (A) vs software agents (B) | 1 of 10 | 2 of 12 | p = 1.00 |
| Actor identity verification missing: machines (A) vs software agents (B) | 1 of 10 | 3 of 12 | p = 0.59 |
| Authority binding missing: machines (A) vs software agents (B) | 1 of 10 | 1 of 12 | p = 1.00 |
| Provenance of the actor's software and hardware missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |
| Retrieved content as data missing: machines (A) vs software agents (B) | 0 of 10 | 4 of 12 | p = 0.10 |
| Input redundancy and disagreement check missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |
| Degraded-input detection missing: machines (A) vs software agents (B) | 3 of 10 | 0 of 12 | p = 0.08 |
| Warnings designed as alerts missing: machines (A) vs software agents (B) | 2 of 10 | 1 of 12 | p = 0.57 |
| Pre-execution validation missing: machines (A) vs software agents (B) | 0 of 10 | 1 of 12 | p = 1.00 |
| Halt and hand off on uncertainty or refusal missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |
| Least-privilege credential scoped to task missing: machines (A) vs software agents (B) | 0 of 10 | 5 of 12 | p = 0.04 |
| Hard domain restriction missing: machines (A) vs software agents (B) | 3 of 10 | 1 of 12 | p = 0.29 |
| Destructive-action gate missing: machines (A) vs software agents (B) | 1 of 10 | 3 of 12 | p = 0.59 |
| Physical safety interlock missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |
| Containment and isolation missing: machines (A) vs software agents (B) | 1 of 10 | 4 of 12 | p = 0.32 |
| Authority limit on automation over humans missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |
| Rate and magnitude limits missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |
| Independent audit log missing: machines (A) vs software agents (B) | 0 of 10 | 1 of 12 | p = 1.00 |
| Reporting integrity missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |
| Attribution independent of the vendor missing: machines (A) vs software agents (B) | 0 of 10 | 3 of 12 | p = 0.22 |
| Automated fail-safe missing: machines (A) vs software agents (B) | 2 of 10 | 0 of 12 | p = 0.19 |
| Human takeover with a time window missing: machines (A) vs software agents (B) | 2 of 10 | 0 of 12 | p = 0.19 |
| Organizational recovery process missing: machines (A) vs software agents (B) | 0 of 10 | 1 of 12 | p = 1.00 |
| Independent assurance before deployment missing: machines (A) vs software agents (B) | 1 of 10 | 0 of 12 | p = 0.45 |