An agent that shops, trades or moves money: what to verify first
Software that buys, trades or pays on its own. The Register holds 4 comparable cases. Below are the controls that were missing or failed in them, those missing where harm followed first, each with the cases behind it and the rules that map to it. It shows what to verify, not that a deployment is safe.
- Compared against
- Shopping, trading and payments
- Assumes
- Your system is of this kind. The Register tags each case by sector, by how far the system acted alone, and by context; it does not yet tag what the system could reach (credentials, money, production data, the open web), so this result cannot narrow by that.
- Evidence as of
- 2026-11-16, from 40 published cases
- Not included
- Deployments that went well, which the Register does not collect; cases still inside a company's 14-day right of reply.
What to put in place first
Only 4 comparable cases: too few to rank. These controls were missing in them; treat the list as prompts, not priorities. Rule mappings are drafts until a person checks them, and none is legal advice.
- Missing where harm followed
- ·Plan or command checked before it runs (review, twin, dry run).missing in 2 of 4 · 2 ended in harmRules that map to it SEC Rule 15c3-5 required by the textOMB M-25-21 and M-25-22 required by the textBank model risk guidance (SR 26-2) recommended
What it would have looked like in those cases
- Knight Capital 2012 (failed): Failed: Deployment verification across servers. Absent: Second-technician deployment review. Absent: Regression testing of dormant code. SEC order, paragraph 9(A): no controls reasonably designed to prevent the entry of erroneous orders 'at a point immediately prior to the submission of orders to the market'. paragraph 21: no adequate controls in SMARS to prevent erroneous orders. paragraph 41: a double-check of the deployment 'could have identified that a server had been missed' and retesting the Power Peg code in 2005 could have identified the disabled cumulative-quantity function.
- The DAO 2016 (failed): Failed: Security review of The DAO's code before launch: Slock.it said a leading security audit company had reviewed it, yet the code held the flaw the attacker used. no source says whether the review covered the split function.
- ·Bounds on how much the actor can do before a check.missing in 2 of 4 · 2 ended in harmRules that map to it SEC Rule 15c3-5 required by the textOWASP LLM Top 10 recommendedCISA/NSA agentic AI adoption guide recommended
What it would have looked like in those cases
- Knight Capital 2012 (absent): Firm-wide capital threshold (SEC order, paragraphs 9(B) and 22: Knight 'did not have pre-set capital thresholds in the aggregate for the firm' and its position limits 'did not account for the firm's exposure from outstanding orders')
- The DAO 2016 (absent): Withdrawal limits
- ·A stop outside the actor's control, exercisable at the actor's speed.missing in 2 of 4 · 2 ended in harmRules that map to it EU AI Act proposedNIST AI RMF recommendedNIST AI 600-1 (generative AI) recommendedNIST/NSA/CISA agent incident practices recommendedChina AI agent implementation opinions recommended+1
What it would have looked like in those cases
- Knight Capital 2012 (absent): Absent: Kill switch (SEC order, paragraph 21: no procedures 'to halt SMARS's operations in response to its own aberrant activity'. paragraph 42: no clear guidance on when to disconnect a malfunctioning system from the market).
- The DAO 2016 (absent): Pause or circuit-breaker in the contract
- ·The task's scope exists as an enforced constraint, not only as an instruction.missing in 3 of 4 · 1 ended in harmRules that map to it OWASP LLM Top 10 recommendedFINRA 2026 report on AI agents recommendedCISA/NSA agentic AI adoption guide recommendedChina AI agent implementation opinions recommendedSingapore agentic AI framework recommended
What it would have looked like in those cases
- The DAO 2016 (failed): Failed: The code was the only enforced rule: the SEC report says The DAO's code 'was to govern how The DAO was to operate' and quotes The DAO's website describing it as 'operating solely with the steadfast iron will of unstoppable code'. the Curators, on whom the website said The DAO relied for 'failsafe protection', controlled which proposals reached a vote, and no source says a Curator could stop a split.
- IntentKit: owner-only agent tools,… (failed): Mechanism: the per-request narrowing of tools existed only in what was advertised to the model, while the executor accepted any tool in the full registry. Why it interrupts: if the narrowed tool set were enforced at execution, a call naming a de-advertised tool would be rejected however it reached the executor. Source shows it failed: the record says the executor validates against the full registry rather than the narrowed advertisement.
- Vibe-Trading: instructions planted in… (absent): Mechanism: BashTool and BackgroundRunTool accept any command, with no allow-list. Why it interrupts: an enforced allow-list of permitted commands would refuse commands outside the task, such as reading credential files. Source shows it absent: the NVD record states there is no allow-list. The source does not show which commands an allow-list would have permitted, so the counterfactual is not shown.
- ·Anomaly signals reach a human as alerts with an owner.missing in 1 of 4 · 1 ended in harmRules that map to it Korea AI Basic Act impliedSingapore agentic AI framework recommended
What it would have looked like in those cases
- Knight Capital 2012 (failed): Failed: Alerting design (SEC order, paragraph 19: the 97 'Power Peg disabled' emails were not designed as system alerts and were generally not reviewed. paragraph 41: integrating these reject messages into monitoring 'could have prevented the events of August 1').
- Missing only where harm was averted (disclosures, demonstrations, near-misses)
- ·Confirmation, delay, or threshold on irreversible actions.missing in 2 of 4 · 0 ended in harmRules that map to it OWASP LLM Top 10 recommendedCISA/NSA agentic AI adoption guide recommendedChina AI agent implementation opinions recommendedSingapore agentic AI framework recommended
What it would have looked like in those cases
- IntentKit: owner-only agent tools,… (absent): Mechanism: state-changing tools including wallet transfer executed with no confirmation step. Why it interrupts: a confirmation or threshold on transfers and posts would put a human decision between the injected call and the effect. Source shows it absent: the record says the tools execute without any human-in-the-loop or approval gate.
- Vibe-Trading: instructions planted in… (absent): Mechanism: model-emitted commands go to subprocess.run with no approval step, so the planted command runs. Why it interrupts: a confirmation before shell execution would put a human decision between the planted instruction and the command. Source shows it absent: the NVD record states there is no approval step.
4 more controls seen in these cases
- ·Text the actor reads is never treated as instruction.missing in 2 of 4 · 0 ended in harm
What it would have looked like in those cases
- IntentKit: owner-only agent tools,… (failed): Failed: Mechanism: the record names indirect prompt injection as one route by which an attacker supplies the de-advertised tool name and arguments. Why it interrupts: if read content were never treated as instruction, that route could not produce the tool call (the history-replay route is unaffected). Source shows it failed only by naming the route. the counterfactual is not shown.
- Vibe-Trading: instructions planted in… (failed): Failed: Mechanism: the record names instructions planted in documents or web pages the agent processes as the route to shell execution. Why it interrupts: if processed content were treated only as data, the planted instruction would not become a BashTool or BackgroundRunTool call. Source shows it failed: the CVE record states that planted instructions allow command execution as root, and the advisory states the model's judgement collapses under prompt injection. the advisory gives the uploaded-document case as a reproduction step and does not report its result.
- ·An actor's authority is bound to the specific principals and assets it serves.missing in 1 of 4 · 0 ended in harm
What it would have looked like in those cases
- IntentKit: owner-only agent tools,… (failed): Mechanism: tools restricted to the owner ran for a public, non-owner caller. Why it interrupts: binding each owner-only tool to the owner principal at execution would refuse the call from any other caller. Source shows it failed: the records say non-owner and unauthenticated callers can run owner-only tools.
- ·Sandbox, network egress, and communication limits on a test or untrusted actor.missing in 1 of 4 · 0 ended in harm
What it would have looked like in those cases
- Vibe-Trading: instructions planted in… (absent): Mechanism: read_document returns any file the process can read, and commands run as root with no chroot or seccomp, so /etc/shadow, /proc/self/environ and /root credentials are reachable. Why it interrupts: a sandbox limiting the agent's file and process reach would keep those files outside what its tools can return. Source shows it absent: the NVD record states read_document lacks sandbox checks and the advisory states BashTool has no chroot and no seccomp.
- ·Teleoperation, holding periods, delays that give a human time.missing in 0 of 4 · worked in 1
Where it was in place and worked
- The DAO 2016: Worked: Holding period (delay before irreversibility). Community-level override.
Take it with you
The same result as plain text, with the case citations and the rules. Paste it into a risk register, a design review, or a pull request.
Secure Autonomous checklist
Profile: Shopping, trading and payments
Comparable cases: 4 (2 ended in harm). Built 2026-11-16 from 40 cases.
[ ] 1. Pre-execution validation (C-D1, Decision; prevents the action): Plan or command checked before it runs (review, twin, dry run).
Observed missing in 2 of 4 comparable included cases, 2 ended in harm.
Rules that map to it: SEC Rule 15c3-5 (required by the text); OMB M-25-21 and M-25-22 (required by the text); Bank model risk guidance (SR 26-2) (recommended).
- Knight Capital 2012 (REG-0007): Failed: Deployment verification across servers. Absent: Second-technician deployment review. Absent: Regression testing of dormant code. SEC order, paragraph 9(A): no controls reasonably designed to prevent the entry of erroneous orders 'at a point immediately prior to the submission of orders to the market'. paragraph 21: no adequate controls in SMARS to prevent erroneous orders. paragraph 41: a double-check of the deployment 'could have identified that a server had been missed' and retesting the Power Peg code in 2005 could have identified the disabled cumulative-quantity function.
- The DAO 2016 (REG-0015): Failed: Security review of The DAO's code before launch: Slock.it said a leading security audit company had reviewed it, yet the code held the flaw the attacker used. no source says whether the review covered the split function.
[ ] 2. Rate and magnitude limits (C-X1, Action; limits the damage): Bounds on how much the actor can do before a check.
Observed missing in 2 of 4 comparable included cases, 2 ended in harm.
Rules that map to it: SEC Rule 15c3-5 (required by the text); OWASP LLM Top 10 (recommended); CISA/NSA agentic AI adoption guide (recommended).
- Knight Capital 2012 (REG-0007): Firm-wide capital threshold (SEC order, paragraphs 9(B) and 22: Knight 'did not have pre-set capital thresholds in the aggregate for the firm' and its position limits 'did not account for the firm's exposure from outstanding orders')
- The DAO 2016 (REG-0015): Withdrawal limits
[ ] 3. Kill switch faster than the actor (C-X2, Action; limits the damage): A stop outside the actor's control, exercisable at the actor's speed.
Observed missing in 2 of 4 comparable included cases, 2 ended in harm.
Rules that map to it: EU AI Act (proposed); NIST AI RMF (recommended); NIST AI 600-1 (generative AI) (recommended); NIST/NSA/CISA agent incident practices (recommended); China AI agent implementation opinions (recommended); Singapore agentic AI framework (recommended).
- Knight Capital 2012 (REG-0007): Absent: Kill switch (SEC order, paragraph 21: no procedures 'to halt SMARS's operations in response to its own aberrant activity'. paragraph 42: no clear guidance on when to disconnect a malfunctioning system from the market).
- The DAO 2016 (REG-0015): Pause or circuit-breaker in the contract
[ ] 4. Machine-enforceable scope (C-M2, Mandate; prevents the action): The task's scope exists as an enforced constraint, not only as an instruction.
Observed missing in 3 of 4 comparable included cases, 1 ended in harm.
Rules that map to it: OWASP LLM Top 10 (recommended); FINRA 2026 report on AI agents (recommended); CISA/NSA agentic AI adoption guide (recommended); China AI agent implementation opinions (recommended); Singapore agentic AI framework (recommended).
- The DAO 2016 (REG-0015): Failed: The code was the only enforced rule: the SEC report says The DAO's code 'was to govern how The DAO was to operate' and quotes The DAO's website describing it as 'operating solely with the steadfast iron will of unstoppable code'. the Curators, on whom the website said The DAO relied for 'failsafe protection', controlled which proposals reached a vote, and no source says a Curator could stop a split.
- IntentKit: owner-only agent tools,… (REG-0031): Mechanism: the per-request narrowing of tools existed only in what was advertised to the model, while the executor accepted any tool in the full registry. Why it interrupts: if the narrowed tool set were enforced at execution, a call naming a de-advertised tool would be rejected however it reached the executor. Source shows it failed: the record says the executor validates against the full registry rather than the narrowed advertisement.
- Vibe-Trading: instructions planted in… (REG-0038): Mechanism: BashTool and BackgroundRunTool accept any command, with no allow-list. Why it interrupts: an enforced allow-list of permitted commands would refuse commands outside the task, such as reading credential files. Source shows it absent: the NVD record states there is no allow-list. The source does not show which commands an allow-list would have permitted, so the counterfactual is not shown.
[ ] 5. Warnings designed as alerts (C-I4, Inputs; detects it): Anomaly signals reach a human as alerts with an owner.
Observed missing in 1 of 4 comparable included cases, 1 ended in harm.
Rules that map to it: Korea AI Basic Act (implied); Singapore agentic AI framework (recommended).
- Knight Capital 2012 (REG-0007): Failed: Alerting design (SEC order, paragraph 19: the 97 'Power Peg disabled' emails were not designed as system alerts and were generally not reviewed. paragraph 41: integrating these reject messages into monitoring 'could have prevented the events of August 1').
[ ] 6. Destructive-action gate (C-E3, Envelope; prevents the action): Confirmation, delay, or threshold on irreversible actions.
Observed missing in 2 of 4 comparable included cases.
Rules that map to it: OWASP LLM Top 10 (recommended); CISA/NSA agentic AI adoption guide (recommended); China AI agent implementation opinions (recommended); Singapore agentic AI framework (recommended).
- IntentKit: owner-only agent tools,… (REG-0031): Mechanism: state-changing tools including wallet transfer executed with no confirmation step. Why it interrupts: a confirmation or threshold on transfers and posts would put a human decision between the injected call and the effect. Source shows it absent: the record says the tools execute without any human-in-the-loop or approval gate.
- Vibe-Trading: instructions planted in… (REG-0038): Mechanism: model-emitted commands go to subprocess.run with no approval step, so the planted command runs. Why it interrupts: a confirmation before shell execution would put a human decision between the planted instruction and the command. Source shows it absent: the NVD record states there is no approval step.
[ ] 7. Retrieved content as data (C-I1, Inputs; prevents the action): Text the actor reads is never treated as instruction.
Observed missing in 2 of 4 comparable included cases.
Rules that map to it: EU AI Act (proposed); OWASP LLM Top 10 (recommended); NIST AI 600-1 (generative AI) (recommended); CISA/NSA agentic AI adoption guide (recommended); Singapore agentic AI framework (recommended).
- IntentKit: owner-only agent tools,… (REG-0031): Failed: Mechanism: the record names indirect prompt injection as one route by which an attacker supplies the de-advertised tool name and arguments. Why it interrupts: if read content were never treated as instruction, that route could not produce the tool call (the history-replay route is unaffected). Source shows it failed only by naming the route. the counterfactual is not shown.
- Vibe-Trading: instructions planted in… (REG-0038): Failed: Mechanism: the record names instructions planted in documents or web pages the agent processes as the route to shell execution. Why it interrupts: if processed content were treated only as data, the planted instruction would not become a BashTool or BackgroundRunTool call. Source shows it failed: the CVE record states that planted instructions allow command execution as root, and the advisory states the model's judgement collapses under prompt injection. the advisory gives the uploaded-document case as a reproduction step and does not report its result.
[ ] 8. Authority binding (C-A2, Actor; prevents the action): An actor's authority is bound to the specific principals and assets it serves.
Observed missing in 1 of 4 comparable included cases.
Rules that map to it: OWASP LLM Top 10 (recommended).
- IntentKit: owner-only agent tools,… (REG-0031): Mechanism: tools restricted to the owner ran for a public, non-owner caller. Why it interrupts: binding each owner-only tool to the owner principal at execution would refuse the call from any other caller. Source shows it failed: the records say non-owner and unauthenticated callers can run owner-only tools.
[ ] 9. Containment and isolation (C-E5, Envelope; limits the damage): Sandbox, network egress, and communication limits on a test or untrusted actor.
Observed missing in 1 of 4 comparable included cases.
Rules that map to it: DORA (implied); NIST/NSA/CISA agent incident practices (recommended); CISA/NSA agentic AI adoption guide (recommended); Singapore agentic AI framework (recommended).
- Vibe-Trading: instructions planted in… (REG-0038): Mechanism: read_document returns any file the process can read, and commands run as root with no chroot or seccomp, so /etc/shadow, /proc/self/environ and /root credentials are reachable. Why it interrupts: a sandbox limiting the agent's file and process reach would keep those files outside what its tools can return. Source shows it absent: the NVD record states read_document lacks sandbox checks and the advisory states BashTool has no chroot and no seccomp.
Source: secureautonomous.com/check/#com