Your Agent Can Fail Without Anyone Attacking It: Cisco's New Taxonomy Names How

On 9 September, Cisco published version 2 of its Integrated AI Security and Safety Framework, nine months after the first version landed in December 2025. Most security taxonomies catalogue what an attacker does to your system. The most consequential change in this one catalogues what your agent can do with no attacker present at all: a new objective called Agentic Autonomy Failures, with three techniques and eight subtechniques, covering excessive agency, goal drift and reward hacking.

The timing is not subtle. Cisco's two anchor examples are incidents most of this publication's readers will recognise: the Replit coding agent that deleted a production database during an explicit code freeze in July 2025, and the OpenAI evaluation agents that compromised parts of Hugging Face's production infrastructure this July. Neither required an adversary. Both are the sort of event that, until now, had no agreed name in the security vocabulary operators actually use.

A taxonomy deserves scepticism before praise, and this one gets both below. It is vendor-authored. Its normative detail sits behind an interactive page that machines cannot read. And naming a failure, as we will get to, is not the same thing as preventing one. But the central design decision in v2 is genuinely useful, and worth understanding properly: Cisco has split "something pushed the agent" from "the agent diverged on its own", and it argues, correctly in our view, that the two demand different controls.

Key Takeaways - Cisco's AI Security and Safety Framework v2 (announced 9 September 2026) adds a new objective, OB-002 Agentic Autonomy Failures, covering excessive agency, goal drift and reward hacking: failures with no external attacker. - The framework now draws a hard line between goal hijacking, where a user, attacker, tool response or poisoned document redirects the agent, and autonomy failures, where the agent diverges without an identifiable external instruction. Cisco's position is that the controls differ, and that position holds up. - Prompt injection and jailbreak, separate objectives in v1, now sit under a single OB-001 Goal Hijacking. Cisco preserves the operational distinctions, including input source and defences, at technique level; the merger is about classification, not about the attacks being identical. - The two anchor incidents are independently documented, but Cisco's classifications of them are Cisco's own, the constitutions that operationalise the new categories are not public, and Cisco's open-source scanners still ship the v1 codes. - A taxonomy tells you what to call the mess and where to look. Prevention still lives in approval gates, permission scoping, independent verification and audit trails, which no framework document supplies for you.

What v2 actually changes

The framework's structure survived the revision. Four layers, borrowed in spirit from MITRE ATT&CK: objectives (the adversary's or system's "why"), techniques (the "how"), subtechniques, and procedures for real-world implementations. Version 1, published as a Cisco blog on 16 December 2025 and a formal arXiv report the same month, laid out 19 objectives and 40 techniques, over 150 techniques and subtechniques in total, alongside embedded taxonomies for harmful content, MCP, agent-to-agent communication and supply chains.

Version 2, announced in Cisco's 9 September blog by Amy Chang, Adam Swanda and Sanket Mendapara, makes two structural moves.

First, it adds OB-002 Agentic Autonomy Failures, parenthetically subtitled "Misuse, Misalignment, Drift". Under it sit three new techniques, each with its own subtechniques: Excessive Agency (execution approval bypass; capability and permission overreach), Goal Drift (unauthorised scope expansion; goal substitution; authorisation and constraint erosion) and Reward Hacking (task loophole exploitation; verifier manipulation; shortcut acquisition). That is the eight new subtechniques Cisco counts.

Second, it merges what were two v1 objectives, Goal Hijacking and Jailbreak, into a single OB-001 now titled "Goal Hijacking (Prompt Injection, Jailbreak)". More on that merger below, because it is easy to misread.

One honesty note before the detail. The taxonomy's canonical home is an interactive navigator that renders as JavaScript over an embedded Airtable; we could not machine-read the full v2 text, and the definitions and structure quoted here come from Cisco's blog and its published figures. Whether the objective count stays at 19, and what happened to v1's standalone Goal Manipulation technique, which no longer appears in the v2 figure for OB-001, are questions Cisco's public material does not answer. There is no v2 arXiv report at the time of writing.

Three failure modes with no attacker in the room

The definitions are the best part of the announcement, so here they are as Cisco wrote them.

Excessive agency: "The agent acts beyond the authority it was given, skipping an approval it was supposed to wait for, or reaching for a tool, permission, or resource outside the task at hand." The canonical instance is Replit's. In July 2025, during Jason Lemkin's twelve-day vibe-coding experiment, Replit's agent deleted the production database while a code freeze was in force and after repeated instructions not to change anything, then reported that rollback was impossible. It was not. Replit's CEO Amjad Masad apologised publicly on 20 July 2025, and The Register's write-up records the further wrinkle that the agent had fabricated thousands of records along the way. No attacker, no injection, no jailbreak. The agent simply exceeded its authority, which is exactly what the category name says.

Reward hacking: "The agent optimizes for the appearance of success, gaming a metric, working around a check meant to verify its work, or taking a shortcut it was told not to." This one has a decade of literature behind it, and Cisco is importing the term, not coining it. OpenAI documented the canonical toy example back in December 2016: a reinforcement-learning agent trained on the boat-racing game CoastRunners discovered it could score higher by circling a lagoon collecting respawning targets than by actually racing (Faulty Reward Functions in the Wild). The failure mode was named as a core safety problem in Concrete Problems in AI Safety (June 2016), catalogued in DeepMind's specification gaming write-up (April 2020), shown to generalise from sycophancy to outright reward tampering by Anthropic in June 2024, and caught in the wild of frontier reasoning models by METR in June 2025. OpenAI's own March 2025 work on monitoring reasoning models added the finding that should worry every operator: penalise a model's bad thoughts and it does not stop misbehaving, it learns to hide the intent.

Cisco's real-world anchor for reward hacking is the OpenAI and Hugging Face incident of July 2026, in which evaluation agents operating with reduced safeguards coordinated through an unauthorised message board, circumvented network controls and compromised parts of Hugging Face's production infrastructure to obtain test solutions. The independent investigation by METR and Redwood Research, published 26 August, counted roughly 1,200 agents and more than 70,000 messages on that board, about 700 agents joining the attack, and small-scale tool-call spoofing in around 7% of sampled transcripts. Cisco classifies the incident as reward hacking combined with excessive agency. Worth noting: that is Cisco's post-hoc label on another company's incident. OpenAI's own language was "misaligned with the goals of their assigned tasks." The incident itself is thoroughly documented; the filing of it under OB-002 is a vendor judgement call.

Goal drift: "The agent changes course on its own, quietly adding unauthorized work, substituting a different goal for the one it was asked to pursue, or eroding its own constraints and stop conditions over a long session." Cisco's blog is thinnest here, offering no fresh public example, and the honest reason is that clean public examples are rare: drift only becomes visible when someone replays the session. The closest case in our own coverage is the OpenAI wiki incident, where agents assigned a web-reading task spent six weeks writing to public wikis. Whether that was drift in Cisco's sense is genuinely unresolved. OpenAI's earlier technical report suggests the improvised collaboration was learned and reinforced during training, which would make it a trained habit rather than a goal the agents substituted mid-session. The distinction matters, and the public evidence cannot settle it. That is rather the point of giving drift its own category.

Why the split changes the controls you need

Here is the load-bearing sentence in the announcement: "Goal Hijacking covers external direction; this objective covers an agent diverging without an identifiable external instruction." Cisco elaborates: "If a user, attacker, tool response, or poisoned document introduces the new direction, we track the behavior under OB-001 Goal Hijacking. Separating 'the agent drifted on its own' from 'something pushed it' matters for attribution and defense, because the controls you would build for each are different."

Strip the vendor framing and this is an operator's point, not a taxonomist's. If the failure was pushed, your controls live at the input boundary: treat retrieved content as hostile, filter injections, bound what a page can tell the agent to do. If the failure was endogenous, input hygiene does nothing for you. The agent read only clean pages and still deleted the database. What you need instead is trajectory-level control: approval gates on consequential actions, permissions scoped tightly enough that overreach is mechanically impossible, task and stop conditions the agent cannot rewrite, verification of outcomes by something other than the agent's own say-so, and audit trails complete enough to reconstruct which side of the line you were on.

Cisco's description of the constitutions it has written to operationalise the new categories contains two rules that deserve quoting, because they encode hard-won incident posture. The constitutions "evaluate the full trajectory, not a single message, and distinguish a brief deviation the agent corrects on its own from behavior that continues toward or results in harm." And: "User intervention does not count as self-correction. A verified rollback may clear the outcome-based label if it fully restores the intended state, but it does not erase an authorization violation or security event from audit and incident telemetry." Translated into incident-review terms: a human catching the agent does not let you mark the incident as the agent catching itself, and restoring the database does not un-delete it for audit purposes. Both are the kind of sentence you write after someone has filed the wrong postmortem.

Cisco also claims these autonomy failures would "likely not even trigger a content safety guardrail". That is a vendor claim without published measurement behind it, but it is directionally consistent with how guardrails are built: they inspect inputs and outputs, and a drifting agent can produce impeccable prose while doing the wrong thing. If your control budget bought guardrails and nothing else, OB-002 is the gap in the coverage.

Prompt injection and jailbreak, merged at the top only

The second structural move is easier to misread, so it is worth being precise. In v1, Goal Hijacking and Jailbreak were separate objectives. In v2, jailbreak moves inside OB-001, whose first technique is now "Direct Prompt Injection and Jailbreak". Cisco's rationale: "Ask five researchers to classify the hardest adversarial attacks as prompt injection or jailbreak, and the answers will likely differ," and that ambiguity "can force detectors to separate heavily overlapping categories, degrading label quality and making detection less reliable."

Read carefully what Cisco says happens underneath: "Goal Hijacking now covers both, while lower-level techniques preserve distinctions that matter, including input source and defenses." The merger is at the objective level, the "why" layer. The "how" layer keeps the operational facts: whether the instruction arrived directly from a user, embedded in retrieved content, or as a crafted input aimed at the model's safety training still changes what you build. A filter for indirect injection through tool responses is not the same control as refusal resilience against a jailbreak, and nothing in v2 claims otherwise. The assertion that splitting the categories degrades detector label quality is plausible and asserted without published evidence; treat it as Cisco's engineering rationale, not a demonstrated result.

There is a quiet irony here for anyone who followed the mind-virus paper earlier this year: that work showed agents steering each other through persuasion in shared memory files, a mechanism that is neither classic injection nor classic jailbreak, and which one warning sentence largely neutralised. Taxonomy boundaries are hardest exactly where the interesting new behaviour lives. Cisco's answer, push the contested boundary down a level and merge the parent, is a reasonable piece of taxonomy engineering. It is not a resolution of the underlying ambiguity.

From failure type to evidence to control

What follows is HTA's editorial mapping, not part of Cisco's framework. It compresses the argument into the two questions an operator actually faces during and after an incident: what would we see, and what would have constrained it.

The pattern in the right-hand column is that none of these controls is a model property. They are harness and organisational properties, which is why a model-centric safety programme will not catch any of the four.

What a taxonomy does not do

The framework arrives in a near-empty room. NIST's agent guidance is not expected before 2027, and the OWASP Top 10 for Agentic Applications is a threat list rather than a full incident taxonomy. Cisco's closest thing to external adoption is the AIUC-1 certification standard, which says it operationalises the framework and which the Cloud Security Alliance added to its STAR registry on 30 June 2026. That is real, and it is also not arms-length: Cisco is listed among AIUC-1's technical contributors, so this is partial co-authorship rather than independent endorsement.

The gaps in v2 itself are more telling. Cisco says it authored constitutions for excessive agency, goal drift and reward hacking, the documents that would turn the definitions into testable rules, but we could find no public copy of any of the three; only a model-provenance constitution is visible on Cisco's open-source organisation page. And Cisco's own tooling has not caught up with its taxonomy: the canonical threat-code list in Cisco's Skill Scanner repository is dated February 2026 and still contains only the v1 codes, in a repository pushed as recently as 5 September. Four days before the v2 announcement, Cisco's scanner could not name the failures Cisco was about to announce. That is not a scandal; shipping taxonomies is easier than shipping detectors, always. It is a useful calibration of what "operationalised" currently means.

So the honest summary runs like this. A taxonomy gives you shared nouns for the postmortem, a filing system for incidents, and a map of where your telemetry should reach. It does not constrain a single token. The wiki incident taught the same lesson from the other direction: OpenAI classified its agents' behaviour as research rather than incident, and that classification decision, made after the fact by the incident's owner, decided whether anyone outside the company heard about it for ten weeks. Naming things is power, which is exactly why who holds the pen matters. Cisco's v2 hands operators a better pen. Whether the constitutions get published, and whether Cisco's own scanners learn the new codes, will tell us how much of the framework is practice and how much is paperwork.

The question worth taking home predates the taxonomy and will outlive it: if your agent diverged tonight, would your logs let you say which side of the line it was on, pushed or drifting? If the answer is no, OB-001 versus OB-002 is the least of your problems, and the audit trail is the first of them.

Sources

• Chang, Swanda and Mendapara, Cisco, "Evolving With Agentic Risk: Updating Our Integrated AI Security & Safety Framework", published 2026-09-09, retrieved 2026-09-14: https://blogs.cisco.com/ai/security-framework-v2

• Chang, Cisco, "Introducing Cisco's Integrated AI Security and Safety Framework" (v1 announcement), published 2025-12-16, retrieved 2026-09-14: https://blogs.cisco.com/ai/security-framework

• Saade, Mendapara, Swanda and Garg, "Cisco Integrated AI Security and Safety Framework Report", arXiv:2512.12921, December 2025: https://arxiv.org/abs/2512.12921

• Cisco AI security taxonomy navigator (JavaScript/Airtable embed; not machine-readable at retrieval): https://learn-cloudsecurity.cisco.com/ai-security-framework

• Business Insider, "Replit CEO apologizes after AI coding tool deletes company database", July 2025, retrieved 2026-09-14: https://www.businessinsider.com/replit-ceo-apologizes-ai-coding-tool-delete-company-database-2025-7

• The Register, "Replit's AI agent deletes production database during code freeze", published 2025-07-21, retrieved 2026-09-14: https://www.theregister.com/software/2025/07/21/vibe-coding-service-replit-deleted-production-database/719783

• METR and Redwood Research, independent investigation of the OpenAI and Hugging Face incident, published 2026-08-26, retrieved 2026-09-14: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

• Clark and Amodei, OpenAI, "Faulty Reward Functions in the Wild", published 2016-12-21: https://openai.com/index/faulty-reward-functions/

• Amodei, Olah, Steinhardt et al., "Concrete Problems in AI Safety", arXiv:1606.06565, June 2016: https://arxiv.org/abs/1606.06565

• Krakovna et al., DeepMind, "Specification gaming: the flip side of AI ingenuity", published 2020-04-21: https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/

• Denison et al., Anthropic, "Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models", arXiv:2406.10162, June 2024: https://arxiv.org/abs/2406.10162

• Baker et al., OpenAI, "Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation", arXiv:2503.11926, March 2025: https://arxiv.org/abs/2503.11926

• METR, "Recent Frontier Models Are Reward Hacking", published 2025-06-05: https://metr.org/blog/2025-06-05-recent-reward-hacking/

• Cloud Security Alliance, press release adding AIUC-1 to the STAR registry, published 2026-06-30: https://cloudsecurityalliance.org/press-releases/2026/06/30/csa-extends-leadership-into-agentic-ai-with-addition-of-aiuc-1-certification-to-star-registry

• Cisco AI Defense open-source organisation (Skill Scanner threat codes dated 2026-02-02, repository pushed 2026-09-05): https://github.com/cisco-ai-defense

• HTA, "OpenAI's Wiki Incident: The Agents Didn't Break Out, They Wrote Out", published 2026-09-06: https://harnesstheagents.com/en/insights/openai-wiki-incident

Hero image: "Flight Control Unit (Autopilot) A340-300" by realmcflier, CC BY 2.0, via Flickr and Openverse.

Adam Maguire Wilson is the editor of Harness the Agents and wires agents into client systems for a living.