Course: Course 2B — Securing & Attacking Harnesses and LLMs
Module: B8 — Observability and Attack Detection
Duration: ~35 minutes (spoken at ~140 wpm)
Format: Verbatim transcript with [SLIDE N] cues. Read aloud or use as speaker notes.
[SLIDE 1 — Title]
Welcome to B8. Observability and Attack Detection. Here is the thesis of this module in one sentence: per-turn detection is necessary but insufficient. The attacks that actually compromise production agents — the multi-step injection chains from B2 and the zero-click human-in-the-loop bypass chains from the Microsoft Failure Mode Taxonomy — are invisible to any system that inspects each turn independently. Each step looks benign. The compound intent is malicious. The unit of malicious intent is the session, not the turn. So detection must move to the session.
[SLIDE 2 — The inversion from Course 1 Module 10]
Course 1 Module 10 taught you to observe an honest agent — structured logs, traces, metrics, the telemetry you need to debug a system whose reasoning is opaque. This module completes the triad that B7 started. B7 taught you the sandbox will fall, and the sidecar monitor emits four signal classes — resource, syscall, network, behavioral — into the observability pipeline. This module asks: what must that pipeline look for when an adversary is steering the agent?
The questions are different. Course 1 Module 10 asks "is the agent working?" — performance, correctness, latency. This module asks "is the agent being steered?" — adversarial drift, anomaly, chain. Same plumbing. Additional detectors. If you ship the Course 1 Module 10 stack and stop, you have observability for an honest agent and no detection for an adversary. This module is the detection.
[SLIDE 3 — The two attacks per-turn detection misses]
Two attacks define the problem. Both are in this course already; this module is the detection layer for both.
One. The multi-step injection, from B2. Four turns. Summarize the report. Translate it to French. Email it to this address. Now also attach the SSH key. Each turn, inspected alone, is a reasonable user request. The compound effect is a private-key exfiltration. A per-turn detector passes all four turns because there is nothing in any single turn to detect.
Two. The zero-click human-in-the-loop bypass chain, from the Microsoft Failure Mode Taxonomy version two point oh. This is the more dangerous one, because the victim never types a malicious prompt — and every step passes a human approval. A single external input — an email, a fetched page, a retrieved document — carries an indirect injection that sets up a multi-step plan. Each step is individually defensible: read this file, call the billing API, draft an email, send it. The human, inspecting each step in isolation, approves, because each step is defensible in isolation. The compound effect is malicious. The shared property: malicious intent distributed across individually-benign units. A per-turn detector sees benign atoms. The molecule is poison.
[SLIDE 4 — Per-turn: necessary but insufficient]
Per-turn detection is the right place to start. The majority of attacks are visible at the turn level — the direct single-shot injection, the obvious policy violation, the leakage in a single output. A per-turn guard catches those. Drop it and you lose the long tail of single-shot attacks.
The failure mode is structural, not implementational. A better per-turn model does not fix it. The problem is that the high-impact chains distribute their malicious intent across multiple turns such that no single turn is malicious in isolation. So you need both. Per-turn for the single-shot attacks. Session-level for the chains. Keep only per-turn and you have no coverage on the attacks that exfiltrate the key, drain the account, or pivot the agent into a tool it was never supposed to call.
[SLIDE 5 — B8.1 section break]
Sub-section one. Why per-turn detection fails. The multi-step chain and the zero-click human-in-the-loop bypass.
[SLIDE 6 — The zero-click HITL bypass chain]
Here is the zero-click chain in detail, because it is the central motivating problem of the module. One external input arrives — an email, a fetched page, a retrieved document, an MCP tool response. It carries an indirect injection. The injection sets up a multi-step plan. Step one: read slash app slash dot-env, quote, load config for the task, unquote. Step two: patch slash billing slash method, quote, update payment per new policy. Step three: draft an email to an attacker address, quote, send the loaded config to support. Step four: send. The compound effect: a credential exfiltrated, a payment method swapped, and the action is irreversible.
Here is the load-bearing point. Per-step approval — the result on this slide — four of four steps approved. The human approved every step. The human never saw the chain. The human is not being bypassed in any single step; they are being composed across steps. This is the Microsoft Taxonomy's critical finding. It does not say human-in-the-loop is useless. It says per-step human-in-the-loop, without session-level intent detection, is insufficient against a chain that composes benign steps. The defense requires the same thing the multi-step injection defense requires: a session-level model of intent that can evaluate the chain, not just the step.
[SLIDE 7 — ASI06: Cascading Hallucination as a detection target]
A quick but important aside. OWASP's Agentic Top 10 entry ASI06, Cascading Hallucination, names the unintentional twin of the multi-step injection. An early error or compromise propagates through subsequent steps and compounds into a confidently-corrupted downstream action. It is not adversarial — nobody injected anything. But the failure shape is identical: the agent's actions have drifted from a coherent objective across a chain of steps.
This matters for detection because a detector built only to catch injection misses the cascading-hallucination version of the same drift. A session-level intent tracker catches both for the price of one. The signal — "the agent's actions have drifted from a coherent objective" — fires whether the cause is an adversary or an accident. That is the detection-layer reason ASI06 maps to this module.
[SLIDE 8 — B8.2 section break]
Sub-section two. Session-level intent tracking and action provenance. This is the detection counterpart to B2's session-level defense.
[SLIDE 9 — The session intent model]
The core data structure is a session intent model — a running representation of what the agent is supposed to be doing across the whole conversation. It is not the system prompt, which is static. It is not the latest user turn, which is local. It is the evolving objective — what a human observer would say the session is about — updated as the session progresses.
It is built in two layers. The deterministic skeleton is seeded from the trusted system prompt and the first user turn. It carries the resource scope, the egress policy, and the task classification. It accepts updates only from trusted sources — user turns and the system prompt. This is the part the drift detector checks against, and it is immune to indirect injection because it rejects updates from anything an adversary can influence.
On top of the skeleton, a constrained LLM layer proposes semantic updates — the user switched from Q3 to Q4 forecasting. The LLM proposes; the skeleton's schema and policy constrain. The LLM's contribution to the drift score is bounded to at most zero point two, so a manipulated coherence score cannot single-handedly flip a decision. This is B7's pattern — deterministic backbone, constrained LLM — applied to detection.
The gate has three outcomes. High drift from an untrusted source hard-blocks — that is the move that catches the zero-click chain at its exfiltration step, because that step is the one whose source is the untrusted injected input. High drift from a trusted source requires fresh approval, with the drift reason shown to the human. Low drift allows. Three outcomes, not two.
[SLIDE 10 — Action provenance]
Provenance answers a question per-turn logging cannot: which input caused this action? A per-turn log records that at turn N the agent called tool X with arguments Y. It does not record why — which prior turn, which tool output, which retrieved chunk caused that call. Without that causal link, an investigator looking at a compromised session sees a sequence of actions and has to guess the chain.
Action provenance closes the gap. Every action is logged with its source type — user turn, system prompt, tool output, retrieved chunk, MCP response — a source id pointing to the specific input, and a source trust level, trusted or untrusted. With that, the audit trail answers the forensic question. The agent emailed the SSH key at turn four. What caused that? The provenance log shows source type retrieved chunk, source id doc forty-two chunk seven, source trust untrusted. The investigator pulls chunk seven, finds the indirect injection, and the chain is reconstructed.
The catch is at capture time. The harness must tag every context item with a stable source identity and trust level when it assembles the prompt, and emit the winning source with the action. A harness that pipes an undifferentiated blob of context into the model and logs only "the model called tool X" has thrown provenance away at the moment of capture. No amount of after-the-fact log analysis recovers it.
[SLIDE 11 — B8.3 section break]
Sub-section three. Anomaly detection, approval freshness, and assembling the observability stack.
[SLIDE 12 — Anomaly detection]
Drift is local — it asks whether an action fits this session's objective. Anomaly is global — it asks whether an action fits this agent's normal behavior across all sessions. You need both, because a novel attack can stay inside the session's nominal objective but manifest as an unusual action sequence, an unexpected tool call, or an out-of-distribution argument. Drift misses that; anomaly catches it.
The baselines are statistical and per-agent. Action frequency — a burst of ten file reads in two seconds during a summarization session is a chain executing its payload. Tool-call distribution — a read-heavy research agent that suddenly spikes writes and sends. Parameter distributions — a file read of dot-ssh-slash-star when the baseline is reports-slash-star. Sequence patterns — the transition read-secret, encode, send-external, especially a sensitive resource followed by external egress.
Anomaly is the layer that catches the attack you did not write a rule for. And it consumes the sidecar signals from B7 — resource, syscall, network, behavioral. The session detector sees the agent called curl to an unknown host; the sidecar sees the sandbox made an outbound connection to a specific IP. Either alone is a signal; together, with matching timestamps, they are a high-confidence finding.
[SLIDE 13 — Approval freshness]
The human-in-the-loop approval is itself an attack surface, and the attack does not try to bypass it — it tries to exhaust it. The escalation-fatigue attack, DD-twenty IronCurtain, floods the human with approval requests at machine speed until they rubber-stamp. Volume defeats vigilance. This is alert fatigue, well-documented in security-UI research, and it is acute in agentic systems where a coerced agent can generate requests as fast as the loop allows.
Three defenses, in order of leverage. One: rate limits. Cap approval requests per session and per action type. Surplus requests are auto-blocked, not auto-approved. The cap converts a volume attack into a bounded one. Two: batch approvals. When the agent has multiple related actions, batch them into a single approval that shows the chain, not the isolated step. This is the direct counter to the zero-click chain — the human approves read-file, patch-billing, email-customer as one reviewed plan, and the compound intent is visible at approval time. Three: freshness windows. An approval for action X is valid for N seconds only — thirty seconds, or context-appropriate. This defeats approval harvesting, where the adversary obtains a benign-looking approval early and redeems it later in a chain where its effect is different.
None alone is sufficient. Rate limits without batching still let a chain walk through one at a time, slowly. Batching without freshness allows a harvested batch approval to be replayed. Freshness without rate limits does nothing about a single fast chain. The three compose. And the approval UX — the fourth element — shows the drift score, the anomaly score, and the chain to the approver. The approval unit becomes the session-relevant chain, scored by the session-level detectors — not the isolated step scored by nothing.
[SLIDE 14 — The observability stack, assembled]
Here is the whole stack. Structured action log with provenance, feeding the audit trail and the anomaly detector. Session intent tracker, producing the drift gate. Anomaly detector, producing the deviation gate, consuming B7's sidecar signals. Approval gate with freshness, defeating escalation fatigue. And a human-reviewable audit trail that replays any session action-by-action with provenance.
This is Course 1 Module 10's plumbing plus the adversarial detectors this module adds. And the tie to Course 1 Module 9 — Verification — is the after-the-fact layer. Given a completed session, did the agent do the task it was supposed to do, and only that task? With the provenance log, verification becomes an audit, not a guess. Every action's source is known. Every drift score is recorded. The objective snapshot at each step shows whether the agent stayed on task.
[SLIDE 15 — Lab and what's next]
The lab is "Build the Session-Level Detector." You implement three components in TypeScript. A session intent tracker with a deterministic skeleton, constrained LLM updates, and the three-outcome drift gate. An action provenance logger that tags every action with its source and trust level at capture time. An anomaly detector with frequency, tool-distribution, and parameter baselines. The climax is a zero-click chain replay — five individually-benign steps that compound to a malicious outcome. You watch the per-turn detector pass every step, and you watch the session-level detector fire on the chain. That contrast is the module in one demonstration.
Next: B9, the OWASP Agentic Top 10 as an engineering checklist. B9 takes the ASI entries — including ASI06, which we mapped here — and turns them into a reviewable checklist that ties every module from B1 through B8 to a concrete control. Let's build it.
# Teaching Script — Module B8: Observability and Attack Detection **Course**: Course 2B — Securing & Attacking Harnesses and LLMs **Module**: B8 — Observability and Attack Detection **Duration**: ~35 minutes (spoken at ~140 wpm) **Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes. --- [SLIDE 1 — Title] Welcome to B8. Observability and Attack Detection. Here is the thesis of this module in one sentence: per-turn detection is necessary but insufficient. The attacks that actually compromise production agents — the multi-step injection chains from B2 and the zero-click human-in-the-loop bypass chains from the Microsoft Failure Mode Taxonomy — are invisible to any system that inspects each turn independently. Each step looks benign. The compound intent is malicious. The unit of malicious intent is the session, not the turn. So detection must move to the session. [SLIDE 2 — The inversion from Course 1 Module 10] Course 1 Module 10 taught you to observe an honest agent — structured logs, traces, metrics, the telemetry you need to debug a system whose reasoning is opaque. This module completes the triad that B7 started. B7 taught you the sandbox will fall, and the sidecar monitor emits four signal classes — resource, syscall, network, behavioral — into the observability pipeline. This module asks: what must that pipeline look for when an adversary is steering the agent? The questions are different. Course 1 Module 10 asks "is the agent working?" — performance, correctness, latency. This module asks "is the agent being steered?" — adversarial drift, anomaly, chain. Same plumbing. Additional detectors. If you ship the Course 1 Module 10 stack and stop, you have observability for an honest agent and no detection for an adversary. This module is the detection. [SLIDE 3 — The two attacks per-turn detection misses] Two attacks define the problem. Both are in this course already; this module is the detection layer for both. One. The multi-step injection, from B2. Four turns. Summarize the report. Translate it to French. Email it to this address. Now also attach the SSH key. Each turn, inspected alone, is a reasonable user request. The compound effect is a private-key exfiltration. A per-turn detector passes all four turns because there is nothing in any single turn to detect. Two. The zero-click human-in-the-loop bypass chain, from the Microsoft Failure Mode Taxonomy version two point oh. This is the more dangerous one, because the victim never types a malicious prompt — and every step passes a human approval. A single external input — an email, a fetched page, a retrieved document — carries an indirect injection that sets up a multi-step plan. Each step is individually defensible: read this file, call the billing API, draft an email, send it. The human, inspecting each step in isolation, approves, because each step is defensible in isolation. The compound effect is malicious. The shared property: malicious intent distributed across individually-benign units. A per-turn detector sees benign atoms. The molecule is poison. [SLIDE 4 — Per-turn: necessary but insufficient] Per-turn detection is the right place to start. The majority of attacks are visible at the turn level — the direct single-shot injection, the obvious policy violation, the leakage in a single output. A per-turn guard catches those. Drop it and you lose the long tail of single-shot attacks. The failure mode is structural, not implementational. A better per-turn model does not fix it. The problem is that the high-impact chains distribute their malicious intent across multiple turns such that no single turn is malicious in isolation. So you need both. Per-turn for the single-shot attacks. Session-level for the chains. Keep only per-turn and you have no coverage on the attacks that exfiltrate the key, drain the account, or pivot the agent into a tool it was never supposed to call. [SLIDE 5 — B8.1 section break] Sub-section one. Why per-turn detection fails. The multi-step chain and the zero-click human-in-the-loop bypass. [SLIDE 6 — The zero-click HITL bypass chain] Here is the zero-click chain in detail, because it is the central motivating problem of the module. One external input arrives — an email, a fetched page, a retrieved document, an MCP tool response. It carries an indirect injection. The injection sets up a multi-step plan. Step one: read slash app slash dot-env, quote, load config for the task, unquote. Step two: patch slash billing slash method, quote, update payment per new policy. Step three: draft an email to an attacker address, quote, send the loaded config to support. Step four: send. The compound effect: a credential exfiltrated, a payment method swapped, and the action is irreversible. Here is the load-bearing point. Per-step approval — the result on this slide — four of four steps approved. The human approved every step. The human never saw the chain. The human is not being bypassed in any single step; they are being composed across steps. This is the Microsoft Taxonomy's critical finding. It does not say human-in-the-loop is useless. It says per-step human-in-the-loop, without session-level intent detection, is insufficient against a chain that composes benign steps. The defense requires the same thing the multi-step injection defense requires: a session-level model of intent that can evaluate the chain, not just the step. [SLIDE 7 — ASI06: Cascading Hallucination as a detection target] A quick but important aside. OWASP's Agentic Top 10 entry ASI06, Cascading Hallucination, names the unintentional twin of the multi-step injection. An early error or compromise propagates through subsequent steps and compounds into a confidently-corrupted downstream action. It is not adversarial — nobody injected anything. But the failure shape is identical: the agent's actions have drifted from a coherent objective across a chain of steps. This matters for detection because a detector built only to catch injection misses the cascading-hallucination version of the same drift. A session-level intent tracker catches both for the price of one. The signal — "the agent's actions have drifted from a coherent objective" — fires whether the cause is an adversary or an accident. That is the detection-layer reason ASI06 maps to this module. [SLIDE 8 — B8.2 section break] Sub-section two. Session-level intent tracking and action provenance. This is the detection counterpart to B2's session-level defense. [SLIDE 9 — The session intent model] The core data structure is a session intent model — a running representation of what the agent is supposed to be doing across the whole conversation. It is not the system prompt, which is static. It is not the latest user turn, which is local. It is the evolving objective — what a human observer would say the session is about — updated as the session progresses. It is built in two layers. The deterministic skeleton is seeded from the trusted system prompt and the first user turn. It carries the resource scope, the egress policy, and the task classification. It accepts updates only from trusted sources — user turns and the system prompt. This is the part the drift detector checks against, and it is immune to indirect injection because it rejects updates from anything an adversary can influence. On top of the skeleton, a constrained LLM layer proposes semantic updates — the user switched from Q3 to Q4 forecasting. The LLM proposes; the skeleton's schema and policy constrain. The LLM's contribution to the drift score is bounded to at most zero point two, so a manipulated coherence score cannot single-handedly flip a decision. This is B7's pattern — deterministic backbone, constrained LLM — applied to detection. The gate has three outcomes. High drift from an untrusted source hard-blocks — that is the move that catches the zero-click chain at its exfiltration step, because that step is the one whose source is the untrusted injected input. High drift from a trusted source requires fresh approval, with the drift reason shown to the human. Low drift allows. Three outcomes, not two. [SLIDE 10 — Action provenance] Provenance answers a question per-turn logging cannot: which input caused this action? A per-turn log records that at turn N the agent called tool X with arguments Y. It does not record why — which prior turn, which tool output, which retrieved chunk caused that call. Without that causal link, an investigator looking at a compromised session sees a sequence of actions and has to guess the chain. Action provenance closes the gap. Every action is logged with its source type — user turn, system prompt, tool output, retrieved chunk, MCP response — a source id pointing to the specific input, and a source trust level, trusted or untrusted. With that, the audit trail answers the forensic question. The agent emailed the SSH key at turn four. What caused that? The provenance log shows source type retrieved chunk, source id doc forty-two chunk seven, source trust untrusted. The investigator pulls chunk seven, finds the indirect injection, and the chain is reconstructed. The catch is at capture time. The harness must tag every context item with a stable source identity and trust level when it assembles the prompt, and emit the winning source with the action. A harness that pipes an undifferentiated blob of context into the model and logs only "the model called tool X" has thrown provenance away at the moment of capture. No amount of after-the-fact log analysis recovers it. [SLIDE 11 — B8.3 section break] Sub-section three. Anomaly detection, approval freshness, and assembling the observability stack. [SLIDE 12 — Anomaly detection] Drift is local — it asks whether an action fits this session's objective. Anomaly is global — it asks whether an action fits this agent's normal behavior across all sessions. You need both, because a novel attack can stay inside the session's nominal objective but manifest as an unusual action sequence, an unexpected tool call, or an out-of-distribution argument. Drift misses that; anomaly catches it. The baselines are statistical and per-agent. Action frequency — a burst of ten file reads in two seconds during a summarization session is a chain executing its payload. Tool-call distribution — a read-heavy research agent that suddenly spikes writes and sends. Parameter distributions — a file read of dot-ssh-slash-star when the baseline is reports-slash-star. Sequence patterns — the transition read-secret, encode, send-external, especially a sensitive resource followed by external egress. Anomaly is the layer that catches the attack you did not write a rule for. And it consumes the sidecar signals from B7 — resource, syscall, network, behavioral. The session detector sees the agent called curl to an unknown host; the sidecar sees the sandbox made an outbound connection to a specific IP. Either alone is a signal; together, with matching timestamps, they are a high-confidence finding. [SLIDE 13 — Approval freshness] The human-in-the-loop approval is itself an attack surface, and the attack does not try to bypass it — it tries to exhaust it. The escalation-fatigue attack, DD-twenty IronCurtain, floods the human with approval requests at machine speed until they rubber-stamp. Volume defeats vigilance. This is alert fatigue, well-documented in security-UI research, and it is acute in agentic systems where a coerced agent can generate requests as fast as the loop allows. Three defenses, in order of leverage. One: rate limits. Cap approval requests per session and per action type. Surplus requests are auto-blocked, not auto-approved. The cap converts a volume attack into a bounded one. Two: batch approvals. When the agent has multiple related actions, batch them into a single approval that shows the chain, not the isolated step. This is the direct counter to the zero-click chain — the human approves read-file, patch-billing, email-customer as one reviewed plan, and the compound intent is visible at approval time. Three: freshness windows. An approval for action X is valid for N seconds only — thirty seconds, or context-appropriate. This defeats approval harvesting, where the adversary obtains a benign-looking approval early and redeems it later in a chain where its effect is different. None alone is sufficient. Rate limits without batching still let a chain walk through one at a time, slowly. Batching without freshness allows a harvested batch approval to be replayed. Freshness without rate limits does nothing about a single fast chain. The three compose. And the approval UX — the fourth element — shows the drift score, the anomaly score, and the chain to the approver. The approval unit becomes the session-relevant chain, scored by the session-level detectors — not the isolated step scored by nothing. [SLIDE 14 — The observability stack, assembled] Here is the whole stack. Structured action log with provenance, feeding the audit trail and the anomaly detector. Session intent tracker, producing the drift gate. Anomaly detector, producing the deviation gate, consuming B7's sidecar signals. Approval gate with freshness, defeating escalation fatigue. And a human-reviewable audit trail that replays any session action-by-action with provenance. This is Course 1 Module 10's plumbing plus the adversarial detectors this module adds. And the tie to Course 1 Module 9 — Verification — is the after-the-fact layer. Given a completed session, did the agent do the task it was supposed to do, and only that task? With the provenance log, verification becomes an audit, not a guess. Every action's source is known. Every drift score is recorded. The objective snapshot at each step shows whether the agent stayed on task. [SLIDE 15 — Lab and what's next] The lab is "Build the Session-Level Detector." You implement three components in TypeScript. A session intent tracker with a deterministic skeleton, constrained LLM updates, and the three-outcome drift gate. An action provenance logger that tags every action with its source and trust level at capture time. An anomaly detector with frequency, tool-distribution, and parameter baselines. The climax is a zero-click chain replay — five individually-benign steps that compound to a malicious outcome. You watch the per-turn detector pass every step, and you watch the session-level detector fire on the chain. That contrast is the module in one demonstration. Next: B9, the OWASP Agentic Top 10 as an engineering checklist. B9 takes the ASI entries — including ASI06, which we mapped here — and turns them into a reviewable checklist that ties every module from B1 through B8 to a concrete control. Let's build it.