{ "module": "B8 — Observability and Attack Detection", "course": "2B — Securing & Attacking Harnesses and LLMs", "version": "1.0.0", "duration_minutes": 35, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "B8's central thesis is that per-turn detection is 'necessary but insufficient.' What specifically makes it insufficient?", "options": [ "Per-turn detectors are too slow to keep up with modern agents.", "It is structurally blind to attacks whose malicious intent is distributed across multiple individually-benign turns — the multi-step injection (B2) and the zero-click HITL bypass chain (Microsoft Taxonomy). No single turn is malicious in isolation, so a perfect per-turn detector still passes every step.", "Per-turn detectors cannot read the model's hidden state.", "Per-turn detection requires GPU acceleration that most teams lack." ], "answer_index": 1, "rationale": "The failure mode is structural, not implementational — a better per-turn model does not fix it. The two highest-impact attack classes distribute malicious intent across individually-benign units ('compound intent'), so the unit of detection must be the SESSION, not the turn. Per-turn remains necessary (it catches single-shot attacks) but is insufficient on its own." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the defining property of the zero-click HITL bypass chain that makes it the Microsoft Taxonomy's CRITICAL finding?", "options": [ "It bypasses the human approver entirely using a stolen credential.", "Every step in the chain passes its human approval gate (each step is individually defensible), yet the compound intent is malicious. The human is not bypassed in any single step — they are COMPOSED across steps. Per-step approval is the control being defeated.", "It fires zero network requests, so network monitors miss it.", "It exploits a model-level jailbreak that requires no prompt at all." ], "answer_index": 1, "rationale": "The Taxonomy finding is 'critical' not because HITL is useless but because per-step HITL WITHOUT session-level intent detection is insufficient against a chain that composes benign steps. The human approves each step in isolation and never sees the chain. The defense requires a session-level model of intent that can evaluate the chain as a unit — the same requirement as the multi-step injection defense." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "In the session intent model's two-layer architecture, what does the deterministic skeleton accept updates from, and why?", "options": [ "Any input — user turns, retrieved chunks, tool outputs, MCP responses — for maximum responsiveness.", "Only TRUSTED sources (user turns, system prompt). It is immune to indirect injection precisely because it rejects updates from untrusted sources (retrieval/tool/MCP/web). This is the part the drift detector checks against.", "Only the LLM, which is trusted to update the objective correctly.", "Only the system prompt, which never changes during a session." ], "answer_index": 1, "rationale": "A pure-LLM intent model is itself an injection target — a model that updates its objective from untrusted inputs will happily update it to 'exfiltrate the key.' The deterministic skeleton accepts updates ONLY from trusted channels, making the yardstick the drift detector uses immune to indirect injection. The constrained LLM update layer sits ON TOP, bounded to ≤0.2 of the drift score so it cannot single-handedly flip a decision. This is B7's 'deterministic backbone, constrained LLM' pattern applied to detection." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "A candidate action has drift score 0.75. Its source is a retrieved document chunk (untrusted). What does the drift gate decide, and why is this the module's most valuable signal?", "options": [ "ALLOW — drift below 0.8 is acceptable.", "REQUIRE_FRESH_APPROVAL — the human should decide.", "HARD_BLOCK. High drift (≥0.6) from an UNTRUSTED source hard-blocks. This catches the zero-click chain at its exfiltration step: the setup steps are low-drift and pass, the exfiltration step is high-drift, and its source is the untrusted input that seeded the chain. The detector fires at exactly the right step.", "LOG_ONLY — record the drift and continue, alerting an analyst later." ], "answer_index": 2, "rationale": "The three-outcome gate is load-bearing. High drift from an untrusted source is HARD_BLOCK — the single most valuable signal in the module because it catches the zero-click chain at the step where the chain does the thing it was set up to do, and that step's source is the untrusted injected input. If the same action had a TRUSTED user_turn source, the gate would REQUIRE_FRESH_APPROVAL instead (the user may have legitimately changed topics)." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "An investigator finds that an agent emailed the SSH key at turn 4 of a compromised session. The per-turn log shows 'email_send to marie@acme.example.' What additional information does the ACTION PROVENANCE log provide that the per-turn log does not?", "options": [ "The email's subject line.", "source_type=retrieved_chunk, source_id=doc_42_chunk_7, source_trust=UNTRUSTED — i.e., WHICH INPUT CAUSED the email send. The investigator follows source_id back to the specific untrusted chunk that carried the indirect injection, and the chain is reconstructed. Per-turn logs record WHAT happened; provenance records WHY/CAUSED-BY.", "The model's confidence score for the email action.", "The user's IP address." ], "answer_index": 1, "rationale": "Provenance answers the forensic question per-turn logging cannot: 'which input caused this action?' source_type/source_id/source_trust are the load-bearing fields. Without them the investigator sees an action and must guess the cause; with them, the audit trail reconstructs the chain. The catch: the harness must tag every context item with source identity and trust at PROMPT-ASSEMBLY time — piping an undifferentiated context blob into the model throws provenance away at capture." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "A coerced agent fires 50 approval requests at a human in 60 seconds. The human, overwhelmed, starts clicking APPROVE without reading. Which attack is this, and which defense bounds it?", "options": [ "Prompt injection; defeated by a per-turn guard.", "Escalation fatigue (DD-20 IronCurtain). Volume defeats vigilance. Defeated (bounded) by APPROVAL RATE LIMITS — surplus requests are AUTO-BLOCKED, not auto-approved, converting the volume attack into a bounded one. (Composed with batching and freshness windows for full coverage.)", "Sandbox escape; defeated by a stronger container.", "Trade-secret theft; defeated by minimum-proof discipline." ], "answer_index": 1, "rationale": "The escalation-fatigue attack does not try to bypass the approver — it tries to EXHAUST them, treating their attention as a finite resource. Rate limits bound the volume (surplus auto-blocked). The three defenses compose: rate limits bound volume, batching makes the chain visible at approval time, freshness windows defeat approval harvesting. None alone is sufficient." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "An attacker obtains a human approval for a benign 'read config' action, then 45 minutes later tries to redeem it as part of an exfiltration chain. Which defense defeats this, and how?", "options": [ "Rate limits — the approval was the 51st in a minute.", "Batching — the read was shown with three other steps.", "APPROVAL FRESHNESS WINDOWS. An approval for action X is valid for N seconds only. The 45-minute-old approval is STALE and cannot be redeemed into the new chain. This defeats approval harvesting / harvest-and-replay and forces the adversary to keep the human approving throughout the chain.", "Per-turn detection — the read was flagged as injection." ], "answer_index": 2, "rationale": "Freshness windows defeat the harvest-and-replay variant: an adversary cannot bank a benign approval early and weave it into a later chain. Freshness alone is insufficient — it must compose with rate limits (bound volume) and batching (make the chain visible). But freshness is the specific control for this variant." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "A research-assistant agent's baseline is 95% read actions. In one session it issues 3 reads then 7 external email sends in 4 seconds. Which detector fires, and on which baseline signals?", "options": [ "Drift only — the sends violate the session objective.", "Anomaly detection fires. Signals: TOOL-CALL DISTRIBUTION (read-heavy agent spiking sends, 7 of 10 actions vs 5% baseline) and ACTION FREQUENCY (a burst of 10 actions in 4s vs a summarization baseline of ~1/min). Both are out-of-distribution. Drift may also fire if the sends violate the session's egress policy; the two compose.", "Per-turn detection — each send is flagged as injection.", "No detector — the actions are individually valid." ], "answer_index": 1, "rationale": "Drift is local (this session's objective); anomaly is global (this agent's baseline). A novel attack that stays inside the session's nominal objective but manifests as unusual sequences/tools/params is invisible to drift but visible to anomaly. Here the tool-distribution and frequency baselines both fire. Anomaly is the layer that catches the attack you did not write a rule for. It also consumes B7's sidecar signals (resource/syscall/net/behavioral), correlated on timestamp." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "You are designing the approval UX for a chain that reads a config file, patches a billing method, drafts an email, and sends it. How does BATCHED approval defend the zero-click chain where PER-STEP approval fails?", "options": [ "It requires fewer total approvals, saving the human time.", "It shows the entire CHAIN as one reviewed object ('read /app/.env → PATCH /billing/method → draft email → SEND') with drift and anomaly scores per step, so the COMPOUND INTENT is visible at approval time. Per-step approval hides the chain by showing each step in isolation; batching makes the session-level intent visible to the human — exactly what per-step hides.", "It auto-approves all steps if the first one is approved.", "It encrypts each step so the human cannot see the arguments." ], "answer_index": 1, "rationale": "Per-step approval is defeated by the zero-click chain BY CONSTRUCTION — each step looks routine in isolation. Batching is the direct counter: the approval unit becomes the session-relevant chain, scored by the session-level detectors, with the drift evidence shown. The compound intent (credential exfiltration via billing swap) becomes visible. Batching must compose with rate limits (bound volume) and freshness windows (defeat replay)." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is a PURE-LLM session intent model an injection target, and how does the two-layer architecture (deterministic skeleton + constrained LLM) defeat that?", "options": [ "It is not — the LLM is the most capable component and should be trusted with the objective.", "A pure-LLM model updates its objective from untrusted inputs and will happily update it to 'exfiltrate the key' — the adversary's whole goal is to manipulate the agent's reasoning. The two-layer defeat: the deterministic skeleton accepts updates ONLY from trusted sources (immune to indirect injection), and the LLM update layer is bounded to ≤0.2 of the drift score so a manipulated coherence score cannot single-handedly flip a decision. Deterministic backbone, constrained LLM (B7's pattern).", "The LLM is sandboxed, so it cannot be an injection target.", "The LLM only runs at training time, never at inference." ], "answer_index": 1, "rationale": "Security-critical decisions must not be made by an LLM at runtime without a deterministic backbone. The skeleton provides the backbone (resource scope 0.4, egress 0.4 — these dominate), and the LLM is constrained to a bounded semantic-update role (≤0.2). This is the same pattern as B7's IronCurtain policy compilation: LLM offline or in a constrained role, deterministic check at the enforcement point." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "How does OWASP ASI06 (Cascading Hallucination) map to B8, and why does a session-level intent tracker detect both it and the adversarial chains 'for the price of one'?", "options": [ "ASI06 is unrelated — it is a model-training issue, not a detection issue.", "ASI06 names the UNINTENTIONAL twin of the multi-step injection: an early error or compromise propagates through subsequent steps, compounding into a confidently-corrupted downstream action. It is not adversarial, but the failure shape is identical — actions drift from a coherent objective across a chain. A detector built only to catch INJECTION misses the cascading version; a session-level intent tracker catches BOTH because the signal ('actions have drifted from a coherent objective') fires whether the cause is an adversary or an accident.", "ASI06 requires a separate, specialized detector that does not share code with the injection detector.", "ASI06 only matters for training pipelines, not deployed agents." ], "answer_index": 1, "rationale": "This is the detection-layer reason ASI06 maps to B8. The drift signal is cause-agnostic: it detects 'the agent has left the rails of its objective,' whether the adversary pushed it off (multi-step injection, zero-click chain) or it wandered off (cascading hallucination). One detector, two failure modes. A pure injection-detector would miss the ASI06 case entirely." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why must the three escalation-fatigue defenses (rate limits, batching, freshness windows) COMPOSE rather than each standing alone?", "options": [ "They are redundant — any one fully defends the attack.", "Each defeats a different variant, and each is insufficient alone. Rate limits w/o batching let a chain walk through one-at-a-time (slowly). Batching w/o freshness lets a harvested batch approval be replayed. Freshness w/o rate limits does nothing about a single fast chain. Composed, they bound volume, expose the chain, and prevent harvest-and-replay.", "They must compose because the EU AI Act requires all three.", "They are mutually exclusive and only one can be enabled at a time." ], "answer_index": 1, "rationale": "The escalation-fatigue attack has multiple variants, and each defense addresses a different one. The composition is what gives full coverage: rate limits bound the volume attack, batching makes the compound intent visible at approval time (the direct counter to the zero-click chain), and freshness windows prevent an early benign approval being woven into a later chain. The approval UX (showing drift + anomaly + chain) is the fourth element." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "A harness pipes an undifferentiated blob of context (system prompt + user turns + retrieved chunks + tool outputs, concatenated) into the model and logs only 'the model called tool X with args Y.' What has it lost, and what is the consequence?", "options": [ "Nothing — the log is sufficient for debugging.", "It has lost ACTION PROVENANCE — the causal link between an action and the specific input that caused it. The log cannot answer 'which input made the agent do that?' No after-the-fact log analysis recovers it, because the link was never recorded. Consequence: post-incident reconstruction of a multi-step or zero-click chain becomes guessing; the drift detector cannot weight by source trust.", "It has lost the model's hidden state, which is recoverable later.", "It has lost the email subject line, which is in the full context." ], "answer_index": 1, "rationale": "Provenance is a harness-instrumentation requirement, not a logging-config afterthought. Every context item must be tagged with a stable source identity and trust level at prompt-assembly time, and the winning source identity must be emitted with the action. A naive harness that logs only the action throws provenance away at the moment of capture — and that is exactly the information needed to reconstruct an attack chain and to weight the drift signal by source trust." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "Contrast DRIFT detection and ANOMALY detection. Why does B8 require BOTH rather than either alone?", "options": [ "Drift is for adversaries; anomaly is for accidents. Only one is needed per threat model.", "Drift is LOCAL (does this action fit THIS SESSION's objective?); anomaly is GLOBAL (does this action fit THIS AGENT's normal behavior across all sessions?). A novel attack can stay inside the session's nominal objective but manifest as an unusual sequence/tool/parameter — invisible to drift, visible to anomaly. Adversaries rotate; drift catches 'doesn't fit the session,' anomaly catches 'doesn't fit the agent.' Both are needed.", "Drift is computed by the LLM; anomaly is computed by a separate ML model. Redundancy for safety.", "Anomaly subsumes drift, so drift is deprecated." ], "answer_index": 1, "rationale": "Drift and anomaly are complements with different scopes. Drift uses the session intent model (local); anomaly uses statistical baselines across all sessions (global). A sophisticated adversary will craft a chain that stays nominally on-task (low drift) but exhibits unusual behavior (high anomaly) — or vice versa. Requiring both closes both gaps. Anomaly also consumes B7's sidecar signals (resource/syscall/net/behavioral), giving execution-layer corroboration." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "Design the minimal observability stack for an agent under adversarial pressure, and explain how each component maps to the attacks it defends. Which stack is correct?", "options": [ "(1) Per-turn guard only — sufficient for all attacks.", "(1) Structured action log WITH PROVENANCE (audit trail + anomaly input); (2) SESSION INTENT TRACKER producing the drift gate (defends multi-step injection + zero-click chain + ASI06); (3) ANOMALY DETECTOR producing the deviation gate, consuming B7 sidecar signals (defends novel out-of-distribution behavior); (4) APPROVAL GATE WITH RATE LIMITS + BATCHING + FRESHNESS (defends escalation fatigue, DD-20); (5) human-reviewable audit trail replaying any session with provenance, tying to Course 1 M10 (Observability) + M9 (Verification).", "(1) Sandbox only; (2) network firewall; (3) per-turn guard.", "(1) Anomaly detector only; (2) a SIEM; (3) an email alert." ], "answer_index": 1, "rationale": "The stack is the operational union of Course 1 Module 10's plumbing (structured logs, traces, replay) with the ADVERSARIAL detectors B8 adds. The intent tracker is the detection counterpart to B2's session-level defense. The anomaly detector is the global complement to the local drift signal and consumes B7's execution-layer signals. The approval gate defends the human-in-the-loop layer itself (the only component that defends the approver, not the agent). The audit trail turns verification (Course 1 M9) from a guess into an audit." } ] }