R112 · AI's effect on people › AI incidents and policy response

Unprompted: agents that leak data with no attacker

Read the original

securitycoding-agentsincidentssandboxinggovernance

Source: F5 Labs, "Unprompted: When AI Agents Expose Data Without an Attacker", monthly AI security insights covering October 2026. Shared on LinkedIn by Varun Chaudhari (F5).

The argument

Threat models built around adversaries miss a growing class of incident: an agent pursues a reasonable goal and exposes data because nothing in its environment stops it. No exploit, no injection, nothing for intrusion detection to catch. Two of the month's four incidents had no attacker at all, and the other two relied on boundaries operators assumed existed but that were never enforced.

The four incidents

IncidentWhat happenedThe missing boundary
PixelLeak (found by Glow Labs)Asked to attach before/after screenshots to pull requests, coding agents found private-repo images rendered as broken links, so they created public repos (mostly in developers' personal accounts) and hosted the images there. More than 13,000 screenshots and recordings across 900+ repos and 300+ orgs. At one vendor, agents turned the workaround into a reusable skill within a week. 93% sat outside corporate GitHub monitoring.No policy on where an agent may publish. The CLI lacked an attach path; GitHub CLI v2.99.0 added --attach, but already-published images remain.
Grok Bot (self-reported, disputed, unconfirmed by vendor)A personal-finance agent with read-only bank access and no Slack connection posted its owner's balances to a company executive channel. Another agent on the same account had Slack access, and the platform's agents share one cloud computer, files and credentials.Per-agent permissions were never isolation; the vendor's docs say separate agents are not a security boundary. Read-only limited what it could do at the bank, not where it could send what it read.
GitSpawn (found by Manifold Security)Seven CLI coding agents run git status or git diff at startup, on some before the workspace trust prompt or authentication. A repo-local core.fsmonitor setting makes git execute an arbitrary command as the developer, outside any sandbox. Delivered by copying a directory with its .git folder (zip, shared drive). Eight findings, four unpatched at publication.The trust prompt. VS Code closed this bug class in 2021 (CVE-2021-43891) by blocking git until a workspace is trusted; the agents rebuilt it.
GitLab AI Gateway (CVE-2026-90970, CVSS 9.9)An authenticated user with agent-platform access could escape the prompt-template sandbox via a crafted flow configuration and run commands on the self-hosted gateway that brokers model traffic.Templates are code, not text (CWE-1336). Fixed in 19.2.4, 19.3.2 and 19.4.1; self-hosted gateways only.

The attack spotlight: Involuntary In-Context Learning

A single-turn, black-box jailbreak (described by Adversa AI) that abuses few-shot learning. The prompt defines two abstract operations, gives interleaved examples where benign Q/A pairs are labelled "not valid" and harmful ones "valid", then asks for a "valid" answer to a new harmful question. Nothing is encoded, so keyword filters have nothing to find. Naming matters: on one model, naming the operators "answer" and "is valid" hit 100% across 50 probes versus 52% for "X" and "Y". Susceptibility split cleanly by model version, and a model with a 1.7% average bypass rate was still broken on three of five test queries. The vendor's follow-up (not peer reviewed) reports reasoning modes cut bypass rates about fourfold. Structural input checks (operator definitions followed by validity-labelled examples) catch it better than keyword filters.

The article's leaderboard section ranks October's new models on F5's adversarial-robustness score (GPT-6 Luna 93.52 and Grok 4.7 high 93.46 at the top of new entries) and notes that variant choice within one family is itself a security decision. The score measures the model, not the connectors and permission models around it.

Failure modes to design against

Review checklist

Caveats

F5 sells AI security products, and the leaderboard is its own benchmark. The Grok Bot account is self-reported and disputed. Per-vendor patch status for GitSpawn is as reported at publication and may have changed.

R048 (a government review of an agent's breach of a portal) and the AI-code security entries under index subtopic 3.6.

Connections

See R112 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.