R089 · Agent principles and patterns › Evaluation data for agents

MetaPI: turn agent benchmarks into prompt-injection suites

Read the original

agent-securityprompt-injectionbenchmarksevaluation

Source: Wang, Xiang, Qu, Li, Zhang, Suh and Tian, "MetaPI: Constructing Prompt Injection Benchmarks from LLM Agent Benchmarks", arXiv 2611.10026, 5 Nov 2026. Surfaced by DAIR.AI.

The problem

Indirect prompt injection (IPI) benchmarks lag behind utility benchmarks: they cover few tasks and environments and use fixed, static attacks. Meanwhile agents get high-privilege tools in exactly the environments utility benchmarks already model.

The approach: meta-benchmarking

MetaPI converts an existing utility benchmark into an end-to-end IPI suite that reuses its tasks, environments and infrastructure. The toolkit:

  1. Finds plausible injection points (tool outputs, files, web content the agent will read).

  2. Generates realistic attacker objectives for that environment.

  3. Optimizes adaptive payloads rather than using fixed strings.

  4. Validates each attack by running it in the real environment and checking whether the objective was achieved.

Interfaces are modular, so new environments, agents, defenses and attack algorithms plug in without rewriting the pipeline.

Findings (author-reported)

What to do with it

Caveats

Review checklist

  1. Which inputs to our agent come from places an attacker could write?

  2. Do we test injections inside our real task environment, not a toy one?

  3. Do we report utility and attack success together for each defense?

  4. Are high-privilege tools gated by something other than the model's judgment?

R086 (graders as an attack surface in Singularity-Bench), R074 (policy on every mutation, not tool hiding), R064, R052.

Connections

See R089 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.