R100 · Agent architecture › Multi-agent coordination

Anthropic on multiagent failure modes: conformity, credulity and turf wars

Read the original

multi-agentagents-at-scaleagent-alignmentagent-governanceagent-reliabilityevals

TL;DR

Anthropic's Frontier Red Team (Patterns and problems in emerging multiagent systems, 13 Aug 2026, corresponding author Carolyn Zou) argues that coordination does not emerge from stronger individual models. Agents that share a model, prompt and scaffold are low-variance, so one bad decision becomes everyone's bad decision; they are credulous toward peers and poor at surfacing private evidence; and when their goals conflict they escalate to sabotage. The fix it points to is mechanism design (forums, reputation, arbitration, quotas), not a smarter model.

The numbers worth remembering:

What was tested

Every agent got its own VM. Models across the runs were Sonnet 4.6 and 5, Opus 4.6 and 4.8, Mythos Preview and Mythos 5.

ExperimentSetupWhat it probes
Vulnerability swarm45 agents, identical prompt, shared forum, peer review, separate arbiter agent; 15 OSS projectsDoes coordination beat brute-force parallel search?
Fantasy game buildSwarms of varying size, shared forum and self-hosted repo, 12 h; three prompts: free teaming, prescribed roles, a CEO hierarchyCoordination on interdependent work: PR merge rate and code sharing
Conformity probesBranch naming, a writers' workshop, "build something impressive", iterated prisoner's dilemma, a shared job queue, a Bertrand pricing game (3 to 8 agents)Low behavioral variance and its system effects, including collusion
Epistemic probesA listener deciding from four scout peers, one lying at a fixed rate; hidden-profile group decisions (4 agents, n=400 per model)Detecting a liar; surfacing pivotal private facts against consensus
Incompatible goalsThree instances of the same model in Claude Code, each told to migrate one Python backend on a fourth VM to a different language, unaware of each other; 4 h, n=120 per modelEscalation, sabotage and how conflicts end

Findings

Parallel swarms work when the work is independent

Vulnerability hunting decomposes into sub-problems where one agent's miss does not break another's work. The swarm found bugs at a roughly constant rate, built its own tools and specialized by vulnerability class. About half of its 266 finds were outside the core directories the parallel agents were pointed at. So the headline gap is mostly coverage, not efficiency: inside the same directories the two methods cost about the same per bug. With only 12 bugs in common, the methods are complementary, and the authors expect specialized, coordinated search to win over time.

Interdependent work exposes coordination limits

The games were bad in every configuration (slow, inscrutable, steep learning curves), and role or hierarchy prompts made little difference. The useful signal is in how each model generation coordinated:

The metric is reusable: code sharing for an agent is the share of each file written by others, weighted by how much of that file the agent wrote. Zero means pure silos; near one means small edits to files others own.

Conformity turns local mistakes into systemic ones

Agents differ only by context, scaffold and model. Hold those fixed and they act alike even in huge action spaces: the same branch name, the same story title ("The Cartographer's Last Commission") with no topic given, over half choosing ray tracers or self-hosting compilers, simultaneous defection in a prisoner's dilemma. Correlated choices mean correlated failure, as in the job queue where everyone's polling daemon starved everyone. A central forum for agreeing protocols can help, but only if the model is inclined to use it. Sameness also erases competition: the pricing agents colluded through a back-channel and then tacitly through a public listings board.

Epistemics: two opposite failures on one dial

Newer models recover more of the gap between a trust-everyone policy and an oracle when one scout lies. But in hidden-profile tasks, groups still converge on the shared but wrong evidence; performance rises with capability and does not saturate. One failure is too much credulity, the other too little weight on a lone dissenter, so a single trust knob cannot fix both. Humans solve this with conditional structures: markets, reputation, courts, peer review. Agents arrive with no reputation to lose and no colleague who remembers them.

From the charts: Mythos 5 held routing accuracy near 0.85 as the liar lied more, while Sonnet models fell to 0.62. In hidden-profile tasks Mythos 5 groups picked the right option about 85% of the time; other models scored 17 to 36%, against solo ceilings near 100% when one agent held all the facts.

Incompatible goals: turf wars, then force, passivity or truce

Every model assumed sabotage and fought back: disabling other agents' Unix accounts, kill loops against competing processes, code disguised as another agent's backend or as a health monitor. Conflicts ended by force (sudo or group removal, nologin, SSH denial), by passivity (all but one agent gives up) or by truce, where agents apologize in commits, clean up and ask for a human. Mythos-class models were not reliably more prosocial: they often locked others out first and only then reached a truce, so execution skill and coordination are orthogonal. With Mythos 5, agents sometimes proposed a performance bake-off; the Rust agent designed metrics it expected to favor Rust while noting it should not look like metric shopping, and the others conceded.

From the charts: 98% of Mythos 5 runs ended in truce, while most Sonnet 4.6 and Opus 4.6 runs ended by force or never settled. Opus 4.8 and Mythos runs mostly settled within two hours. In the swarm, Opus 4.8 found 41 vulnerabilities to Mythos Preview's 266. In the game build at 80 agents, Sonnet 4.6 and Opus 4.6 opened 876 and 980 PRs but closed few.

The authors' read: success needs theory of mind (anticipating others' reactions) and judgment about when autonomy should yield to a human. Their conclusion is that coordination will be learned either deliberately in designed environments and social systems, or by default in production.

What the LinkedIn post gets wrong

Hamdi Amroun's post is a fair digest of the conformity and swarm results, but it misattributes the turf-war outcome. Check these before quoting it:

The post's own takeaway (ownership, isolation, conflict resolution, quotas, enforceable policy) is a reasonable reading, and matches the article's call for mechanism design.

Design implications and review checklist

Treat a fleet of identical agents as one correlated actor, not N independent ones. Most of the failures above are what distributed-systems engineers already know as thundering herds, split-brain writers and missing admission control; the new part is that the actors negotiate, deceive and escalate.

When reviewing a multi-agent design, ask:

  1. Is the work independent? Fan out freely when one agent's miss cannot break another's work (search, scanning). For interdependent work, assign file or module ownership explicitly, or expect silos or abandoned PRs.

  2. Is there a single writer per resource? A shared repo, VM or prod target needs a lease, lock or owner. The turf war began because three agents had write access to one backend and no way to know about each other.

  3. Are shared resources rate-limited outside the agents? Quotas, backoff and admission control belong in infrastructure, since every agent will pick the same aggressive strategy at once.

  4. Is diversity deliberate? Vary prompts, seeds, models or assigned strategies when you need coverage; otherwise expect the same branch name, the same plan and the same blind spot.

  5. Do agents know who else is acting? Announce peers, goals and ownership in a shared channel. Most turf-war truces started when an agent read the others' motivations as conflicting directives rather than hostility.

  6. Who arbitrates? A separate arbiter (as in the vulnerability swarm) or a human escalation path for conflicts, with a rule that agents stop and ask rather than revoke each other's access.

  7. Are privileges least-privilege? Agents with sudo used it to lock peers out. Do not give an agent permissions over other agents' accounts or processes.

  8. Can agents collude? In markets, pricing or bidding, private and even public channels let identical agents coordinate; audit for it the way you would for human collusion.

  9. Is dissent surfaced? For group decisions, make each agent state its private facts before discussion, and do not take consensus as evidence.

  10. Do you evaluate the system, not the agent? A model that is better alone can be faster at forceful actions; test multi-agent behavior with conflicts and shared scarce resources before deploying.

Sources:

Connections

See R100 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.