Source: NVIDIA NeMo Switchyard (github.com/NVIDIA-NeMo/Switchyard, Apache-2.0). Surfaced by André Lindenberg's LinkedIn post (2026-11-15).
What it is
Switchyard is an open-source library that decides, per request, which model in a pool an agent should use. Its Rust core, libsy, is meant to be embedded in a gateway or agent harness: you keep the model clients and credentials, Switchyard returns the routing decision. Integrations exist for Agent Router (Envoy), LiteLLM (experimental), NeMo Relay and OpenRouter (model = nvidia/switchyard). A standalone server and experimental macOS/Windows desktop and terminal apps exist for local evaluation only; the desktop app's default profile targets Codex CLI.
The core idea
The post's claim: a repeated failure is a better difficulty signal than the task description. Switchyard's Execution (Stage) router acts on that. For each LLM call it reads the conversation's tool-result history and scores two directions:
Toward the capable model: windowed error severity, spinning (deep churn with no reads or writes) and exploring (reading or planning without producing).
Toward the efficient model: production intensity (share of recent operations that are writes or edits), with extra weight for production with no recent errors.
The signed score is tanh-squashed into a confidence; the default threshold is 0.5. Repeated failures, critical errors and context compaction are hard overrides to the capable tier. After a switch to capable, capable_hold_turns (default 2) holds that tier for the next requests, so a switch plus hold is three requests on the capable model. Holding avoids flip-flopping between models, which would also throw away the prompt cache on each switch. Read and search tool contents do not count as errors unless the tool itself failed.
Other algorithms:
Task (LLM classifier): a model judges up front whether the efficient model can handle the request.
Composite: classifier sets the default tier when execution signals are ambiguous.
Escalation: the weak model answers, a judge rates the completed turn, and the session latches to the strong model after a configurable number of confirmed escalate verdicts with fresh evidence for the same failure category.
Advisor, sub-agent, random, and prefill routing (a classifier trained on a model's prefill activations).
Auto, the default, is a fixed preset (Stage, efficient-first, threshold 0.5, no classifier call); it does not compare strategies at runtime.
Published numbers (vendor-run)
| Benchmark | Strategy | Solved | Cost per run |
|---|---|---|---|
| Terminal-Bench 2.1 (Opus 4.8 + GLM 5.2) | Opus 4.8 only | 77.15% | $148.83 |
| GLM 5.2 only | 74.16% | $71.31 | |
| Stage | 79.03% | $109.81 | |
| LLM classifier | 79.40% | $109.04 | |
| Escalation | 79.80% | $122.63 | |
| DeepSWE v1.1, 113 tasks (GPT-5.6 Luna + Sol) | Sol only | 62.8% | $720.95 |
| Stage | 61.36% | $466.28 | |
| Luna only | 52.2% | about $49 |
On Terminal-Bench the routed pool beats the strong model alone at about 26% lower cost. On DeepSWE, Stage keeps most of Sol's score for about 35% less. These are vendor runs, mostly three-run means.
Why it matters for agent builders
Difficulty is easier to observe during a run than to predict before it. Tool errors, loops and test failures are free, exact signals that any harness already has.
Holds are a cost tool as much as a quality tool: every model switch is a cache miss, so routing must be sticky.
The README itself warns that a cheaper model call does not guarantee a cheaper solved task; evaluate the whole agent, pool and config against a single-model baseline.
Caveats
Benchmarks are vendor-run on two suites; no independent replication yet.
The built-in signal vocabulary is calibrated for coding agents and recognizes specific tool names and outputs; a press critique (Towards AI) argues it keys on hardcoded strings and ignores the prompt. That is the design, but it means non-coding agents or custom tools need tuning.
Router overhead and silent misconfiguration (the docs list settings that disable routing without an error) need checking.
The desktop app and standalone server are experimental and not supported for production.
The post's "keep your Claude login" and "3.5k stars" are the author's claims; the desktop app's documented default is Codex.
Related entries
R027 (provider tool-call error rates), R060 (planner and executor tiers in one family), R042 (routing a prover's compute on checker signals), R059 (memory and context cost).