TL;DR
The human-in-the-loop jobs of an agentic coding session (steer, approve, say no, check, start) have each moved to a specific layer: the model, the harness, or the environment. Lydia Hallie (Claude Code team, Anthropic) argues that the engineer's job is now to configure those layers well and review what comes out, not to supervise each step.
The load-bearing ideas for a staff engineer:
Guarantees live in the harness, not the prompt. Prompt rules and the auto-mode classifier are probabilistic (the classifier blocked 89% of clearly dangerous commands in Anthropic's study; humans caught about 14%). Deny rules, hooks and CI are deterministic.
Reversibility is the policy test. If Claude gets it wrong once, can you take it back? Reversible: leave it to the model. Irreversible (lost uncommitted work, a leaked key, a push, a sent message): make it a rule.
Permissions gate the call; sandboxes gate the side effects. A deny rule sees
npm test, not the dependency that opens.envduring the test run. Only OS-level sandboxing, or a disposable cloud machine with no secrets on it, covers that.Verification points beat end-of-run review. Same prompt, different runs, different plausible-looking outcomes; checkpoints that pass or fail catch divergence early. Hooks must exit 2 to actually block.
Orchestration is the next hand-off. One coordinator session (a judging model such as Fable 5.2) dispatches threads (a doing model such as Opus 5.5) and sends back work that isn't done.
Nate Herk's post and video are a faithful, condensed retelling of the talk with a few product details added. The few-sentence version of this doc: CLAUDE.md shrinks to what the code cannot tell the model, irreversibility drives the deny list, and every long run needs pass/fail checks it cannot talk its way past.
Sources and how they relate
The primary source is Lydia Hallie's 17-minute AI Engineer talk; Nate Herk's post and 10-minute video retell it with his own examples. Read the talk if you only read one.
| Source | Who | Published | What it adds |
|---|---|---|---|
| Rethinking Claude Code Best Practices (talk, 17:32) | Lydia Hallie, Claude Code team, via AI Engineer channel | about 2026-11-12 (re-recorded at home after on-stage technical problems) | The argument, the sandbox and cloud-session detail, the review tiers, the orchestration model |
| Anthropic Just Released the New Rules of Claude Code (video, 9:40) | Nate Herk, Uppit AI | 2026-11-17 | Clips from the talk, a knowledge-work framing, /auto-mode-setup, /goal, the Projects plan tiers; a gated free guide in his Skool community |
| LinkedIn post | Nate Herk | 2026-11-17 | The five-item summary; the video link is in his first comment |
Fidelity check. Nate's retelling matches the talk on every substantive claim. Differences worth knowing:
He adds
/auto-mode-setup(drafts auto-mode environment entries from your project and recent sessions) and the Projects plan tiers (public beta on Pro, Max and Ultra). Both are correct per the auto mode docs and Projects docs; neither is in the talk.He drops the most technical part of the talk: what the sandbox does not cover (Claude's file tools, MCP servers, hooks), network secrets in cloud sessions, CI autofix, merge-when-ready and the per-risk reviewer counts.
His auto-captions render "2,500 changes a day" as "200"; the post and the talk both say 2,500 (May) to 30,000+ (September), citing an October Anthropic blog post.
He calls the audit command "/doctor prompt audit"; the documented spelling is
/doctor prompt-audit(best practices).
One post comment (SJK Systems) adds a useful security footnote: approving a shell call that carries an inline token writes the literal key into .claude/settings.local.json, and it cites GitGuardian counts of 31,088 such secrets on GitHub and 613 in Docker images. Those numbers were not verified here.
Who does what now
Hallie's frame: everywhere the human steps out of the loop, something else must take over, and it is always one of three things: the model, the harness or the environment. The useful split for an engineer is which of those give guarantees.
Read it top to bottom as a session's lifecycle. The two amber jobs are where the remaining risk sits, so they are where deny rules, checks and acceptance criteria earn their keep.
The five jobs in depth
Each job below names what moved, the mechanism, and what the docs say about it.
1. Steering moves to the model
Early-2026 sessions on Opus 4.5 stopped early and needed constant correction, so CLAUDE.md files filled with IMPORTANT and NEVER. On Opus 5.5 and Fable 5.2 that emphasis over-steers: Hallie reports more tool calls and worse results.
The test for each CLAUDE.md line: could Claude figure this out by reading the code? Stack, layout and "where to look" fail the test; a legacy service that is live but not referenced in code passes it.
Tooling:
/doctor prompt-auditchecks CLAUDE.md and skills against the current model and flags instructions written for older models, dead file references and contradictions (best practices).Implication: instruction files are model-version-specific artifacts. They should be re-audited on each model upgrade, like a dependency bump.
2. Approving moves to auto mode
Auto mode (the default since August) sends nearly every action except reading and editing project files through a separate classifier. Tests and installs run; uncertain actions block.
Default trust boundary: the current folder and repo only. A push to a separate deploy repo blocks, which puts you back in the loop.
Configuration is prose:
autoModesettings take plain-English descriptions of trusted repos, buckets and domains plus always-allow and never-do rules; that text goes to the classifier./auto-mode-setupdrafts entries from the project and recent sessions (auto mode config).Measured performance: in Anthropic's August study, auto mode blocked 89% of clearly dangerous commands; about 14% of human approvers caught them. So roughly 1 in 10 still got through.
3. Saying no moves to deny rules and the environment
The classifier is a second gate. permissions.deny is evaluated first, and a matching call never reaches the classifier; neither the classifier nor user intent can override it (auto mode config).
Rule of thumb: if Claude gets it wrong once, can you take it back? Wrong code reverts. Discarded uncommitted changes, a key read into the transcript, a force push or a sent message do not.
The gap in permissions: a rule only sees the tool call.
npm testis allowed, but the test run executes dependency code and throwaway scripts; a compromised one can read.envor~/.sshand exfiltrate. A human approver would have clicked allow too, because there is nothing visible to refuse.Sandbox (
/sandbox): the OS enforces readable files, writable folders and reachable hosts for shell commands and all their child processes, so the.envdeny rule now also binds the dependency. By default it can still read most of the machine, so add rules for SSH keys. It covers shell commands only: Claude's file tools, hooks and local MCP servers run outside it (sandboxing).Cloud sessions: the strongest form. A fresh machine with a clone of the repo has no
.env, SSH keys or GitHub token. API keys become network secrets the model uses without seeing; network defaults to package registries and GitHub. The machine is disposable, but what leaves it (commits, connector messages) is not, so commit the deny rules into the repo so they travel.
4. Checking moves to tests, hooks and CI
Same prompt, different runs: migrating auth to a new API might leave the old API in place, migrate only new logins, or log everyone out. All three look reasonable at a glance.
Verification points: split the task into steps that each return pass or fail (a test, a build, a screenshot). Review those checkpoints rather than the whole diff. Ask Claude to show actual output, since you were not there to see it.
Hooks for what must run: lint on every edit, the touched area's tests on task completion, the full suite before the session may finish. Exit code 2 is a blocking error that sends Claude back; any other non-zero code is a non-blocking error, so the task completes anyway (hooks). This is the single most common hook bug.
CI as the uniform gate: every session ends in a PR, and CI runs the same checks however the session was set up. CI autofix lets Claude react to failures; merge-when-ready lets GitHub merge on green.
Anthropic's review policy: every PR is reviewed by Claude first, in a fresh context, with a separate group of agents verifying each bug it reports. Human review scales with risk: docs and copy need none once checks are green, ordinary product changes one reviewer, risky changes two.
5. Starting sessions moves to an orchestrator
The remaining human job is kicking off and tracking sessions, which Hallie calls bad for anyone who context-switches poorly.
Claude Code Projects: one coordinator conversation splits the work into threads, each a separate session, and tracks what comes back; it sends back work that is not done (Projects).
Model split: the coordinator needs judgment (deciding what to do, judging results), so Fable 5.2; threads need execution, so Opus 5.5. Both are set separately in Project settings (Coordinator model, Thread model).
Context management disappears: with Projects, compaction and clearing are handled for you.
Suggested first experiment (Hallie): pick one task far bigger than usual, make sure CI catches what you care about, configure the cloud environment (network access, secrets, setup script), run it in the cloud, and note where the session stalls or still needs you. That is the next thing to hand over.
Failure modes and gaps
The talk is a vendor's view of its own best case; these are the places it under-specifies or where teams will get hurt.
The 11% is the whole story for security. A classifier that misses 1 in 10 clearly dangerous commands is a convenience layer, not a control. Treat every irreversible action as needing a deny rule or an environment boundary, and assume adversarial inputs (prompt injection via issues, docs, dependencies) will aim at that 11%.
Deny rules are syntactic. They match tool calls, not effects.
Bash(cat .env)denied still leavespython -c,node -eor a test script that reads it. Only the sandbox or a secret-free machine closes that, and the sandbox does not cover hooks or local MCP servers, which run with your full user rights.Approval leaks secrets into config. Approving a command with an inline token can persist the literal token into
.claude/settings.local.json(per a post comment; plausible given how allow rules are recorded). Gitignore it, scan it, and never COPY the repo root into images without a.dockerignore.Hook exit codes fail open. Exit 1 is a non-blocking error; malformed JSON on exit 0 is also non-blocking. A policy hook that crashes lets the action through. Write policy hooks to exit 2 on any doubt, and test the failure path.
Prose policy drift. Auto-mode rules are natural language read by a model. They are hard to diff, test or reason about formally, and a one-off boundary stated in chat can be lost on compaction (the docs say so). Keep hard boundaries in
permissions.deny.Verification is only as good as the check. A run that logs everyone out passes if no test asserts sessions survive. The approach shifts review effort from diffs to test design; teams with weak tests get faster wrong code. Agents also learn to satisfy the check ("teaching to the test").
Claude reviewing Claude. Fresh context and separate verifier agents reduce, but do not remove, correlated blind spots from the same model family. Risk-tiered human review is the mitigation; the tiering itself (what counts as "docs or copy") needs an owner.
Orchestrator judgment is a new single point of failure. If the coordinator mis-splits work or accepts a thread's "done", every downstream thread inherits it. Its acceptance criteria should be the same pass/fail checks, not its own reading.
Cost. Dozens of parallel sessions, a coordinator on the top model and verifier agents multiply spend; the Projects docs warn that projects use plan limits faster. Neither source quantifies it.
Review checklist
Use this when auditing a team's Claude Code setup or a repo before handing it long unattended runs.
Instructions
Every CLAUDE.md line passes "could Claude learn this from the code?"; no shouting (
IMPORTANT,NEVER) left over from older models/doctor prompt-auditrun after the last model upgrade, and findings resolved
Permissions and environment
Every irreversible action has a
permissions.denyrule, committed to the repo so cloud sessions inherit it (force push,.envand key reads, destructive deletes, outbound messages)Auto-mode
environmententries list the trusted repos, buckets and domains, so the classifier isn't blocking routine workSandbox on for local sessions, with explicit read-deny for
~/.sshand other credential pathsHooks and local MCP servers reviewed as fully trusted code, since they run outside the sandbox
Long runs go to cloud sessions; secrets there are network secrets, network allowlist is minimal
.claude/settings.local.jsonis gitignored and secret-scanned; Docker builds use a.dockerignore
Verification
Tasks are split into steps with pass/fail checks, and the prompt says what done looks like
Hooks: lint on edit, scoped tests on task completion, full suite on stop, all exiting 2 on failure; the failure path itself tested
Tests assert the behaviour you'd hate to lose (sessions survive a migration), not just that code runs
CI runs on every agent PR with autofix on; merge-when-ready only for low-risk paths
Risk tiers for human review are written down (who decides what counts as docs-only)
Orchestration
Coordinator model chosen for judgment, thread model for execution, and the cost of that split measured
The coordinator accepts work on the checks passing, not on a thread's self-report
Related entries and sources
Related in this index
R004 The best Claude setup is no setup: same "config rots across model versions" thesis from the consumer side; this talk adds the test for what to keep.
R064 SkillEvaluator: the measured version of
/doctor prompt-audit, benchmarking each skill with and without it.R068 Plan on Fable, hand off to Opus: the same judge/doer model split, done by hand before Projects did it.
R058 AI-native SDLC: Anthropic's playbook vs AWS AI-DLC: Anthropic's broader delivery playbook; this talk is its per-session mechanics.
R132 Fold subagents into the main loop: a counterpoint on when one loop beats orchestration.
R100 Anthropic on multiagent failure modes: what can go wrong when a coordinator judges threads.
R112 Unprompted: agents that leak data with no attacker: why the sandbox and secret-free environments matter beyond deny rules.
R105 Sonar State of Code 2027: data on AI code outpacing verification, the problem section 4 addresses.
Sources
Lydia Hallie, Rethinking Claude Code Best Practices, AI Engineer, about 2026-11-12
Nate Herk, Anthropic Just Released the New Rules of Claude Code, 2026-11-17
Nate Herk, LinkedIn post, 2026-11-17, with comments
Claude Code docs: Best practices, Configure auto mode, Sandboxing, Hooks reference, Projects