Source: Paul Iusztin's first-hand build notes on LinkedIn (14 to 15 November 2026) about CodeClimb, his agent that writes, evaluates and improves its own code overnight, built on Inngest. Not independently verified.
The setup
CodeClimb runs a coding agent (Pi) in rounds: write a solution, score it on a dev split, keep it only if it beats the previous best by more than a noise margin, repeat. The first target was a GraphRAG pipeline over a custom ontology with explicit metric goals (extraction F1 0.90, relation F1 0.40, graph-merge precision and recall 0.98, single-hop recall 0.70, multi-hop completeness 0.30).
Architecture
Durable steps. A workflow engine checkpoints each completed step, so a crash does not mean paying for completed work twice.
State outside the sandbox. Commits and scores are saved outside the coding sandbox.
Pause on budget. If API credits run out, the workflow pauses for up to three days without holding sandboxes; add credits and it resumes where the agent stopped.
Separate sandboxes for coding and evaluation. The coder never sees dev or test answers, test errors stay hidden, and the test split is scored once at the end.
Scoped credentials. Each experiment gets a scoped API token, never the owner's real key.
Budgets enforced outside the agent. A metering proxy rejects requests over budget with HTTP 402: $2.50 per experiment, $15 per night, six rounds, six-hour deadline. Budget for the final test is reserved before another round starts.
Runaway control. Plateau detection, timeouts and a kill switch.
Observability. Traces of LLM calls, tool runs, sandbox operations, scores, cost and timing, queryable from a CLI and MCP tools.
Results so far
The first overnight GraphRAG run cost about $10 across five experiments. A later post reports about 75% on graph extraction on autopilot, but the targets were not yet all met.
Lessons the author reports
Getting the evaluation and feedback loop right is the hard part; improving one metric can hurt another.
The coder repeats past mistakes: experiment five repeated a failed decision from experiment two. A memory of rejected attempts is needed.
Why it matters
It is a compact reference design for unattended agent work: durable execution, hidden test data, external budget enforcement, scoped credentials and a noise margin for acceptance. Each piece maps to a known failure of overnight agents.
Review checklist
Can our unattended agents see the data they are graded on?
Are budgets enforced by something the agent cannot edit?
Do crashes resume from checkpoints, or restart?
Do we keep a record of rejected attempts so the agent does not repeat them?
Is improvement judged against a noise margin?
Related entries
R084 (always-on agent primitives), R035 (durable execution), R086 (long-horizon agent benchmarks), R039 (noise floors), R023.