Source: Stanford CS329A (Autumn 2025), taught by Aakanksha Chowdhery and Azalia Mirhoseini. Course site. Lecture playlist (Stanford Online, 9 parts, about 10.5 hours). Surfaced on LinkedIn by Mayank A., 2026-11-16.
What it is
A graduate seminar about agents that get better by interacting with themselves and their environment. It is a research-reading course: each session pairs a topic with three to five papers, and students run an original research project. It is not a build-your-own-harness course. Stanford Online posted nine core lectures; the guest lectures (Google DeepMind, LMCache, Reflection AI, Physical Intelligence) are listed on the schedule but are not in the playlist.
For engineers it is the theory half of the agent stack: why sampling more, verifying better and training on your own successes work, and where they stop working. It pairs well with an applied agents course such as CS329Z (R087).
The nine recorded lectures, as a dependency graph
Course overview. Self-improvement as a loop: generate, check, learn, repeat.
Test-time compute scaling. Repeated sampling (Large Language Monkeys), compute-optimal test-time scaling (Snell et al.), and inference-time architecture search (Archon). The core result: coverage keeps rising with samples, so the bottleneck moves to picking the right answer.
Robust verification. That picking problem. Outcome vs process reward models (Cobbe et al., Let's Verify Step by Step, Math-Shepherd) and combining weak verifiers to narrow the generation-verification gap.
Learning from feedback with tools and code. ReAct, RL from execution feedback (RLEF), and Constitutional AI as feedback from a written policy.
Planning and multi-step reasoning. Tree search over agent actions (LATS, adaptive branching search), as-needed decomposition (ADaPT), parallel plan execution (SPRINT), synthetic multi-step RL data (SWiRL).
Train-time scaling and RL. STaR (train on your own correct rationales), DeepSeekMath and GRPO, DAPO as an open RL system.
Self-improvement and deep-research agents. Open-ended evolution (automated agent design, The AI Scientist, AlphaEvolve) and search-augmented reasoning (AlphaCode 1 and 2, Search-o1).
Agentic evaluations and long-horizon tasks. Task-length measurement (METR's long-tasks work), GDPval, and live research-synthesis benchmarks.
Future research areas.
The schedule also covers software-engineering agents (CodeMonkeys, KernelBench, agent-system interfaces for program optimization) and memory (MemGPT, Cartridges, KV-cache reuse), mostly via guest sessions.
The through-line for practitioners
Generation is cheap; verification is the product. Lectures 2 and 3 together say that once you can sample many candidates, quality is set by your checker. In production terms: invest in tests, type checks, executable specs and graders before buying a bigger model.
Feedback must be grounded. Code execution, tool results and explicit rubrics (lecture 4) beat self-critique alone. Self-judgment without an external signal tends to reinforce its own errors.
Search trades latency for reliability. Tree search and branching (lecture 5) help on hard, checkable tasks, and cost a multiple of tokens and wall-clock. Use it where a verifier exists and a wrong answer is expensive.
Training on your own successes compounds, if the filter is honest. STaR-style loops and GRPO (lecture 6) only improve the model if the success signal is not gameable. Reward hacking is the central failure mode.
Measure horizon, not just accuracy. Lecture 8's framing, how long a task an agent can finish reliably, is the metric that tracks real usefulness.
Tradeoffs and limits of the material
Research-first. Little on cost control, latency budgets, durable execution, observability or rollout safety in production.
Snapshot of late 2025. The readings predate many 2026 harness and RL results; treat it as foundations, not current state of the art.
Recorded core only. The guest lectures, often the most applied, are not in the playlist. Slides and homework are not posted on the public site.
Math-heavy examples. Many verifier and RL results are on math and competitive programming, where correctness is cheap to check. Transfer to open-ended work is the hard, unsolved part, and the course says so.
Failure modes the course helps you recognise
Best-of-N with a weak picker: accuracy plateaus or falls as N grows because the verifier selects confident wrong answers.
Self-reward drift: the model learns to satisfy its own grader rather than the task.
Search blow-up: branching factor times depth exhausts budget before finding a verified answer.
Benchmark contamination and short horizons: good scores on short, leaked tasks that do not predict long-task success.
Review checklist (using the course to evaluate an agent design)
What is the verifier for each step, and how was its false-positive rate measured?
Is feedback external (execution, tests, tools) or only self-critique?
Where is test-time compute spent, and is there a stop rule tied to verifier confidence?
If the system trains on its own outputs, what prevents reward hacking, and who audits the filter?
Is success reported by task horizon and cost, not just pass rate?
Suggested viewing order for an engineering team
Lectures 2 and 3 first (the generation-verification argument), then 4 and 8 (grounded feedback and evaluation), then 5 and 6 if you train or fine-tune. Lecture 7 is the most speculative and can come last.
Related
R087 Stanford CS329Z: Engineering AI Agents (the applied companion course).
R095 Learn Harness Engineering (builder-checker loops in practice).
R097 MiniMind (hands-on GRPO and agentic RL at small scale).
R086 Agents doing AI engineering: four November 2026 benchmarks (long-horizon evaluation).
R094 CodeClimb (a self-improving coding agent with budgets and hidden tests).