R098 · AI education › Learning paths for AI engineers

Stanford CS329A: Self-Improving AI Agents, a free lecture map

Read the original

educationlearning-resourcesreinforcement-learningevaluationtest-time-computeteaching

Source: Stanford CS329A (Autumn 2025), taught by Aakanksha Chowdhery and Azalia Mirhoseini. Course site. Lecture playlist (Stanford Online, 9 parts, about 10.5 hours). Surfaced on LinkedIn by Mayank A., 2026-11-16.

What it is

A graduate seminar about agents that get better by interacting with themselves and their environment. It is a research-reading course: each session pairs a topic with three to five papers, and students run an original research project. It is not a build-your-own-harness course. Stanford Online posted nine core lectures; the guest lectures (Google DeepMind, LMCache, Reflection AI, Physical Intelligence) are listed on the schedule but are not in the playlist.

For engineers it is the theory half of the agent stack: why sampling more, verifying better and training on your own successes work, and where they stop working. It pairs well with an applied agents course such as CS329Z (R087).

The nine recorded lectures, as a dependency graph

  1. Course overview. Self-improvement as a loop: generate, check, learn, repeat.

  2. Test-time compute scaling. Repeated sampling (Large Language Monkeys), compute-optimal test-time scaling (Snell et al.), and inference-time architecture search (Archon). The core result: coverage keeps rising with samples, so the bottleneck moves to picking the right answer.

  3. Robust verification. That picking problem. Outcome vs process reward models (Cobbe et al., Let's Verify Step by Step, Math-Shepherd) and combining weak verifiers to narrow the generation-verification gap.

  4. Learning from feedback with tools and code. ReAct, RL from execution feedback (RLEF), and Constitutional AI as feedback from a written policy.

  5. Planning and multi-step reasoning. Tree search over agent actions (LATS, adaptive branching search), as-needed decomposition (ADaPT), parallel plan execution (SPRINT), synthetic multi-step RL data (SWiRL).

  6. Train-time scaling and RL. STaR (train on your own correct rationales), DeepSeekMath and GRPO, DAPO as an open RL system.

  7. Self-improvement and deep-research agents. Open-ended evolution (automated agent design, The AI Scientist, AlphaEvolve) and search-augmented reasoning (AlphaCode 1 and 2, Search-o1).

  8. Agentic evaluations and long-horizon tasks. Task-length measurement (METR's long-tasks work), GDPval, and live research-synthesis benchmarks.

  9. Future research areas.

The schedule also covers software-engineering agents (CodeMonkeys, KernelBench, agent-system interfaces for program optimization) and memory (MemGPT, Cartridges, KV-cache reuse), mostly via guest sessions.

The through-line for practitioners

Tradeoffs and limits of the material

Failure modes the course helps you recognise

Review checklist (using the course to evaluate an agent design)

Suggested viewing order for an engineering team

Lectures 2 and 3 first (the generation-verification argument), then 4 and 8 (grounded feedback and evaluation), then 5 and 6 if you train or fine-tune. Lecture 7 is the most speculative and can come last.

Connections

See R098 on the knowledge map

Part of the Research notes on nowucca.com. Figures and claims are the source's own unless the summary says otherwise.