Source: Tang et al., AutoRL: A Multi-Agent System towards Autonomous Reinforcement Learning, arXiv 2611.17790 (submitted 2026-11-13). Highlighted by Elvis Saravia (DAIR.AI).
The idea
Most "AI does research" systems run complete experiments end to end and compare results. AutoRL instead copies how an RL researcher actually works: it watches a single ongoing training run and intervenes when something goes wrong, the way an on-call engineer watches a production service.
Architecture
Perceptors track training dynamics, rollouts, evaluations and efficiency, and trigger deliberation on anomalies, degradation or stagnation.
Three Proposers diagnose the problem from shared evidence and refine candidate interventions, cross-checking each other.
An Orchestrator reviews the recommendation and applies approved changes by online hot update or by resuming from a checkpoint.
The separation is the useful part: detection, diagnosis and the authority to change the run are different roles, and only one role can act.
Results (as reported)
Qwen3 on AIME 2024: 46.67% to 66.67%. AIME 2025 and HMMT 2025 each up 11.67 points.
Agentic coding with LongCat-Flash-Lite: pass@1 52.29% to 65.13% over a long unattended run that included interruptions.
Given a deliberately broken recipe, it found and fixed the problems.
Beats both static RL recipes and a general-purpose agent supervising the same run.
Tradeoffs and failure modes
Validation cost. The authors' own conclusion: routine supervision is within reach of current agents, but validating each change still takes substantial compute. That, not diagnosis, is the bottleneck.
Small eval sets. AIME-style benchmarks have few problems, so single-digit-problem swings produce large percentage moves. Treat headline gains with that in mind.
Goodhart risk. An agent optimizing a live run against the eval it watches can overfit to that eval; held-out checks matter.
Change control. Hot updates to a running job need an audit trail of what changed, when and why, or results become irreproducible.
Review checklist
Are detection, diagnosis and action separate roles, with one approval point?
Is every intervention logged with its evidence and reversible via checkpoint?
Is the signal the agent optimizes held out from the one used to report results?
What is the compute budget for validating a proposed change, and who stops a bad one?
Related entries
Other self-improvement entries under index subtopic 1.2, including R103 (EvoTrias) and R107 (PostTrainBench's cheating audits for agent-run post-training).