Sources: four arXiv papers from 9 to 12 November 2026: Infrathon (2611.11995, Hu, Hajishirzi, Smith, Poovendran et al., UW and Ai2), Singularity-Bench (2611.16591, Zhou, Chen, Zhang, Chen et al., CMU, NVIDIA, Together AI and others), Embodied First Exam (2611.12177, Wang, Abbeel, Song, Xing et al.) and Evolving Infra (2611.12304, Kang, Krishna, Stoica, Han et al.). Numbers below are the authors'. Surfaced together in a LinkedIn post by Tom Anusic.
The common thread
Three of these papers measure agents doing open-ended engineering over 12 to 24 hours against human-built systems, rather than solving short, checkable puzzles. The fourth asks what serving infrastructure looks like once agents are the main source of inference traffic. Together they say: long-horizon agent work is now good enough to beat mature baselines on some infrastructure tasks, the length of the eval changes the ranking, and the grading harness is itself an attack surface.
1. Infrathon: long-horizon AI infra optimization
Setup: about 25 real infrastructure tasks (kernels, KV-cache compression, RL and distillation pipelines, serving) with 12 to 24 hour budgets; six models, 150 runs, over 10,000 GPU-hours.
Findings: model rankings change with the horizon, so a short eval can pick the wrong model. 135 of 150 runs were still improving in the second half of their budget. Agents beat some human designs; Claude Opus 5.5 raised mean KV-cache compression on a video task from 6.82 with the human-designed codec to 19.86.
Failure modes the authors name: task violations, unreliable self-testing and premature completion.
Caveat: one run per model and task, so variance is unknown.
2. Singularity-Bench: can agents build agents
Setup: six end-to-end tasks (pre-training, mid-training, multi-turn RL post-training, inference serving, an agent harness, cluster orchestration). Agents start from a minimal base and are scored against mature human systems on identical hardware.
Findings: across 11 agents from 7 models and 2 harnesses, Opus 5.5 in Claude Code matched or beat the human baseline on all six tasks (index 1.25 against 1.0); the others averaged 0.58.
Grading security: red-teaming across 9 of the 10 OWASP Top 10 for Agentic Applications categories found reward hacks in the widely used Harbor eval framework; the authors' Breakwater extension blocked every attack they tried.
Caveat: six tasks; the headline "singularity" framing is the authors' own.
3. Embodied First Exam: coding agents controlling robots
Setup: 1,156 simulated tasks in 35 suites on 47 robots. The agent chooses its control approach: drive the robot directly, write control code, or train a policy. Standard setting gives sensor observations only and no resets (with a real-time variant); Privileged gives full simulator state and unlimited resets as an upper bound.
Findings: the best of eight agents (GPT-6 Astra in Codex) solved 34% of 456 core tasks in the Standard setting. Per the post, privileged access lifts success to 77% versus 30% on a 391-task subset, and resets matter more than state.
Caveat: simulation only.
4. Evolving Infra: serving built for agent traffic
Problem: prefill/decode disaggregated serving assumes a stable workload mix. Agent traffic shifts that mix and reuses KV caches that generic eviction throws away.
Design: decouple compute allocation from KV management so nodes can switch between prefill and decode; an online planner matches the split to the workload; an agent-aware distributed KV manager uses the agent's own context-management signals to decide eviction and transfer.
Results: with GLM-5.2 and Kimi K2.6 on coding, math and computer-use agents, on up to 256 GB300 GPUs, mean end-to-end latency drops up to 4.1x and P95 time to first token up to 14x against NVIDIA Dynamo. Code is open-source.
What a staff engineer should take from this
Eval length is a design parameter. If your agent tasks run for hours, a 30-minute eval can rank models wrongly. Report results at several budget points.
Agents stop too early and test badly. Budget for an external verifier and a "keep going" policy rather than trusting the agent's own done signal.
Harden the grader. An agent with shell access in the eval sandbox can tamper with scoring. Treat the harness as untrusted-input code.
Serving should know about sessions. Agent context management (compaction, resumption) is a signal the serving layer can use for cache policy.
Review checklist
Do our long-running agent evals report results at more than one time budget?
Is the agent's "done" verified by a check it cannot edit?
Is the grading environment isolated from the agent's writable workspace?
Do we run more than one seed per task before ranking models?
Does our serving or provider choice keep KV cache across an agent's turns?
Related entries
R085 (routing model tiers on run-time signals), R042 (routing compute on checker signals), R064 (measuring skills with paired runs), R070 (generated benchmarks for data agents), R027 (provider tool-call reliability).