TL;DR
Hybrid Reasoning Agents (Zou, Qi, Chen, Bianchi, Zhu, Lu, He and Zou; Stanford, Together AI and UIUC; arXiv 2610.18861, code; surfaced by Aishwarya Naresh Reganti) lets a tool-using agent switch, inside one generation, between ordinary text and "latent" steps where the model feeds its own hidden state back in without decoding a token. Text is kept for tool calls, answers and the steps worth showing; the rest of the thinking happens in latent space.
Qwen3-Base at 4B, 8B and 14B, trained with a hybrid SFT stage and then RL, evaluated on 11 math, science, medical, search and agentic benchmarks.
Reported versus matched text agents: 35.5% to 54.7% fewer tokens, 1.8× to 2.6× faster inference, 2.7× to 3.7× lower cost, and about 10% higher average accuracy after RL.
HybridAgent-14B averages 52.4 on GAIA and 49.5 on WebWalker, ahead of a text agent trained with ARPO on the same backbone (43.7 and 40.5).
Latent steps can be read back as approximate "pseudo-text", and a monitor that sees only the visible text predicts correctness about as well as it does for a text agent.
How it works
HybridLink. A small residual adapter (about 2·d² parameters, roughly 10M for a 4B model) maps the last-layer hidden state back into the input embedding space. Feeding raw hidden states back, as earlier latent-reasoning methods do, scores much worse in the paper's ablation (MATH500 71.4 vs 88.2).
Mode switching. In text mode the model samples tokens as usual until it emits a <latent> token. In latent mode each step's hidden state goes through HybridLink and back into the model with no decoding. The span ends when the adapter's output is nearest to the </latent> embedding. The model decides span lengths itself.
Working memory across turns. A hybrid trace can't be re-tokenized, so the agent keeps its KV cache (text, latent steps and tool results) across tool calls and resumes from it. The paper proves this matches a full-prefix forward pass. In practice the KV cache becomes the agent's state, and the authors had to extend their serving stack, since current engines don't support it.
Training.
Data. Existing text agent traces are converted to hybrid ones, with steps assigned to latent mode by LLM judges and the model's own token entropy. Tool-interaction and key steps stay in text. The SFT set is 54K trajectories.
HybridSFT. Format activation teaches the anchor tokens, then the adapter is trained on frozen-backbone hidden states, then a two-pass self-conditioned stage trains the backbone on the latents its own adapter produces. That last stage is what closes the gap to plain text SFT.
HybridRL. GRPO-style RL needs a token likelihood at every position, and latent steps have none. The authors show the policy factorizes over sampled text positions only, record latents during rollout and replay them exactly during the update. The reward is correctness, plus format, plus a bonus for using both Python and search. Mode usage is not rewarded; the model moves more work into latent mode as training goes on.
Evidence worth weighing
Latents carry the reasoning. Replacing latent spans with another problem's latents drops MATH500 from 84.6 to 44.8 (8B), and random-token replacement to 41.2. Forcing all-text reasoning costs 4 to 9 points.
Tool use survives. Valid tool-call rates are higher than a GRPO text agent, execution success matches (99.7%), and accuracy on questions that used a tool is 2 to 44 points better.
Auditability. Visible text is roughly half as long but denser in key steps. Weak-monitor AUROC is higher than for the text agent and strong-monitor AUROC is level. A judge still localizes the first error in 98% to 99% of failures.
Tradeoffs and failure modes
It needs the weights and the server. This is a post-training recipe for open models plus a custom inference stack that persists KV state across tool calls. It does not apply to API-only models, and it does not run on stock serving engines.
Hidden reasoning is still hidden. Pseudo-text readouts are a nearest-token approximation, not the computation. The monitor results are averages; a step that matters but stays latent is invisible to anyone reading the trace.
The KV cache is the session. You can't replay, edit, compact or hand off a hybrid context as text. Checkpointing, context compaction and moving a session between replicas all get harder, and the cache is large.
Baseline matching. Frontier agents are run through the authors' harness and tools, with a 10-call tool cap. The cleanest comparisons are the same-backbone GRPO and ARPO rows, where gains are smaller but consistent.
Training data comes from text traces. Which steps go latent is decided by judges and entropy heuristics, so the method inherits the quality and biases of the source traces.
Reward shaping matters. The multi-tool bonus moves GAIA by about 5 points; what generalizes beyond these benchmarks is not shown.
Implications
For teams that serve their own open-weight agents, latent spans are a credible way to cut tokens and latency without giving up tool use. Everyone else should take the auditability result as the useful signal: compressing visible reasoning did not make the trace less monitorable in these tests, so trace-length budgets need not cost oversight.
Confirm you control the weights and serving stack before considering it
Plan how sessions are checkpointed and resumed when state is a KV cache, not text
Keep tool calls, decisions and answers in text, and log the latent pseudo-text next to them
Re-run your own monitors on hybrid traces rather than relying on the paper's AUROC
Compare against a same-backbone text agent with RL, not against frontier agents in a different harness
Budget for memory: persistent KV state across turns is the main operational cost
Related entries
R025 LongCapHarness: the opposite direction for efficiency, moving work out of the model into code and explicit state.
R019 s3: another RL recipe that changes how agents spend tokens, there on messages between agents.
R020 The model proposes, the harness enforces: if more reasoning goes invisible, guarantees rely even more on what the harness checks.
Sources
Hybrid Reasoning Agents, Zou et al., arXiv, October 2026
togethercomputer/HybridAgent on GitHub
Aishwarya Naresh Reganti LinkedIn post, 2026-11-10