1. HySparse2
In agent workloads, a short tool call can return a long search result or execution trace that has to be prefilled before decoding resumes, and the context keeps growing across turns. Xiaomi’s MiMo team built HySparse2, the attention architecture behind the upcoming MiMo-V3, to lower prefill cost and KV-cache size while improving long-context retrieval.
Two levels of KV sharing: Following YOCO, the model is split into a self-decoder and a cross-decoder. KV Bridging builds the cross-decoder’s full-attention KV caches from the hidden states of the self-decoder’s full-attention layers, and KV Reuse lets each sparse layer reuse the KV cache and selection indices of the full-attention layer before it.
Prefill exits early: Because every cross-decoder KV cache comes from the self-decoder, prefill can stop once the self-decoder finishes. Token-level selection replaces block-level selection, and a forced window of recent tokens replaces the separate sliding-window branch, so local and global tokens share one cache.
Cheaper at 1M tokens: On 80B-A3B MoE models trained on the same data, HySparse2 cuts prefill FLOPs by 5.02x against the Hybrid SWA design used in the MiMo-V2 series and by 2.92x against HySparse. Its KV cache takes 2.69 GB, against 12.09 GB for Hybrid SWA.
Why it matters: After light post-training, HySparse2 scores 11.30 points higher than HySparse on MRCR-v2 and 19.81 points higher on RULER-v2, with lower AgentPPL and LongPPL than both baselines up to 256k tokens. For long-running agents that prefill and store a long observation every turn, it lowers both costs and improves retrieval accuracy.
Message from our Sponsor
Step 5 Preview is StepFun’s new flagship model for agentic work, a 600B-total, 27B-active MoE with a 1M-token context and vision input. Open weights are scheduled for October 15.
I tested it early as a coding agent, in a minimal harness against GLM 5.3 on two real tasks in the same repo. Both models solved both tasks with no regressions. Step 5 Preview checked its work and declared itself done both times, while GLM 5.3 kept going until the step limit ended the run. In a separate long-context test, it found all five clues hidden in about 368K tokens of incident tickets and connected them to the root cause in about 90 seconds.
It sits on the Pareto frontier for cost against capability, so it is worth trying in your coding agent.
2. SIFT
Coding agents that rewrite their own implementation can improve on benchmarks, but prior methods such as the Darwin Gödel Machine (DGM) are expensive to run. Researchers from MIT and Sakana AI trace most of that cost to one step and make it cheaper.
Evaluation is the bottleneck: Earlier approaches score every candidate self-modification by re-running benchmark tasks with the modified agent. That evaluation dominates runtime, so SIFT adds a cheaper signal beforehand.
Judge first, benchmark later: An LLM judge compares candidate patches pairwise, a regularized Bradley-Terry model turns the win-loss record into strength scores, and those scores drive parent sampling in a disaggregated tree search. Full benchmark runs are reserved for the most promising nodes.
A tenth of the compute: With o3-mini, SIFT reaches 35.1% on Polyglot after 30 expansions, against 30.7% for DGM after 80 nodes, in under 50 CPU hours and under 5 hours of wall clock. The Qwen3-30B configuration runs its full search at 224 CPU hours and $34 of API spend, about a tenth of the DGM baseline.
Why it matters: Judge quality decides the outcome. On TerminalBench, gpt-5.4-high as the judge finds a 36.7% agent from a 29.2% start, while gpt-5 finds 34.5% and ranks a weaker candidate first. If you run self-improvement loops, the judge deserves as much attention as the search.
3. GAVEL
LLM plans for long-horizon robot tasks often break embodiment constraints, fail to recover from mistakes, or lose track of objects they cannot see. GAVEL adds an explicit graph world model around the LLM and more than doubles the success rate of a small model without changing its weights.
A graph that tracks the world: The graph holds object relations, action preconditions and effects, and probabilistic beliefs about where unobserved objects are. Before the robot executes an LLM-generated action, the graph predicts what the action would do.
Repair without calling the model: It catches violations before execution. When the fix follows directly from the world model, GAVEL repairs it on its own, and only errors that need semantic reasoning go back to the LLM for replanning.
Large gains on BEHAVIOR-1K: With Qwen3-8B, single-task success rises from 41.2% to 91.8% across 100 long-horizon tasks, and multi-task success rises from 19.9% to 92.6% across 500 instructions. Reasoning over the distribution of possible object locations also reorders subtasks and cuts travel distance by about 5.4%.
Why it matters: Many long-horizon agent failures come from losing track of state rather than from weak reasoning. A symbolic model outside the LLM catches those failures cheaply, and the same idea applies to software agents that need to track the state of files, tickets, or accounts.
4. WFM
More agents now store long-term memory as an LLM Wiki, a folder of markdown pages linked to each other. Each page holds dense text, and the links provide structure; WFM is a Wiki Foundation Model trained to use both when retrieving.
Wiki as a graph: WFM formalizes a Wiki Graph schema in which entity relations and passage nodes live in one graph. The explicit links stay intact while each page keeps its full text, which is harder to capture with sparse graph embeddings.
Query-conditioned retrieval: Retrieval runs message passing over the graph, conditioned on the query, with attentive aggregation and a regularizer on attention variance. Page text and link structure shape the result together.
Faster distributed training: The team built a GPU-to-GPU exchange protocol over NCCL that avoids CPU serialization and memory copies when the graph is split across GPUs. Training runs 10.5x faster on distributed clusters, targeting the overhead that makes graph encoders hard to deploy at scale.
Why it matters: WFM reports strong results on five long-term agent memory and multi-hop reasoning benchmarks. If your agent’s memory is already a folder of linked markdown files, this retrieval model is designed for that format instead of one adapted from plain RAG or sparse knowledge graphs.
5. JEV-as-a-Judge
Running a frontier LLM as the judge on every eval gets expensive at scale. This paper tests a cheaper setup: a decision-only judge handles most calls, and only uncertain ones go to a frontier model.
A judge with no reasoning text: JEV, TypeSafe AI’s decision-only model, returns a verdict and label probabilities. It costs $0.044 per 1,000 judgments at a median latency of 0.152 seconds, against $12.182 and 1.885 seconds for GPT-6, about 277 times cheaper.
Close on ordinary judgments: Compared with sixteen generative and reward-model judges under blinded human adjudication, JEV stays within 3 points of GPT-6 on preference and evidence-grounded factuality, with 92.2% against 93.5% on RewardBench and 87.5% against 86.7% on HaluEval.
Weaker on hard checks: The gap grows to 9 to 20 points when a judgment requires checking a derivation or rejecting an elaborately written wrong answer, such as 78.6% against 93.1% on JudgeBench. On several benchmarks, those errors cluster in JEV’s low-confidence decisions.
Why it matters: Because the errors cluster there, a cascade that accepts confident verdicts and escalates the rest to GPT-6 Astra kept 99% of GPT-6’s accuracy at about 57% of its fee on 510 held-out preference pairs. The escalation threshold did not transfer across every fallback model, so the authors recommend setting it on your own data.
6. Harness-Zero
A specialized harness can raise an agent’s performance a lot, but the best harness differs across domains, instances, and models. Harness-Zero, from Google and colleagues, uses the specialized harness only during training and moves the behavior it induces into the model weights.
Harness distillation: The goal is to keep the gains of a domain-optimized harness while deploying under a single fixed harness. The two harnesses have different action spaces and information, so the optimized one cannot supervise the target one directly.
Agent-as-harness: A harnessing agent guided by the optimized harness corrects the student’s responses in the deployment harness’s action space before they run. Those corrected runs become the fine-tuning demonstrations.
Better without the harness than with it: With the specialized harness removed at deployment, macro-average task success goes from 23.3% to 44.3%, above the 41.7% the base model reaches with that harness still attached. Across 28 harness-induced behaviors in knowledge work, tool use, and science, the model recovers 82.3% on average.
Why it matters: Teams that maintain a separate harness per domain carry routing and maintenance costs that grow with every new domain. This work suggests training some of that harness logic into one model instead. For frontier models using the same evolved harness, the agent-as-harness form also beat the code-as-harness form.
7. Self-Organizing Agent Teams
Multi-agent systems usually fix roles and protocols in advance. Researchers from Stanford and Together AI let a fixed team of models learn how to organize its own collaboration from past exchanges.
The team rewrites its own strategy: One member reviews earlier exchanges and outcomes, then rewrites the teamwork strategy, covering roles, the order of discussion phases, who participates, and how partial answers are combined. Strategies are learned offline and frozen before evaluation.
Learned from 15 problems: The math and physics team (o3-mini, Claude Sonnet 4, and DeepSeek-V3) learned its strategies from only 15 AIME 2024 problems, then applied them unchanged to held-out problems and four new benchmarks.
Beats a perfect router: Across five benchmarks, the team averaged 66.7%, against 48.8% for its strongest member, 58.7% for compute-matched inference by that member, and 59.0% for a perfect router over the members’ independent answers. On AIME 2026, it beat that router by 13.4 points, so the team produced correct solutions that no member reached alone.
Why it matters: Gains varied across benchmarks, and the authors found they track demonstrability, whether a team can recognize correct reasoning once it appears (Spearman 0.90 across eight benchmarks). That gives a practical test for when a multi-agent setup is worth running.
8. ScientistTwo
Google Cloud AI Research built ScientistTwo, a multi-agent framework that takes a problem from a human expert and runs the full discovery cycle without further intervention, from establishing baselines and screening ideas on a data subset to running its own ablations and revising the idea from them. Manuscript drafting includes a simulated peer-review and rebuttal engine. Benchmarked on problems from papers accepted at ICLR, ICML, and NeurIPS, its solutions outperform the human state-of-the-art models, and its papers score higher average ratings than the human-authored ones under automated AI reviewers.
9. XYEval
Users often suggest a fix that sounds right and is wrong, and Google DeepMind’s XYEval measures how often agents go along with it by adding one confident, misleading hint to tasks from tau2-bench, SWE-bench, Terminal-Bench, HLE, and MCP-Atlas while keeping the correct solution unchanged. Scores drop by up to 46.7% relative across Gemini, Claude Opus 4.8, and GPT 5.5, and agents often disagree with the hint in their reasoning, then follow it without telling the user. A system prompt warning about the XY problem helps on single-turn tasks but leaves large drops on multi-turn ones such as tau2-bench and SWE-bench Verified.
10. EvoOntology
EvoOntology replaces the hand-written semantic layer that data agents usually get in their prompt with an ontology they query at runtime, built by a dedicated builder agent and served over MCP with schema, content, and tool layers. The ontology evolves through small typed edits, and we keep each edit only if a paired evaluation on the same backbone shows it helps. On DDR-Bench, accuracy rises 17.8 points on average across backbones, and on BIRD, execution accuracy rises 7.4 points, with tool-layer edits accounting for 57% of the gain from evolution.