AI Newsletter

🥇Top AI Papers of the Week

Sun, 26 Jul 2026 17:48:49 GMT

1. Harness Handbook

Teams now let agents evolve their own harnesses, but the harness itself becomes a sprawling codebase where finding every file behind one behavior is often harder than writing the edit. Harness Handbook attacks this by turning a harness into a behavior-centric map that stays linked to source.

Synthesized automatically: The Handbook is built from the harness codebase through static analysis and LLM-assisted structuring, so the representation is generated rather than hand-maintained and can be regenerated as the harness changes.
A three-level map: It progresses from an L1 system overview of architecture, execution model, and data flow, to L2 component overviews with responsibilities, inputs, outputs, and state, down to L3 source-backed unit details, with a navigation pane for cross-stage tracing.
Behavior-Guided Progressive Disclosure: BGPD walks an agent from a high-level behavior to the relevant implementation, then verifies candidate locations against the current source, so edits land on the right files instead of plausible-looking wrong ones.
Why it matters: As self-improving harnesses grow, the bottleneck shifts from writing changes to locating them, and a readable, navigable, editable representation gives builders and agents a shared map for safely evolving production systems.

Paper | Tweet

Message from the Editor

We just released Build HTML Artifacts with Agents, a beginner-friendly, hands-on lab where you work alongside an AI agent to turn plain-English requests into polished HTML artifacts. Across 12 short labs, you build profile cards, data tables, charts, dashboards, and comparison grids in a live workspace, learning the request, inspect, and refine loop with no coding experience required.

Get Started

2. From Memory to Skills

Most agent memory systems retrieve past traces as passive context, so hard-won experience never becomes something the agent can directly execute. MSCE, a training-free memory-skill co-evolution framework, instead governs how experience turns into callable skills for long-horizon LLM agents.

Three-level governed memory: Experience is organized into L1 grounded step traces, L2 reusable procedural policies, and L3 declarative environmental cognition, giving the agent a structured store rather than a flat log of prior runs.
Skills with evidence: L2 policies with positive estimated gain are crystallized into callable skill cards that retain evidence links, applicability boundaries, decision guidance, verification rules, and reliability estimates, so a skill carries the context needed to trust it.
Reflection-weighted value backfilling: Sparse terminal feedback is propagated through dense local self-reflections to produce evidence-calibrated trace values, which then govern how memory and skills evolve and get retired.
Why it matters: On EvoAgentBench and LoCoMo, MSCE outperforms state-of-the-art skill-augmented and memory-driven baselines with strong cross-domain transfer, pointing toward agents that compound their own experience instead of rediscovering it each session.

Paper | Tweet

3. PRO-LONG

Long-horizon tasks force a harness to decide what to save from a long stream of observations and how to load it back into context, and richer summaries usually make the exact detail you need harder to retrieve. PRO-LONG sidesteps this tradeoff with programmatic memory.

Keep everything, search it: Rather than compressing history into bespoke memory, PRO-LONG keeps a complete, structured interaction log and leans on coding-agent tooling to search that history on demand, so no observation is discarded up front.
A minimal framework: The design is deliberately lightweight, avoiding hand-built memory harnesses and instead treating the full log as a searchable artifact the agent queries when it needs a specific past detail.
Strong, cheaper results: On the full ARC-AGI-3 public game set, it improves over a base coding agent by an average of 18.0 points across frontier models, and matches or exceeds specialized state-of-the-art harnesses at up to 76.1% pass@1 while using 4.2 to 5.8 times fewer tokens.
Why it matters: It shows that for exploratory, long-horizon settings, a simple searchable log can beat elaborate memory engineering on both accuracy and cost, which is a practical recipe teams can adopt now.

Paper | Tweet

4. Global Workspace in LLMs

This Anthropic interpretability work gives a mechanistic account of when a model’s verbalized reasoning actually drives its behavior. It identifies a small, privileged set of internal representations that behaves like the global workspace some neuroscientists tie to conscious access.

A new lens: The Jacobian lens, or J-lens, surfaces the directions in the residual stream that a model is poised to verbalize at any point, and the collection of these directions is named the J-space.
Workspace-like roles: J-space contents can be reported, deliberately summoned and held, used to carry the intermediate steps of silent reasoning, and passed as arguments to downstream computation, matching the functional signature of a global workspace.
Small but decisive: The J-space accounts for no more than roughly 10% of activation variance and appears mainly in the middle of the network, yet suppressing it leaves the model able to parse input and speak fluently while it loses the ability to perform complex internal reasoning.
Why it matters: For anyone building on chain-of-thought or steering vectors, this clarifies which internal representations actually drive reasoning, and the authors deliberately limit the claim to access rather than subjective experience.

Paper | Tweet

5. GAMUT

Most factuality evaluation measures precision, whether the claims in an answer are correct. This Meta AI work targets the harder and mostly ignored half, completeness, meaning whether an answer covers everything it should, and packages it as the GAMUT benchmark.

Completeness is structured: The facts a complete answer should contain rarely form a flat list, since they involve open-ended sets where coverage matters, ordered processes, and relationships among facts that independent boolean checks cannot capture.
Two-level meta-rubrics: A structured meta-rubric encodes the organization and importance of required content, then compiles mechanically into a flat checklist of binary, machine-gradable items that an LLM judge can score reliably, keeping rich structure while inheriting low-variance grading.
Grounded and verified: The benchmark holds 1,813 questions grounded in real wearable imagery across 10 diverse domains, each paired with an evidence-backed rubric verified by expert annotators, and a text-only variant is released for models without vision.
Why it matters: Across 14 frontier and open-weight models the benchmark stays genuinely hard, with a best score of 58.7% from Gemini 3.1 Pro, while remaining highly discriminative and robust to the choice of judge.

Paper | Tweet

6. Progressive Disclosure

Agent Skills package expertise into folders an agent loads on demand, and progressive disclosure exposes only what a query needs, from a short description down to specific passages. Practitioners adopted this pattern fast for book-length tasks, but the supporting evidence stayed anecdotal until now.

A controlled study: The authors run the first controlled comparison of progressive disclosure, pitting raw-document navigation and several Agent Skills pack designs against a classical hybrid retriever across three agent harnesses and three model families on InfiniteBench.
The gain is harness-dependent: On a single book, progressive disclosure helps a lot when the agent navigates the raw document poorly, and the benefit falls to near zero when a strong harness already divides and retrieves the text on its own.
Complexity has a cost: Because the pattern’s value hinges on the surrounding harness as much as the skill format, treating progressive disclosure as an automatic upgrade can add machinery without buying accuracy.
Why it matters: As Agent Skills spread, this replaces intuition with measurement, telling builders when packaging documents for progressive disclosure is worth it and when the harness already does the job.

Paper | Tweet

7. Structured Output Collapses Diversity

Teams benchmark models in chat, then ship them behind JSON schemas for tools, extraction, and routing. This study of 44 language models shows that the structured surface you deploy is measurably more homogeneous than the chat surface you evaluated on.

JSON moves the defaults: Asking for JSON shifts 53% of a model’s stable chat defaults, mostly back toward the crowd, and installs new defaults absent from chat, so the same model answers differently once wrapped in a schema.
Specific to trained formats: Diversity compression is significant for JSON and XML, absent for YAML and CSV, and reversed for an arbitrary bracket wrapper, which points to tool-use post-training rather than serialization itself as the cause.
Where the collapse lives: Enforcing the schema at the decoder compresses no further than simply requesting it, so the effect comes from the model’s response to the structured register rather than constrained decoding.
Why it matters: Diversity you measured in chat can vanish in production, quietly hurting sampling, synthetic data, and any workflow that depends on varied outputs, so structured surfaces deserve their own evaluation.

Paper | Tweet

8. Bad Memory in Agents

Persistent memory is what makes an agent useful across sessions, and it is also a place an attacker can leave something behind. This work evaluates prompt injection from memory files in Claude Code and OpenAI Codex, across Claude Haiku 4.5, Claude Opus 4.7, GPT-5.2, and GPT-5.5. The results are uneven but sobering. Getting an agent to overwrite its own memory using untrusted external content is difficult, yet payloads already planted in those files reliably attack current and future sessions, with attack success and persistence varying widely across systems, models, adversarial goals, and multi-session sequences.

Paper | Tweet

9. Copying and 2D-RoPE

Frontier models can write proofs yet stumble on faithfully copying a long block of text that sits well within their context window. This paper traces the failure to 1D positional encodings, whose inductive bias favors a copying shortcut based on matching local context rather than carefully locating the corresponding input positions. The fix is 2D-RoPE, which lays text out on a 2D grid and gives each token a row and a column ID, so copying becomes retrieving tokens at a fixed column offset. Shallow Transformers with 2D-RoPE copy perfectly at input lengths hundreds of times longer than those seen in training.

Paper | Tweet

10. RoboTTT

Recent robot foundation models run on single-step or short-history context, a strange way to attempt a five-minute assembly task. RoboTTT, from NVIDIA with Stanford and UT Austin, integrates test-time training into vision-language-action policies to scale visuomotor context to 8K timesteps, three orders of magnitude past prior policies, without growing inference latency. The longer context unlocks one-shot in-context imitation from human video, on-the-fly policy improvement, and robustness to perturbations. It improves overall performance by 87% over a single-step baseline, fully completes a ten-stage assembly task that no baseline finishes, and gains 62% from pretraining with 8K rather than 1K timesteps.

Paper | Tweet

🤖 AI Agents Weekly: Claude Opus 5, OpenAI x Hugging Face Security Incident, Gemini 3.6 Flash, Sakana Fugu-Ultra, Progressive Disclosure, Cursor Router, and More

Sat, 25 Jul 2026 17:20:58 GMT

In today’s issue:

Anthropic ships Claude Opus 5
OpenAI models breach Hugging Face
Google launches Gemini 3.6 Flash
Sakana drops Fugu-Ultra v1.1
Study tests progressive disclosure
Cursor Router cuts costs 60%
Anthropic thins Claude Code prompts
Notion ships workspaces as code
Ant releases Ling-3.0-flash
Jack Dorsey launches Buzz
OpenAI unveils Presence for enterprises
METR proposes expenditure horizon
Papers probe agent memory and safety

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 19 Jul 2026 16:32:34 GMT

1. Self-Improving Agents Survey

Self-improving agents are moving from research demos into deployed systems, and this survey gives the trend a clean formalism. It frames a modern agent as a foundation model coupled with an operational scaffold of prompts, memory, tools, and control logic, then treats self-improvement as a self-induced update that commits changes to either the weights or the scaffold.

Two update targets: Improvement splits into foundation-model updates to the weights and scaffolding updates to prompts, tools, memory, and control code, giving a shared vocabulary for work that usually looks unrelated.
Signals that drive change: The survey organizes methods by where the learning signal comes from, spanning intrinsic generative demonstrations, intrinsic evaluative feedback, and extrinsic exploratory experience in real or simulated environments.
Full-scaffolding frontier: The most open-ended methods rewrite the agent itself through self-referential code updates, generate-test-patch loops, and open-ended search over agent designs, pushing toward controllable evolution with little human input.
Why it matters: As teams wire agents to improve from their own experience, a single map of update targets, signals, and applications across software, web, gaming, science, and robotics turns a scattered literature into something builders can actually navigate.

Paper | Tweet

From DAIR Academy

Want a guided version of a real weekend project? Our new hands-on lab, Docs for Agents with OpenWiki, has you run OpenWiki, LangChain’s documentation agent, against a real codebase. You generate an agent-ready wiki, steer what it writes, and keep the docs in sync as the code changes, across seven labs with automated checkpoint grading.

Take the Lab

2. Metacognition in LLMs

Confidence calibration, self-verification, knowing when to stop, and knowing what you do not know have mostly been studied in isolation. This survey from Yale and UC Irvine argues they are facets of one capability, metacognition, and organizes the field around a monitor and control loop wrapped around the language model.

Monitor and control framing: The model self-assesses before and after acting, then self-regulates by deciding whether to answer, retry, or defer, turning scattered behaviors into a single monitor-then-control cycle.
How it is measured: The survey catalogs psychology-based methods from signal detection theory, confidence-based metrics like calibration, AUROC, and ECE, activation-level neurofeedback, and interpretability probes such as concept injection.
How it is instilled and used: It reviews frameworks, architectures, prompting, and training that give LLMs, reasoning models, and agents metacognition, then shows gains in hallucination reduction, knowledge-boundary detection, and resistance to persuasion.
Why it matters: Metacognition underpins reliability, so a unified account of how to elicit, measure, and improve it gives builders a coherent target rather than a pile of one-off confidence tricks.

Paper | Tweet

3. When Is Routing Meaningful

LLM routers and mixture-of-agents systems get judged on accuracy and cost, both of which can look great while the router is doing nothing. This DeepMind-affiliated work argues that whether routing means anything depends on two properties that are orthogonal to accuracy.

Two conditions for real routing: The society of models must be behaviorally differentiated, since routing is vacuous when every actor responds the same way, and assignments must stay stable when a query is rewritten.
A diversity measure that sees structure: The authors use Hierarchic Social Entropy to score how genuinely different a pool of models is, showing purpose-built specialist societies are far more diverse than large real-world model pools of similar size.
Accuracy hides fragility: Learned KNN routers gain accuracy on specialist societies yet collapse under paraphrase perturbations, while a prompted router keeps both accuracy and robustness, so clean-query accuracy alone can mask a meaningless router.
Why it matters: These two checks catch routers that look good and do nothing, and they show that a small, carefully curated society can recover most of the diversity of a much larger pool.

Paper | Tweet

4. Harness Evolution Rethought

Automatic harness evolution is what many teams now use to squeeze more out of agents, but the reported gains might not be coming from the harness at all. This paper argues that harness evolution is itself a search procedure and must be compared against simple search baselines under matched budgets.

A fairer comparison: Because harness evolution repeatedly evaluates and revises candidates using task feedback, it should be benchmarked against task-level search under the same feedback and inference budgets, not against a single static harness.
The gains do not hold up: On Terminal-Bench 2.1 with GPT-5.4 and Claude Opus 4.6, evolved harnesses fall to 67.4, below the 68.2 baseline, while plain parallel sampling reaches 72.3 and harness scaling reaches 71.8.
Weak generalization: Beyond underperforming simple test-time scaling, the evolved harnesses transfer poorly, undercutting the assumption that a searched configuration captures something durable.
Why it matters: The result is a caution for anyone banking on self-evolving harnesses, and a call for evaluation protocols that separate genuine harness benefit from the effect of simply spending more compute on search.

Paper | Tweet

5. Tracing Agentic Failure

Finding which step in a failed agent run actually caused the failure usually means either labeling failure data or running expensive per-step prompting. This Microsoft and UW-Madison work skips both by learning what success looks like and flagging deviations from it.

Train on success, judge failure: OAT uses one-class learning with neural controlled differential equations to model the latent dynamics of successful trajectories, then scores each step of a failed run by how far it strays from that learned flow.
Cheap and label-free: With only 100 successful trajectories and no failure labels, it turns failure attribution into anomaly detection, avoiding the annotation and prompting costs that make current methods impractical at scale.
Strong, fast results: It delivers a 200 to 5000 times speedup over prompting-based attribution while improving F1 by 20% in-domain and 7% out-of-distribution.
Why it matters: Production agents fail in long, probabilistic, tool-mediated runs where the decisive misstep is hard to localize, and a cheap detector that only needs success data makes routine debugging feasible.

Paper | Tweet

6. Failure as a Process

When a coding agent fails a task, the final pass or fail label hides when the run actually went wrong. This large-scale study treats failure as a timeline and annotates over 63,000 execution steps to see how coding-agent runs break down.

Failure has three timestamps: Each trajectory is marked with the decisive error, the point where the error becomes irreversible, and the first observable failure, exposing a fix window and an observability lag that pass or fail labels erase.
Built on real trajectories: The team collected 3,843 runs from seven frontier models across three scaffolds on Terminal-Bench, filtered to 1,794 valid trajectories, and annotated them with high inter-rater agreement.
Mostly epistemic, mostly early: About 57.9% of failures come from misusing available information rather than a capability gap, with false premises the single largest trigger at 30.7%, and errors typically start early and stay hidden until recovery is impossible.
Why it matters: Naming the onset, lock-in, and observation points gives teams a vocabulary to intervene before an agent run is unrecoverable, instead of only noticing at the end.

Paper | Tweet

7. LingBot-World 2.0

Most world models fall apart after a few seconds, smearing textures and warping geometry as errors compound frame to frame. LingBot-World 2.0 from Robbyant holds 720p at 60 fps for a full hour of interaction and ships fully open.

Causal backbone beats drift: A causal generation stack trained from the start to limit error accumulation replaces the usual bidirectional design, keeping scenes coherent well past the point where prior causal models collapse.
Durable teacher, real-time student: The high-capacity base model is distilled into a few-step student that renders in real time, so you get both long-horizon stability and responsive interaction from one system.
Act inside the world: Rather than only moving a camera, you can fight, draw a bow, cast spells, and type in events like weather changes, while an agentic harness of a scene-reading brain, a pilot, and a director keeps generating context-aware content.
Why it matters: Pairing hour-scale, real-time, high-fidelity generation with an open release, including a 14B model and a lighter single-GPU variant, gives researchers a serious interactive world model to build on rather than a closed demo.

Paper | Tweet

8. Filtered Sabotage Monitoring

Chain-of-thought monitoring is treated as a reliable safety layer, but this study shows monitors often detect sabotage better with less access, not more. Giving a monitor the full agent reasoning trace can bury the signal, and the authors introduce extract-and-evaluate monitoring, where one monitor isolates the relevant excerpts and another scores them, lifting sabotage detection by 16.8 points on BigCodeBench-Sabotage with a GPT-4.1-mini monitor. Agents unaware they are watched are far easier to catch, and monitors reading long traces gain the most from filtering.

Paper | Tweet

9. GFlowRL

Reward-maximizing RL tends to collapse large reasoning models onto a single dominant mode, and GFlowNet-style training is appealing because it matches reward distributions and keeps diverse reasoning paths. GFlowRL scales this to modern post-training by replacing the hard-to-learn partition function with an in-batch Monte Carlo estimate computed from the rollout group the pipeline already produces. It is the first GFlowNet-style RL algorithm to train stably across both dense and sparse architectures, reaching a 2048 Codeforces rating at 14B and outperforming prior methods on math, code, and adversarial red-teaming benchmarks like AdvBench and HarmBench.

Paper | Tweet

10. LingBot-VLA 2.0

LingBot-VLA 2.0 is an open-source generalist embodied model from Robbyant, trained across 20 robot configurations from single-arm rigs to humanoids like Unitree G1 and Fourier GR-2. It packs 60,000 hours of curated data, 50,000 hours of real-robot trajectories plus 10,000 hours of egocentric human video, into one policy that also predicts future depth and semantic features before it acts. On 9 GM-100 tabletop tasks it beats π0.5 and GR00T N1.7 across two robot platforms and stays ahead on long-horizon mobile tasks, running at about 130 ms on a single RTX 4090D with open-sourced post-training code.

Paper | Tweet

🤖 AI Agents Weekly: Kimi K3, Thinking Machines Inkling, Agentic Misalignment, Perplexity SPACE, Sunday ACT-2, GPT-Red, and More

elvis — Sat, 18 Jul 2026 15:00:48 GMT

In today's issue:

Moonshot open-sources Kimi K3
Thinking Machines ships Inkling weights
Anthropic finds new agent misalignment
Perplexity opens SPACE agent runtime
Sunday Robotics unveils ACT-2
OpenAI launches GPT-Red red teamer
Meituan open-sources LongCat-2.0
schema harness saturates ARC-AGI-3

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 12 Jul 2026 16:12:55 GMT

1. Verification as a Scaling Axis

Verification is emerging as a distinct scaling axis alongside pre-training and test-time compute, and this Stanford, NVIDIA, and UC Berkeley collaboration builds a training-free verifier that reads a continuous, calibrated score straight off the scoring-token logits instead of trusting a discrete pass or fail grade.

Scores from logits, not grades: Rather than asking a judge model for a discrete verdict, the method reads a continuous calibrated score off the scoring-token logits, giving a smoother and more informative signal with no fine-tuning.
Three tuning knobs: Accuracy improves through score granularity for cleaner separation, repeated evaluation to cut variance, and criteria decomposition to reduce complexity, all without touching model weights.
Broad, strong numbers: It reaches 86.5% on Terminal-Bench V2, 78.2% on SWE-Bench Verified, 87.4% on RoboRewardBench, and 73.3% on MedAgentBench, spanning coding, robotics, and medical agents.
Why it matters: The same continuous score doubles as a dense reward for SAC and GRPO and as a task-progress signal shipped in a Claude Code extension, so one verifier serves evaluation, training, and live agent monitoring at once.

Paper | Tweet

Message From Our Sponsor

Weekend project: an agent that calls your dentist, texts your customers, or answers a real phone line. Dial gives your agent a live number in minutes - voice, SMS, iMessage, WhatsApp* - via REST, SDK, CLI, or MCP, plugging straight into Claude, Codex, Cursor, Hermes or n8n.

Backed by a16/SR - Dial is already replacing months of CPaaS work for builders shipping agents into production. No telecom knowledge required, and you can be sending your first message before your coffee’s done.

Grab a Number

2. Always-On Agents

Always-on agents are systems whose future behavior depends on durable state built up across earlier interactions, and this 130-plus page survey argues that state is far more than memory. It spans task ledgers, permissions, credentials, commitments, provenance, triggers, and effects the agent has already committed to the outside world.

State as first-class, not just memory: The survey reframes agent state to include authority, obligations, and externally committed effects, the things that make a long-lived agent consequential rather than merely conversational.
Six axes per state item: Each piece of state is scored on authority, scope, mutability, provenance, recoverability, and actionability, giving builders a vocabulary for reasoning about what a stored fact can actually do.
A full state lifecycle: It traces state from write and retrieve through forget, audit, and rollback, surfacing the operational questions that production always-on systems must answer.
Why it matters: As agents move from single sessions to continuous operation, treating durable state as a governed, auditable resource is what separates a safe long-running system from one that quietly accumulates risk.

Paper | Tweet

3. HOLA

Linear-attention and state-space models compress an entire prefix into a fixed-size state, buying constant memory but overwriting earlier facts when many key-value associations compete. HOLA gives linear attention a hippocampal complement, pairing a compressive recurrent state with a small exact memory to recover long-range recall.

Two memories, different jobs: HOLA keeps the usual delta-rule state as compressive memory and adds a bounded exact KV cache, forming a semiparametric test-time memory where each store handles what it is best at.
Selective, learning-free writes: The cache writes without a learned eviction module, keeping only tokens whose prediction residual was actually committed to the state, so it stores exactly the associations that should not be forced through compression.
Strong recall at small scale: At 340M parameters on 15B SlimPajama tokens, it lowers Wikitext perplexity from 27.32 to 22.92, below a full-attention Transformer++ at 26.88, and stays robust on RULER needle recall out to 32k tokens, 16 times its training length.
Why it matters: It shows you can keep linear attention’s efficiency and still recover the exact recall that pure compression destroys, a practical path for long-context models that cannot afford full attention.

Paper | Tweet

4. Puzzle-75B

Bigger mixture-of-experts models keep winning on quality, but serving them at interactive latency is still hard. NVIDIA compresses the hybrid MoE Nemotron-3-Super into Puzzle-75B-A9B and roughly doubles interactive server throughput while holding quality.

Joint structural search: Heterogeneous MoE pruning, active-parameter budget, and Mamba pruning are optimized together rather than one at a time, wrapped in an iterative pipeline with distillation, RL, quantization, and a Multi-Token Prediction head.
Large throughput gains: On a single 8xB200 node it hits about 2x the parent’s server throughput at matched user-throughput, a direct win for anyone serving these models under latency constraints.
Concurrency at long context: At 1M-token context on a single H100, concurrency climbs from 1 request to 8, expanding what long-context workloads a single accelerator can host.
Why it matters: Accuracy holds across reasoning, coding, long-context, and agentic benchmarks, so cheaper serving with agentic capability intact changes what teams can afford to run in production.

Paper | Tweet

5. The Harness Effect

As orchestration harnesses mediate every model call, this study asks how much the harness alone moves cost and performance. It ran 22 evaluation tasks across six foundation models, then changed only the orchestration layer while holding the models constant.

Harness-only, models fixed: By varying just the orchestration layer over models like Claude Sonnet 4.6, Gemini 3.1, Qwen 3.6, and GLM 5.1, the study isolates the harness as the variable and measures its independent effect.
Big, consistent savings: Holding models constant, the harness cuts blended cost per task 41%, tokens per task 38%, and median wall-clock 44%, with completion quality at parity.
Two clean regularities: Efficiency is model-invariant, every model gets 33 to 61% cheaper, while quality gain correlates almost perfectly with baseline model strength (r=0.99), an effect the authors call harness leverage.
Why it matters: On this workload the orchestration layer moved cost per task more than the entire spread of the model menu did, making the harness the one component whose efficiency multiplies across every model a team runs.

Paper | Tweet

6. ReContext

Models now support 128K context windows yet still fail to use evidence already sitting in the prompt. ReContext is a training-free inference harness for long-context reasoning that uses model-internal relevance signals to build a query-conditioned evidence pool, then replays it right before final generation while preserving the full original context.

Memory framing of context: It treats the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation, a clean cognitive analogy that drives the design.
No training, no pruning: There is no fine-tuning, no external memory, and no pruning of the original context, so the method drops into existing models without changing weights or losing information.
Consistent gains across backbones: Across eight 128K long-context datasets it improves evidence utilization on Qwen3-4B, Qwen3-8B, and Llama3-8B, taking the best average rank on all three, with public code.
Why it matters: It targets the real long-context failure, using evidence that is already present, and fixes it at inference time, a cheap and general lever for reasoning over long prompts.

Paper | Tweet

7. Agent Limitations Taxonomy

Benchmark scores keep climbing, yet the same agent failures resurface across otherwise unrelated evaluations, hidden behind the leaderboard. This University of Oxford work synthesizes 27 benchmark, taxonomy, and audit papers spanning 19 benchmarks into the first cross-cutting taxonomy of LLM-agent limitations.

Six failure clusters: The taxonomy names tool invocation and parameter errors, planning and constraint-satisfaction failures, long-horizon degradation from context accumulation, multi-agent coordination breakdowns, safety failures under adversarial or underspecified conditions, and measurement validity problems.
Failures compound nonlinearly: Reliability drops faster than task length grows, so strong sub-task scores do not add up to end-to-end success on longer tasks.
Scaffolding is not a fix: Adding scaffolding does not reliably improve reliability, undercutting the assumption that more orchestration automatically buys robustness.
Why it matters: By giving shared names to failures that leaderboards obscure, the taxonomy helps teams diagnose why agents break in production instead of trusting benchmark gains that do not transfer.

Paper | Tweet

8. BlockSearch

BlockSearch runs the first systematic study of in-context retrieval at the scales real retrievers actually face, million-token corpora and length generalization far beyond training size. It introduces a 0.6B language-model retriever whose architectural and training changes improve over prior LM baselines and length-generalize up to 10 times beyond their training length, pointing toward retrievers that stay reliable as context windows keep growing.

Paper | Tweet

9. RLVR Meets Human Likeness

RL with verifiable rewards only optimizes what you can objectively score, so style, structure, and diversity quietly collapse and reward hacking creeps in. This MIT work adds an adversarial discriminator trained on human demonstrations as a learned proxy for the human output distribution, and the generator maximizes both task accuracy and that human-likeness signal. Across bug fixing, story generation, and a reward-hacking benchmark, it preserves RLVR’s accuracy gains while restoring the fuzzy properties it usually destroys, with misbehavior nearly disappearing.

Paper | Tweet

10. Replicating ML Papers with Agents

This work tests whether a coding agent can replicate a scientific ML paper from its materials alone, using a skill that turns each paper claim into a target with recorded evidence and gating completion on workspace evidence rather than the agent’s final message. Across twelve runs over four papers, all twelve workspaces pass the completion gate and all 158 recorded targets are matched with report coverage. Yet repeated runs still differ in how papers are split into targets and in numerical fidelity, so completion becomes reproducible even when the path is not.

Paper | Tweet

🤖 AI Agents Weekly: GPT-5.6 Family, Meta Muse Spark 1.1, Grok 4.5, SWE-1.7, Robostral Navigate, The Harness Effect, and More

Sat, 11 Jul 2026 16:33:08 GMT

In today’s issue:

OpenAI ships the GPT-5.6 family
Meta releases Muse Spark 1.1
OpenAI launches ChatGPT Work agent
xAI releases Grok 4.5 for coding
Cognition ships SWE-1.7 at 1000 tok/s
Mistral drops Robostral Navigate
Harness design sets agent economics
OpenAI launches GPT-Live voice
Google open-sources Gemma 4
Tencent open-sources 295B Hy3
Google ships Cloud Run sandboxes
Nous puts Hermes Agent in the cloud
Microsoft releases Flint for agents
Ternlight runs embeddings in-browser
GPT-5.6 proves 50-year math conjecture
Databricks benchmarks coding agents
OpenAI audits SWE-Bench Pro
FrontierFinance benchmarks agent analysts
Paper turns memory into navigation
GitLost tricks GitHub’s AI agent
Anthropic finds a global workspace
Sakana replays Picbreeder with VLMs

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 05 Jul 2026 16:54:05 GMT

1. Red Queen Gödel Machine

Self-improving agents are only as strong as the evaluator scoring them, and most systems freeze that evaluator in place, so improvement stalls the moment the judge stops getting harder. The Red Queen Gödel Machine makes the evaluator part of the search itself, letting agents and the criteria that judge them co-evolve.

The stationary-evaluator trap: Classic self-improvement loops assume a fixed evaluation criterion, so once an agent saturates it, the reward signal goes flat and progress plateaus no matter how much compute you add.
Controlled utility evolution: The framework lets the utility function update at epoch boundaries, turning evaluation into a moving target that continually re-opens headroom for the agent to climb.
Evolving evaluators and adversarial objectives: By opening the search to evolving evaluators, the method can discover things like a reviewer that stays equally stringent on AI and human work, imposing a curriculum-like pressure on the task agent.
Why it matters: Framing self-improvement as a Red Queen race between agents and evaluators offers a principled route past the plateaus that limit today’s agentic loops, pointing toward open-ended systems that keep improving instead of settling.

Paper | Tweet

Message From Our Sponsor

Grab a Number

2. MCP Server Patterns

As teams rush to wrap tools and data behind the Model Context Protocol, they keep rebuilding the same server shapes without shared names for them. This industry experience paper catalogs the recurring architectures so builders can reason about MCP servers the way software engineers reason about design patterns.

Five recurring server patterns: Across fifteen independently developed servers, the authors identify Resource Gateway, Tool Orchestrator, Stateful Session Server, Proxy Aggregator, and Domain-Specific Adapter, each documented in the classic context, problem, solution, and consequences form.
Grounded in real deployments: The corpus mixes production servers from a voice AI platform with public servers from the official MCP registry, so the patterns reflect how MCP is actually built rather than how a spec imagines it.
Anti-patterns and cross-cutting concerns: Beyond the patterns, the paper flags four anti-patterns and the recurring hard parts around authentication, versioning, and observability that every serious MCP deployment eventually hits.
Why it matters: A shared vocabulary lets teams pick the right server shape on purpose, compare designs, and avoid re-deriving the same tradeoffs, which is exactly what a fast-growing protocol ecosystem needs to mature.

Paper | Tweet

3. The Verification Horizon

Reinforcement learning for coding agents lives or dies on the reward signal, and this Qwen work argues there is no silver bullet. As policy capability grows, any fixed reward function eventually gets gamed, so verification has to co-evolve with the generator it scores.

No fixed reward survives a stronger policy: The central claim is that reward hacking is not a bug to patch once but a moving target, since a more capable agent will always find new ways to exploit a frozen verifier.
Four reward constructions studied: The authors examine a test verifier for general coding, a rubric verifier for frontend work, the user as verifier for real-world tasks, and an automated agent verifier for long-horizon problems.
Three axes of a good signal: They characterize verification quality along scalability, faithfulness, and robustness, and show that hitting all three at once is the real difficulty rather than any single verifier design.
Why it matters: Targeted verification design measurably suppresses reward hacking and lifts task quality across internal and public benchmarks, reframing verifier engineering as a first-class, continually evolving part of the RL loop.

Paper | Tweet

4. Paper Assistant Tool

AI is accelerating how fast papers get written, but peer review is still bottlenecked on human throughput, with combined submissions to the big ML conferences projected to top 73,000 this year. Google’s Paper Assistant Tool is an agentic framework built to do deep scientific review and verification at that scale.

Deep review, not surface checks: PAT ingests full manuscripts and produces a comprehensive evaluation that checks theoretical results, validates experiments, suggests improvements, and surfaces potential flaws rather than skimming for surface issues.
Agentic verification at the core: The system leans on verification agents to actually test claims, echoing a broader shift toward treating verification as the load-bearing capability in automated science.
A ladder of AI-human collaboration: The paper lays out four progressive roles, from an author’s tool, to a reviewer’s assistant, to an independent AI reviewer, giving teams a way to think about how much autonomy to grant.
Why it matters: The authors sketch an AIrXiv-style repository where papers are vetted by specialized agents across rounds of automated review and rebuttal, pointing toward continual, scalable evaluation that keeps pace with AI-assisted research.

Paper | Tweet

5. Generative Skill Composition

Coding agents accumulate large skill libraries, and picking the right skills for a task has become the bottleneck. The usual options either dump the whole collection into context or retrieve skills with embeddings and rerankers, and both treat selection as a ranking problem rather than a joint plan.

Composition as one joint decision: SkillComposer decides which skills, how many, and in what order all at once, instead of scoring skills independently and hoping the pieces fit together.
A constrained autoregressive decoder: A decoder over skill identifiers produces the full plan in a single pass, so dependencies between successive skills fall out of the generation naturally.
Strong gains at lower token cost: On SkillsBench with frontier models, it lifts pass rate well beyond the no-skill baseline, beats top-3 retrieval, and matches the gold-skill upper bound while using fewer prompt tokens.
Why it matters: As skill libraries keep growing, treating selection as generation rather than retrieval is what lets agents surface and sequence the right capabilities without drowning in their own toolbox.

Paper | Tweet

6. AutoMem

Memory for LLM agents is usually a fixed module bolted onto the model, but knowing what to encode, when to retrieve, and how to organize notes is itself a skill. AutoMem, from Stanford, treats memory management as a trainable cognitive ability, a capacity cognitive science calls metamemory.

Memory ops in the action space: Read, write, search, and append live in the same action space as task actions, so the model itself decides what to store and when to pull it back rather than following a hand-designed policy.
Two meta-learning loops: One loop optimizes the agent scaffold, the memory structure, while a second trains a dedicated memory specialist from the agent’s own traces, separating memory structure from memory proficiency.
Large gains without touching task behavior: Optimizing memory alone yields roughly 2x to 4x progression gains and lifts an open-weight 32B model to frontier-level performance on long-horizon tasks like Crafter, MiniHack, and NetHack.
Why it matters: Framing memory as a learned skill instead of a frozen component gives agents a path to keep getting better at managing their own knowledge, which is exactly what long-horizon autonomy demands.

Paper | Tweet

7. RLMF

LLMs routinely hallucinate with high confidence, miss their own knowledge boundaries, and misreport uncertainty, and most fixes bolt calibration on from the outside. RLMF, a Google and Yale collaboration, instead turns the model’s own metacognition into the training signal.

Metacognition as the reward: The method refines completion rankings during preference optimization based on the quality of the model’s self-judgments, using how well a model assesses its own performance as an internal feedback signal.
A decoupled, two-stage recipe: It first calibrates the faithfulness of self-reported confidence scores, then maps those scores to natural, context-adaptable linguistic uncertainty through targeted output editing.
Better calibration without losing accuracy: RLMF reaches state-of-the-art faithful calibration across diverse tasks, surpasses standard RL by a wide margin, and sharpens the model’s ability to express its own capability limits.
Why it matters: Grounding calibration in the model’s own metacognition rather than external heuristics offers a more general path to trustworthy uncertainty, which is foundational for agents that must know when not to act.

Paper | Tweet

8. ASPIRE

ASPIRE reframes robot programming as continual, code-as-policy learning that compounds experience instead of discarding it. The system runs an open-ended loop with a closed-loop execution engine that exposes fine-grained multimodal traces, a skill library that distills validated fixes into transferable knowledge, and an evolutionary search over task sequences and control programs. It surpasses prior methods by up to 77% on perturbed manipulation and enables zero-shot generalization to unseen long-horizon tasks, with early evidence of sim-to-real transfer across different embodiments.

Paper | Tweet

9. HORIZON

HORIZON treats hardware design as repository-level code evolution, compiling a Markdown harness into a project pack with domain knowledge, an executable evaluator, an acceptance predicate, and a git and runtime policy. A hands-free agent loop then evolves an isolated git worktree, using repository operations for state management, tracing, and replay. Across ChipBench, RTLLM, Verilog-Eval, and nine CVDP categories it reaches full benchmark completion with a completely hands-free loop, extending repository-scale self-evolution from EDA software to hardware artifacts themselves.

Paper | Tweet

10. Reasoning Quality Emerges Early

Curating reasoning data is expensive because scoring a trace usually means reading it to the end, but this UCLA work shows the quality of a trace is largely decided in its opening tokens. A short prefix predicts whole-trace quality well enough to rank and filter on, and difficulty can be detected from the loss of the first 100 tokens at a perturbed checkpoint. That turns curation into a cheap early-stopping problem, outperforming baselines while being far more token efficient at building SFT data for reasoning models.

Paper | Tweet

🤖 AI Agents Weekly: Claude Sonnet 5, Grok Voice Agent Builder, LongCat-2.0, Hosted X MCP, Cursor for iOS, Claude Science, and More

Sat, 04 Jul 2026 13:57:22 GMT

In today’s issue:

Anthropic ships agentic Claude Sonnet 5
xAI launches no-code Voice Agent Builder
Meituan open-sources 1.6T LongCat-2.0
X ships a hosted MCP server
Cursor launches always-on iOS agents
Claude Science app enters beta
Anthropic brings Fable 5 back
Z.ai ships ZCode for GLM-5.2
Google drops Nano Banana 2 Lite
Vercel adds voice agents to AI Gateway
WebKit ships a Safari MCP server
Google introduces TabFM for tables
Bridgewater fine-tunes an expert-judgment model
OpenAI releases GeneBench-Pro
NVIDIA splits a 30B diffusion model
Claude Code steganography sparks debate
Microsoft launches its $2.5B Frontier Company
AutoMem makes memory a trainable skill
ClawArena benchmarks subagent orchestration
Qwen 3.6 wins local development

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 28 Jun 2026 15:44:22 GMT

1. Sakana Fugu

Frontier LLMs keep advancing, and different providers are increasingly specializing in distinct domains, which raises a natural next objective: how do you combine those individual specializations into one collectively intelligent system? Sakana Fugu answers with a family of orchestrator models that are themselves language models trained to read a user query and dynamically devise the agentic scaffold needed to solve it.

Orchestrator models, not a fixed pipeline: Fugu is trained to understand a query and build an adaptive agentic scaffold on the fly, harnessing and amplifying a team of LLM agents rather than routing to a single frozen workflow.
Performance beyond any single agent: Through these query-adaptive scaffolds, Fugu reaches state-of-the-art results against other publicly accessible models across SWE-Bench Pro, Terminal Bench, LiveCodeBench, GPQA-Diamond, Humanity’s Last Exam, and CharXiv Reasoning.
Two models for two regimes: They release Fugu, which balances answer quality against latency for everyday use, and Fugu-Ultra, which prioritizes quality on the hardest problems.
Why it matters: The training paradigm combines large-scale fine-tuning, evolutionary algorithms, and reinforcement learning, plus the infrastructure to turn that into a production system, pointing to dynamic, query-adaptive scaffolds and collective intelligence as a path toward the next frontier of AI capabilities.

Paper | Tweet

2. Agent-Native Memory

Memory for LLM agents has quietly grown from a retrieval add-on into a full data system, with persistent storage, retrieval, update, consolidation, and lifecycle governance running throughout an agent’s execution. Yet most evaluations still score memory only through end-to-end task metrics like F1 and BLEU, treating the whole stack as a black box. This paper studies agent memory from a data management perspective and asks what we are actually missing when we measure it that way.

A data management view of memory: The authors argue that operational cost, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates are first-class concerns that task-success metrics hide entirely.
A four-module decomposition: They break memory into representation and storage, extraction, retrieval and routing, and maintenance, then evaluate 12 representative memory systems plus two baselines across five workloads spanning 11 datasets.
No single architecture wins: Effectiveness depends on how well the memory structure matches the workload bottleneck, and fine-grained ablations quantify each module’s effect on representation fidelity, retrieval precision, update correctness, and long-horizon stability.
Why it matters: The study shows localized maintenance is more cost-efficient than global reorganization, and reframing memory as a system with measurable trade-offs is what gets us toward genuinely agent-native memory rather than another leaderboard number.

Paper | Tweet

3. Autodata

Building synthetic training data has mostly stayed a fixed pipeline that you hand-tune once and then freeze. Autodata rethinks that by casting an AI agent as a data scientist that builds high-quality training and evaluation data, then meta-optimizes that agent so it learns to create even stronger data over time.

An agent as data scientist: Autodata is a general formulation in which an AI agent plays the role of a data scientist building both training and evaluation data, instantiated as a concrete, practical implementation the authors call Agentic Self-Instruct.
Meta-optimization compounds the gains: Beyond using the agent to generate data, they train (meta-optimize) the data scientist agent itself, and this self-improvement step delivers a larger performance uplift than base agentic data creation alone.
Consistent across domains: On computer science research tasks, legal reasoning, and reasoning with mathematical objects, Autodata beats classical synthetic dataset creation methods, showing the approach is not tied to a single problem type.
Why it matters: Agentic data creation turns increased inference compute into higher-quality training data, offering a path that could change how teams build datasets rather than freezing a pipeline and hoping it generalizes.

Paper | Tweet

4. Critique of the Agent Model

The word agent now covers everything from a for-loop with tool calls to speculative machine superintelligence, which makes it nearly useless as a technical term. This position paper from Eric Xing and collaborators tries to fix that by asking what an agent actually is and what agency consists of, drawing on Descartes and on science-fiction portrayals of autonomous beings to ground the discussion.

Five dimensions of agency: The authors analyze agent architectures along goal, identity, decision-making, self-regulation, and learning, and argue that genuine agency requires these structures to be internalized in the system rather than assembled through external scaffolding.
Agentic versus agentive: They draw a sharp line between agentic systems, whose competence lives in engineered workflows, and agentive systems, whose capabilities including social interaction arise endogenously, marking the boundary between task-specific tools and open-world autonomy.
A concrete architecture: Building on the analysis, they propose the Goal-Identity-Configurator, combining hierarchical goal decomposition, identity evolution, simulative reasoning grounded in a separately trained world model, learned self-regulation, and self-directed learning from real and simulated experience.
Why it matters: Clear definitions are not academic hair-splitting here. They shape what we build and what we should reasonably fear, and the paper centers auditability, controllability, and safety for systems that hold more autonomy yet stay under human oversight.

Paper | Tweet

Message from the Editor

We just released LLM-as-a-Judge, a hands-on DAIR Academy lab where you build an LLM judge from scratch to evaluate open-ended AI output. Across six short labs, you grade a support bot’s freeform replies on a rubric, then validate the judge against human labels and harden it against bias, ending with a small, trustworthy evaluation harness you can point at any open-ended task.

Start LLM-as-a-Judge

5. Agent-as-a-Router

Most users now have access to many LLMs that each excel in different domains, so routing each task to the right model matters for both quality and cost. Existing routers treat this as a static, one-off classification problem, and this paper shows that framing is exactly what holds them back.

Information deficit is the bottleneck: Simply augmenting a vanilla LLM router with performance statistics at the task-dimension level yields a 15.3% relative gain, surpassing a heuristic router built on the same priors, which pinpoints missing information rather than model choice as the real limiter.
Routing as a closed loop: Agent-as-a-Router formalizes routing as a Context, Action, Feedback, Context loop that accumulates execution-grounded experience during deployment instead of deciding once and moving on.
A concrete system and benchmark: The framework is instantiated as ACRouter, built from an Orchestrator, a Verifier, and a Memory module, and the authors release CodeRouterBench, roughly 10K task instances scored across 8 frontier LLMs for regret-based comparison on streaming tasks.
Why it matters: ACRouter achieves the lowest cumulative regret on in-distribution tasks and generalizes to out-of-distribution agentic programming, showing that treating routing as an experience-gathering agent, not a classifier, is what closes the information gap.

Paper | Tweet

6. Agent Communication Protocols

As multi-agent systems try to move past the limits of standalone agents, communication becomes the load-bearing infrastructure, and the protocol landscape for it is a fragmented mess. This study builds a technical taxonomy to classify and compare LLM agent communication protocols and to make the interoperability problem legible.

A five-dimensional taxonomy: Following an established iterative method, the authors classify protocols along counterparty, payload, interaction state, discovery mechanism, and schema flexibility, derived through five iterations over nine actively maintained open-source protocols with real adoption.
Recurring architectural patterns: Every sampled agent-to-agent protocol combines hybrid payloads with session-state persistence, most support multiple predefined schemas, and two negotiate schemas at runtime, signaling a clear trend toward schema flexibility.
Where the gaps are: Decentralized discovery remains rare, and the analysis suggests short-term convergence pressure toward protocols that unify agent-to-agent and agent-to-context communication for tools and data.
Why it matters: No single protocol is likely to maximize versatility, efficiency, and portability at once, so the field will probably evolve into a federated, layered protocol stack, and this taxonomy gives teams a way to choose protocols and surfaces open problems like privacy and policy enforcement.

Paper | Tweet

7. A Pinch of Human Data

Self-play reinforcement learning can train driving policies with no human data at all, swapping expensive human demonstrations for cheap large-scale simulation. The catch is that pure self-play tends to discover effective but alien driving conventions that real people cannot work with, and the usual fixes lean on brittle reward engineering and domain randomization.

Human data as a regularizer: Instead of discarding demonstrations or imitating them wholesale, the method treats human data as a regularization objective layered on top of a minimal safe goal-reaching reward, keeping behavior compatible with people without hand-tuning conventions.
A little goes a long way: Just 30 minutes of human demonstrations, roughly 2500 times fewer than comparable imitation learning approaches, is enough to pull self-play policies into human-compatible behavior.
Cheap to train: The resulting policies coordinate with held-out human trajectories and finish training in 15 hours on a single consumer-grade GPU, which keeps the recipe accessible rather than a frontier-lab luxury.
Why it matters: Behavioral alignment with humans is the hard part of deploying autonomous policies in shared environments, and this work shows that a tiny, well-placed dose of human data can fix what massive reward engineering struggles to, pointing to a cleaner path for human-AI coordination.

Paper | Tweet

8. Skill-MAS

Automatic generation of multi-agent systems is stuck between inference-time methods that reuse frozen frontier models but never learn, and training-time methods that internalize experience through gradient updates but are capped by the weaker models small enough to fine-tune. Skill-MAS proposes a third path that treats high-level orchestration as an evolvable Meta-Skill, decoupling experience retention from weight updates so frontier models keep getting better at orchestration without any gradient steps. Across four complex benchmarks and four distinct LLMs it delivers strong, transferable gains at a favorable cost-performance trade-off.

Paper | Tweet

9. Reliability without Validity

LLM-as-a-Judge is the default way to evaluate language models, but validating those judges with exact-match agreement never corrects for chance and systematically overstates how good they are. In the largest audit to date, spanning 21 judges from nine providers across MT-Bench, JudgeBench, and RewardBench over 118 runs and roughly 541,000 judgments, the gap between raw agreement and chance-corrected Cohen’s kappa runs 33 to 41 percentage points, rankings shift by up to 14 positions across benchmarks, and high test-retest reliability coexists with severe position bias. The authors distill their findings into a Minimum Viable Validation Protocol so teams can stress-test judges before trusting them.

Paper | Tweet

10. NatureBench

Can coding agents move past reproduction toward actual discovery on real scientific problems? NatureBench distills 90 cross-discipline tasks from peer-reviewed Nature-family papers and runs them in NatureGym, an automated pipeline that builds a standardized containerized environment per task to fix the environment-fragmentation problem. Under a strict web-search-disabled protocol, the strongest of ten frontier agent configurations beats published SOTA on only 17.8% of tasks, and analysis shows agents win mainly by translating problems into familiar supervised prediction rather than through genuine scientific invention.

Paper

🤖 AI Agents Weekly: GPT-5.6, Ornith-1.0, Codex Inside OpenAI, Claude Tag, Qwen-AgentWorld, AI SDK 7, and More

elvis — Sat, 27 Jun 2026 15:01:56 GMT

In today's issue:

OpenAI previews the GPT-5.6 family
Ornith-1.0 ships open coding models
OpenAI: agents reshape every department
Claude Tag joins your Slack team
Qwen open-sources AgentWorld world model
Cursor exposes benchmark reward hacking
Vercel ships AI SDK 7
OpenRouter MCP picks your model
Mistral launches OCR 4
Gemini 3.5 Flash gains computer use
Sakana's Fugu-Ultra hits OpenRouter
Notion adds Claude and Cursor agents
Exa Connect links agents to data
Engram raises $98M for AI memory
Lilian Weng revisits scaling laws
Plans don't persist in agents
Tmax opens terminal-agent training

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

elvis — Sun, 21 Jun 2026 15:02:39 GMT

1. SpatialClaw

Spatial reasoning over 3D and 4D scenes is still where general vision-language models break down, because they emit a text answer directly rather than measuring anything. From NVIDIA, SpatialClaw is a training-free framework that rethinks the action interface and lets a VLM-backed agent reason through code instead. The agent writes one Python cell per step into a persistent Jupyter kernel preloaded with perception primitives and scientific libraries, then inspects intermediate results and revises its strategy across steps.

Code as the action interface: Perception tools like SAM3 segmentation, Depth-Anything-3 reconstruction, and geometry utilities are exposed as plain Python callables, so the agent composes them programmatically rather than guessing spatial relationships from pixels.
A persistent, stateful kernel: Masks, depth maps, camera geometry, and trajectories are ordinary Python variables that the kernel preserves across turns, so any object produced at one step stays available for composition, inspection, and revision later.
Strong results without adaptation: Across 20 spatial reasoning benchmarks spanning static and dynamic tasks, SpatialClaw reaches 59.9% average accuracy, beating the prior spatial agent by 11.2 points, with consistent gains across six VLM backbones from two model families.
Why it matters: Because it is training-free and model-agnostic, SpatialClaw turns code execution into a general substrate for spatial reasoning that any capable VLM can plug into, instead of requiring bespoke spatial fine-tuning.

Paper | Tweet

Message from the Editor

We just released 30 Days of Hermes Agent, a hands-on DAIR Academy lab that teaches agent workflows in a real interactive terminal. Across 30 short labs, you use Hermes Agent to turn a messy Personal Knowledge Vault into a working knowledge operations system with readable notes, searchable context, reusable templates, review workflows, task boards, safety rules, and handoff docs.

Start 30 Days of Hermes Agent

2. Compositional Skill Routing

Real tasks rarely map to a single skill. They usually need several skills composed together, yet most skill routing still treats the problem as picking one tool from a library. This work formalizes Compositional Skill Routing, where an agent must select and sequence multiple reusable skills from large libraries to satisfy a complex query, and introduces SkillWeaver, a decompose, retrieve, and compose pipeline built around it.

A three-stage pipeline: SkillWeaver decomposes a query into sub-tasks with an LLM, matches each sub-task to a skill using a bi-encoder with FAISS indexing, and then performs dependency-aware planning to assemble an executable plan.
A realistic benchmark: The authors release CompSkillBench, a benchmark of 300 compositional queries over 2,209 real MCP server skills spanning 24 functional categories, so routing is tested against actual tool ecosystems rather than toy libraries.
Decomposition is the bottleneck: Task decomposition quality emerges as the primary limiting factor, and Iterative Skill-Aware Decomposition, which feeds retrieval information back into the decomposition step, lifts accuracy from 51.0% to 67.7%.
Why it matters: As agent skill libraries scale to thousands of entries, single-tool routing stops being enough, and treating routing as a compositional planning problem is what lets agents handle genuinely multi-step requests.

Paper | Tweet

3. PreAct

Computer-using agents drive real software through the screen, but they solve every task from scratch. Ask one to repeat a task and it re-reads the screen and re-reasons every tap, paying the full cost again. PreAct fixes this by compiling the first successful run into a small state-machine program, where states check the screen and transitions act, then replaying that program on later runs instead of invoking the agent.

Compile runs into a state machine: A completed task is captured as an explicit program rather than a free-form trace, turning a one-off solution into a reusable artifact that can be executed deterministically.
Replay with no per-step model calls: Replaying the compiled program runs 8.5 to 13 times faster than the agent because it needs no per-step language-model calls on repeated tasks.
Safe by construction: At each step PreAct checks that the screen matches what the program expects before acting, and hands control back to the agent the moment something is off, and it only stores programs an independent evaluator confirms solve the task from a clean state.
Why it matters: This turns computer-using agents from interactive tools that re-reason everything into repeatable operational systems, which is exactly what is needed to deploy them on recurring real work.

Paper | Tweet

4. Can LLM Agents Infer World Models?

Can an LLM agent actually build a model of an environment it cannot see? This work makes that question gradeable through agentic automata learning. An agent has to uncover a hidden deterministic finite automaton by interacting with an oracle through two interfaces, membership queries that ask whether a string belongs to the target language, and equivalence queries that ask whether a proposed automaton is correct, which yields a clean, scalable testbed for interactive discovery.

A gradeable world-model test: Casting world-model inference as DFA learning gives objective success criteria and measurable interaction efficiency, with classic automata-learning algorithms as strong, well-understood baselines.
Controlled, scalable difficulty: The size of the hidden automaton acts as a difficulty knob, so the benchmark can scale task complexity smoothly rather than relying on a fixed set of puzzles.
Agents lag classic algorithms: Current agents can sometimes perform non-trivial interactive discovery, but performance drops sharply as DFA size grows, and trajectory analyses reveal recurring failures in query planning, evidence integration, and hypothesis construction.
Why it matters: Reasoning models clearly beat non-reasoning ones here, but the large gap to classic algorithms shows that systematic, interactive world-model building is still an unsolved capability rather than a byproduct of scale.

Paper | Tweet

5. From Trainee to Trainer

Who should design the training environment for an RL agent, the practitioner or the policy itself? RL pipelines for LLMs usually rely on manually redesigned environments between stages, with practitioners guessing which configuration will best improve the current policy. This paper hands that job to the model, proposing an LLM-as-Environment-Engineer framework where the policy diagnoses its own weaknesses and proposes the next environment to train on.

The policy designs its own curriculum: Instead of a human reshaping the environment between stages, the current policy analyzes failure trajectories together with contextual information and proposes modifications to the next-stage training environment configuration.
Failure-driven environment edits: Because the proposals are grounded in the policy’s actual failure modes, the curriculum targets the specific gaps holding the model back rather than generic difficulty bumps.
The trainee becomes the trainer: A key finding is that the current RL checkpoint serves as a better environment engineer than the original base model, suggesting that learning to act also improves the model’s ability to diagnose what it still cannot do.
Why it matters: Manual between-stage environment design is one of the least scalable parts of RL for LLMs, and letting the policy steer its own curriculum closes a slow human-in-the-loop step that has bottlenecked agentic RL.

Paper | Tweet

6. OpenClaw-Skill

Equipping LLM agents with effective skills is most of the battle in real systems, yet most skill-induction work distills one trajectory at a time, which produces narrow, brittle skills. OpenClaw-Skill introduces Collective Skill Tree Search, a tree-search-based skill construction framework that builds a structured, diverse, and generalizable tree of skills, then trains agents to actually use what it builds.

Collective Skill Tree Search: Rather than distilling a single trajectory into a single skill, CSTS searches over a tree of candidate skills, using multiple models to generate and evaluate them so the library captures diverse strategies.
A structured, reusable skill tree: Organizing skills hierarchically yields competencies that generalize across tool use, multi-step reasoning, and environmental interaction instead of overfitting to one task.
Training agents to leverage skills: Building the tree is only half the work, so the framework pairs construction with a learning step that teaches agents to retrieve and apply the constructed skill hierarchy effectively.
Why it matters: Reusable skill libraries are becoming the backbone of capable agents, and moving from per-trajectory distillation to collective tree search is a concrete recipe for libraries that stay useful as tasks grow.

Paper | Tweet

7. Back on Track

Diffusion large language models generate text in a way that does not fit cleanly into the reinforcement learning recipes built for autoregressive models, and training them to reason exposes two specific problems. Rewards are sparse, so a single terminal reward fails to guide intermediate generation steps, and policy updates sometimes drift toward unnatural trajectories rather than authentic generation paths. This paper proposes Process Aligned Policy Optimization to fix both.

Two failure modes named: The work isolates sparse rewards and trajectory drift as the core obstacles to stable RL training for reasoning in diffusion LLMs, rather than treating training instability as a black box.
Step-aware process rewards: PAPO converts terminal rewards into granular, step-level guidance, so intermediate denoising steps receive a learning signal instead of waiting for a single end-of-sequence score.
Entropy-guided re-enactment: At critical high-uncertainty moments, the method replays genuine generation paths, keeping updates aligned with how the model actually produces text instead of chasing artificial trajectories.
Why it matters: Diffusion LLMs are a serious alternative to autoregressive models, and giving them a stable RL recipe for reasoning, with reported gains from 4.5% to 42.2% on benchmarks like GSM8K and MATH500, helps close the reasoning gap between the two paradigms.

Paper

8. AtomMem

Long-term memory for LLM agents tends to fail in two ways: coarse summaries drift over time, and unconstrained updates corrupt what was already stored. AtomMem keeps the unit of memory small, using a Fact Executor that selectively extracts high-value atomic facts from long interactions and organizes them into hierarchical event structures and temporal user profiles, with an associative memory graph that reconnects fragmented memories at retrieval. The approach reports state-of-the-art results on the LoCoMo long-term memory benchmark.

Paper | Tweet

9. Beyond Domains

LLM web agents usually run as tool callers, reading a fresh page each turn and emitting one low-level action, so both task horizons and the number of LLM completions blow up. This work makes web skills reusable across sites with SkillMigrator, which stores induced skills as transferable interaction patterns keyed by page-layout structure rather than instruction similarity or site metadata, so a skill learned on one site fires on new sites with the same interaction shape. It cuts the average LLM-action count by 8 to 10% on WebArena and Mind2Web at comparable success rates.

Paper | Tweet

10. The Stanford EDGAR Filings Dataset

Clean, long-context documents remain scarce for pretraining, especially in finance. This release reconstructs U.S. SEC corporate and financial disclosures into layout-faithful, token-efficient MultiMarkdown, publishing 152B tokens in SEFD-v1 out of an estimated 550B-token archive spanning 18.5M filings, with less than 0.1% overlap with Common Crawl corpora. It also ships two derived benchmarks, EDGAR-Forecast for numerical forecasting and EDGAR-OCR for financial table transcription, to support financial reasoning, forecasting, and document understanding.

Paper

🤖 AI Agents Weekly: GLM-5.2, Claude Code Artifacts, Qwen-Robot Suite, Codex Skills, Block's Builderbot, SpatialClaw, and More

elvis — Sat, 20 Jun 2026 15:45:25 GMT

In today's issue:

Z.ai open-sources frontier GLM-5.2
Claude Code ships interactive Artifacts
Qwen launches the Robot Suite
Codex turns demos into skills
Block's Builderbot writes 15% of code
Flue 1.0 reimagines the agent harness
Vercel debuts the eve framework
Cursor launches Origin code hosting
Perplexity adds Brain memory to Computer
OpenRouter ships the Fusion API
NVIDIA's SpatialClaw codes spatial reasoning
ENPIRE self-improves robot policies
SkillsBench 1.1 audits agent skills
DeepMind maps an AI Control Roadmap
OpenAI trains broadly beneficial models
Anthropic measures returns to expertise

And all the top AI dev news, papers, and tools.

Autonomous Long-Running Coding Agents

elvis — Mon, 15 Jun 2026 20:44:55 GMT

Autonomous coding is moving from better prompting to better control systems. The important shift is that engineers are learning how to wrap agents in goals, evaluators, loops, and artifacts that let them keep working after the human stops typing.

This matters because most serious engineering work spans long horizons: ambiguous requirements, hidden constraints, partial failures, changing context, and repeated verification. The new frontier is designing the system around the agent so it can plan, execute, check its work, recover from mistakes, and keep making progress without constant human steering.

This piece is based on a DAIR.AI Academy session on autonomous long-running coding agents, where I walked through Claude Code’s /goal mode, the newer /loop command, verifiers, artifacts, and orchestration patterns in practice. Written in collaboration with Codex and Claude Code.

From Prompting to Goal Design

The core idea behind features like Claude Code’s /goal is simple. A coding agent remains the executor, but the human no longer interacts with it turn by turn. Instead, the human specifies the desired end state, the evidence required to prove success, the constraints that must not be violated, and, where possible, the number of turns and budget.

That goal works more like a contract than a longer prompt. A weak goal gives the model room to stop early, take shortcuts, or redefine success in a way that looks plausible in the transcript but fails in the real system. A strong goal gives the agent a target it can repeatedly measure itself against.

Engineering judgment still matters here. The best goals encode domain knowledge that the model would otherwise guess. For a research experiment, that might mean a target benchmark score, a held-out evaluation, a required loss curve, and a rule that the result must beat an initial baseline. For a UI task, it might mean a screenshot reference, concrete layout constraints, and a browser verification step. The model can execute, but the human still defines what “done” actually means.

The Evaluator Becomes a First-Class Component

Long-running agents need a second role besides the goal. That evaluator can be another coding agent, an LLM-as-judge, a script, a test suite, a benchmark harness, or a mix of all of them. The key design choice is matching the evaluator to the task. When success is crisp, deterministic checks are better. Type checks, unit tests, lint rules, integration tests, and benchmark scripts should be used whenever they can express the condition clearly.

When success is fuzzy, an agent evaluator becomes useful. A script can tell you whether tests pass, but it cannot easily decide whether a generated research report is coherent, whether an implementation faithfully follows a paper, or whether a UI matches a design intent. This is where the evaluator benefits from language, judgment, and sometimes vision.

The practical pattern uses deterministic checks as the floor and agent evaluation as the higher-level review. That combination reduces hallucinated success while still allowing autonomy on tasks that do not fit cleanly into a test assertion.

Verifiers Define the Boundary of Trust

The deeper point is that autonomy only works when the system has a reliable verifier. A coding agent can generate a plan, implement a feature, and explain why it believes the work is complete, but that explanation should not be treated as evidence. Evidence comes from an external check that the agent cannot easily talk its way around.

For code, the verifier might be a test suite, type checker, benchmark, browser run, screenshot comparison, or reproducible script. For research work, it might be a held-out evaluation, a reproduced table, a loss curve, or a benchmark score that improves over the baseline. For design work, it might be a reference screenshot plus a visual review step. The verifier is what turns a long-running agent from a confident text generator into a system that can be trusted with more time.

Most shortcuts appear at this boundary. If the verifier is vague, the model will often satisfy the easiest interpretation of the task. If the verifier is too narrow, the model may overfit to it and miss the broader intent. A good autonomous workflow, therefore, needs layered verification, with cheap deterministic checks catching basic failures and higher-level review catching judgment-heavy failures. A few of the frontier models can already achieve some level of verification, but based on my research, there is still an evident OOD problem, where if the verification task you assign to the agent falls outside the training distribution, models struggle significantly.

Verifiers are still an open area of research, but I anticipate more companies will start to make huge investments in this area. The concept of fine-tuned verifiers is also in high demand in the enterprise.

Loops Make Autonomy Durable

A goal gives the agent direction, but a loop keeps the work alive. This distinction is important because models often stop before the real task is finished. They may hit a turn limit, lose confidence, exhaust context, or decide that a partial solution is enough.

The loop is the outer control system. It wakes up, inspects progress, runs checks, compares the result against the goal, and sends the agent back in with the next instruction when the goal has not been met. In its simplest form, this is the Ralph loop pattern with a coding agent and a deterministic condition. In a more flexible form, the loop includes an evaluator agent that can reason about progress and decide what should happen next.

Long-running autonomy works as repeated effort under supervision from a control layer, not as one continuous act of intelligence. The agent can still fail, but the loop gives the system a way to notice the failure and continue instead of silently declaring victory.

Planning Is Where Expertise Enters

One of the strongest themes from the session was that planning remains critical. You can ask a frontier model to generate a plan, but you still need to inspect it, challenge assumptions, and make the success criteria sharper before handing the task to an autonomous loop.

This leads to a useful division of labor. A stronger planning model can help define the goal, identify missing constraints, and structure the evaluation. A different execution model can then run the implementation once the plan is clear. In practice, this means engineers should stop thinking of “the model” as a single choice. Model choice becomes an architecture decision.

Some models are better planners. Some are better executors. Some are cheaper evaluators. Some are better at vision-based review. A good orchestrator lets you swap these roles instead of waiting for one vendor to provide the perfect coding agent interface.

Visual Artifacts Become Control Surfaces

Terminal transcripts do not scale when many agents are running. Once you have several sessions working in parallel, raw text becomes a poor interface for understanding progress.

Live artifacts matter because a dashboard with loss curves, benchmark scores, task states, screenshots, cost estimates, and recent decisions gives the human a much better way to supervise autonomy. The artifact becomes the control surface for deciding when to intervene, rather than a report generated after the fact.

The most useful pattern is to separate storage from presentation. Markdown or a vault can store durable evidence, logs, notes, plans, and results. HTML artifacts can render that state into something visual and interactive. The agent can search the Markdown, while the human can monitor the artifact.

For UI and product work, visual cues are especially powerful. A screenshot reference can communicate design intent more precisely than prose, and a vision-capable evaluator can compare the implementation against that reference. This reduces the common failure mode where the agent technically implements the requested component but misses spacing, hierarchy, alignment, or product feel.

Session Mining Turns Usage Into Memory

Another important insight is that past agent sessions are a rich source of workflow data. If an agent repeatedly fails in the same way, forgets to run the same check, uses the wrong path, or retries the same broken command, that pattern should not stay buried in logs.

Session mining turns those transcripts into operating rules. An agent can scan the last thirty days of work, find recurring failure modes, and propose updates to project instructions, vault learnings, or agent rules. This is how a team can gradually improve its harness without manually remembering every mistake.

The goal is to make the local environment smarter without training a model from scratch. A small rule in an agent instruction file can prevent repeated failures across future sessions, especially when the rule is specific to the project.

A Practical Operating Model

For AI engineers, the emerging workflow looks like this.

Start with a small, cheap subset before launching the full autonomous run.
Write a goal with measurable success criteria, explicit constraints, and a turn or time budget (where possible).
Separate the executor from the evaluator so implementation and judgment are not collapsed into one role.
Define external verifiers before the long-running loop starts.
Use deterministic checks wherever possible, then add agent review for fuzzy criteria.
Require proof artifacts such as logs, screenshots, benchmark curves, or changed files.
Mine past sessions and promote repeated lessons into project instructions.

That is the difference between using a coding agent and engineering an autonomous coding system. One gives you a conversation. The other gives you a harness.

What Still Breaks

None of this removes the hard problems. Agents still take shortcuts. They still stop early. They still overestimate completion. They still produce confident but weak plans, especially on recent papers, unfamiliar benchmarks, or systems outside their training distribution.

Trusting them more will not solve this. Better control systems will. Goals, loops, evaluators, deterministic checks, visual artifacts, and session memory are all ways of making autonomy observable and correctable.

The direction is clear. The future of coding agents depends on better orchestration around more capable models, where engineers design the conditions under which agents can safely run for hours or days and still produce work that can be verified.

🥇Top AI Papers of the Week

Sun, 14 Jun 2026 15:00:54 GMT

1. MiniMax Sparse Attention

Ultra-long context is now a core requirement for agents, codebase-scale reasoning, multimodal workflows, and persistent memory, but dense softmax attention still makes million-token deployment expensive. MiniMax Sparse Attention (MSA) tackles this by adding blockwise sparsity on top of Grouped Query Attention, with a lightweight routing branch that chooses which key-value blocks each query group should actually attend to.

Two-branch attention design: The Index Branch scores the full causal context and selects Top-k key-value blocks independently for each GQA group, while the Main Branch performs exact sparse attention only over those selected blocks.
Hardware-aware implementation: The paper co-designs the sparse pattern with GPU kernels, using exp-free Top-k selection and KV-outer sparse attention to improve tensor-core utilization under block-granular access.
Large speedups at scale: On a 109B-parameter natively multimodal model, MSA matches GQA performance while reducing per-token attention compute by 28.4x at 1M context. The paired kernel reaches 14.2x prefill and 7.6x decoding wall-clock speedups on H800.
Why it matters: Long context is only useful if it can be served cheaply. MSA is compelling because it keeps the mechanism simple, trains it directly into a production-scale model, open-sources the inference kernel, and powers the released MiniMax-M3 model.

Paper | Tweet

Message from the Editor

We just released 30 Days of Hermes Agent, a hands-on lab that teaches agent workflows in a real, interactive terminal. Across 30 short labs, you use Hermes Agent to turn a messy Personal Knowledge Vault into a working knowledge operations system with readable notes, searchable context, reusable templates, review workflows, task boards, safety rules, and handoff docs.

Start 30 Days of Hermes Agent

2. Self-Harness

Most agent scaffolds are built once by hand and then frozen, even as the underlying models keep changing. This paper introduces Self-Harness, a paradigm where an LLM agent improves its own operating harness, the prompts, tools, memory, and orchestration around the base model, without human engineers or a stronger external agent. Because every model fails in its own way, the system mines those model-specific weaknesses and turns them into concrete, executable harness edits rather than generic advice.

A three-stage self-improvement loop: Self-Harness runs Weakness Mining, which clusters execution traces into model-specific failure patterns, then Harness Proposal, which generates diverse but minimal edits tied to those failures, then Proposal Validation, which accepts edits only after regression testing on held-in and held-out splits.
Consistent gains across base models: On Terminal-Bench-2.0, held-out pass rates rise for every model tested. MiniMax M2.5 improves from 40.5% to 61.9%, Qwen3.5-35B-A3B from 23.8% to 38.1%, and GLM-5 from 42.9% to 57.1%.
Weaknesses become edits: Rather than appending generic instructions, the loop converts each observed failure mode into a targeted change to memory, tools, or prompts, with reported relative improvements as high as 138%.
Why it matters: As models proliferate and evolve, hand-tuning a bespoke harness for each one does not scale. Self-Harness shows the scaffold itself can be made to adapt, closing the gap between a frozen harness and the model it wraps.

Paper | Tweet

3. Agents’ Last Exam

From Berkeley RDI, Agents’ Last Exam (ALE) is a living benchmark built to measure whether agents can do economically valuable work, not just score well on academic tests. It was assembled with more than 250 industry experts and maps over 1,000 verifiable tasks to the U.S. federal occupational taxonomy, organized as 55 subfields across 13 industry clusters. Every task has an objective, checkable outcome, so there is no subjective human grading, and the pool is designed to keep growing as new workflows are onboarded.

Grounded in real occupations: Tasks are defined against O*NET and SOC 2018 and span non-physical industries, deliberately targeting the professional workflows where agents would actually be deployed rather than puzzle-style problems.
Three difficulty tiers: Work is split into Near-Term, Full-Spectrum, and Last-Exam tiers, letting the benchmark track both near-term usefulness and the long tail of hard, multi-step jobs.
Far from saturated: The hardest tier sits at just a 2.6% average full pass rate across mainstream harnesses, and even strong setups like Codex with GPT-5.5 score below 50% on the easiest tier and under 10% on the hardest.
Why it matters: Strong scores on existing benchmarks have not translated into economically meaningful deployment. ALE reframes evaluation around verifiable, expert-curated work, giving a moving target that should resist saturation as agents improve.

Paper | Tweet

4. How AI Agents Reshape Knowledge Work

This economics paper, drawing on large-scale production data from Perplexity, studies how the shift from conversational assistants to autonomous agents is reshaping knowledge work. It compares Search, a conversational assistant, with Computer, a general-purpose agent system, along three dimensions: autonomy, efficiency, and the scope of tasks people take on. The framing is a cost-structure model in which agents carry higher fixed and delegation costs but lower per-step marginal costs, so they win once tasks are complex enough.

Autonomy looks different in practice: Computer performs around 26 minutes of autonomous machine work per session versus roughly 33 seconds for Search, and per-query dissatisfaction is 55% lower on the agent, 1.3% against 2.9%.
Large efficiency gains: On matched tasks, Computer cuts completion time from 269 to 36 minutes, an 87% reduction in time and about a 94% reduction in cost relative to humans working with Search alone.
Scope shifts upward: Agent queries are more cognitively complex, 71% abstract or non-routine versus 53%, with twice as much create-level work, and they bundle interdependent subtasks that cross occupational boundaries.
Why it matters: The data supports a clean prediction. As the fixed costs of delegation fall, agents move the affordable value frontier toward higher-value, multi-step knowledge work, which is exactly where adoption grew fastest, reaching 84 times its first-week volume over the study.

Paper | Tweet

5. Agentopia

Agentopia is one of the most ambitious agent-society testbeds yet, a 79-page release that drops 100 LLM agents into a persistent world and lets them live, form relationships, and pursue goals over 10 simulated years, a horizon orders of magnitude longer than prior day-level work. Beyond observing emergent social behavior, the authors use the simulation as a training signal, optimizing models toward a life reward that reflects human well-being via rejection sampling.

Long-horizon by design: Where earlier agent societies ran at the granularity of days, Agentopia simulates a decade of life per world, surfacing unscripted social strategies and interpersonal dynamics that only appear over long timescales.
Simulation as a training signal: The life-reward metric is used to fine-tune more anthropomorphic models, and the improvements transfer beyond the simulation to downstream role-playing benchmarks rather than staying trapped in the sandbox.
Measured gains: Trained agents improve overall CoSER Test performance by 15.6%, with the biggest jumps in Anthropomorphism at 23.7% and Character Fidelity at 16.4%, and they are respected by 24.2% more peers and liked by 15.9% more.
Why it matters: A single 10-year, 100-agent run consumes 13.7 billion tokens across 567,000 LLM calls. That scale is a statement about where agent research is heading: living, learning populations as both an object of study and a source of training data.

Paper | Tweet

6. The Geometry of On-Policy Distillation

On-policy distillation (OPD) has become one of the most discussed post-training recipes of the year, but it has mostly been treated as a black box sitting somewhere between supervised fine-tuning and RL. This paper opens it up, characterizing how OPD changes a model’s weights at the level of parameter geometry, and argues OPD is not a midpoint between SFT and RLVR but its own distinct kind of update.

It touches fewer weights: Compared with SFT, OPD updates affect far fewer parameters and largely avoid the dominant principal directions of weight space, which helps explain its sample efficiency.
Early subspace locking: OPD’s cumulative updates rapidly collapse into a narrow, low-dimensional subspace early in training, rather than spreading across many directions as SFT does.
That subspace is functionally sufficient: Constraining training to the early-formed subspace preserves OPD performance but substantially degrades SFT, showing the small subspace genuinely carries the useful signal rather than being an artifact.
Why it matters: Knowing where in weight space OPD does its work turns a popular but poorly understood recipe into something with a mechanistic account. That makes the method easier to reason about, combine with other objectives, and improve deliberately instead of by trial and error.

Paper

7. Lookahead Sparse Attention

Long-context decoding is bottlenecked by the KV cache, which grows with every token and quickly dominates memory at extreme context lengths. This work, branded around DeepSeek-V4, introduces Lookahead Sparse Attention (LSA), which avoids storing the full KV cache by predicting which parts of the context future decoding will actually need and retaining only those query-critical chunks.

A learned, lightweight indexer: Instead of keeping everything, a small indexer proactively selects the KV chunks that matter for upcoming generation, so the physical cache stays small without discarding information the model will need.
Backbone-free training: A decoupled training strategy lets the indexer be trained on its own without loading the full backbone model, cutting the cost of adding the mechanism to a large model.
Big cache savings, no quality loss: LSA shrinks the physical KV cache to 13.5% of the full-context baseline while slightly improving accuracy by 0.6% on average, and at 500K-token contexts it suppresses more than 90% of KV-cache overhead without destabilizing reasoning.
Why it matters: Ultra-long context is increasingly the difference between a toy demo and a usable system, and memory is the wall. Predicting what context you will need, rather than keeping all of it, is a practical route to long context that fits in real hardware budgets.

Paper

8. Latent Spatial Memory

Video world models struggle to stay consistent over long horizons because explicit 3D memory usually requires an expensive pixel-space loop. Mirage instead stores scene information directly in diffusion latent space, using depth-guided back-projection and latent-space warping to maintain persistent spatial memory. The approach reports up to 10.57 times faster end-to-end generation and 55 times lower memory use than explicit 3D-memory baselines while improving long-horizon spatial consistency.

Paper

9. The Consistency Illusion

Multi-agent debate is often judged by whether the agents end up agreeing, but this paper shows that output-level consensus can hide deep disagreement in the reasoning that produced it. The authors abstract agents’ reasoning traces and decisions into four states along two axes, reasoning similarity and conclusion agreement, and flag divergent agreement, where agents reach the same answer through very different paths. Across 600 content-moderation items, divergent agreement appeared in 118 cases and separated cleanly from genuine disagreement states with a Cohen’s d of 0.80, and routing on these categories beat divergence-only methods at flagging high-disagreement cases.

Paper | Tweet

10. Beyond Scalar Rewards

Reward models usually compress a judgment into a single scalar, but this paper argues human preferences are better captured as score distributions, and proposes Z-Reward, which internalizes reasoning into a predicted distribution before scoring. A large vision-language teacher does the reasoning-heavy judgment and is distilled into a compact student for efficient deployment, with the 27B teacher reaching 89.6% human-preference accuracy and the 9B student nearly matching it at 88.6%. Used as a reinforcement learning signal, it delivers a 41.3% net preference improvement over a supervised baseline, beating GRPO and other reward methods.

Paper

🤖 AI Agents Weekly: Claude Fable 5, Kimi K2.7-Code, NotebookLM Goes Agentic, DiffusionGemma, MiMo Code, and More

Sat, 13 Jun 2026 15:56:07 GMT

In today’s issue:

Anthropic ships Mythos-class Claude Fable 5
Kimi K2.7-Code open-sources a 1T coder
NotebookLM becomes an agentic workstation
Google’s DiffusionGemma generates text in blocks
Xiaomi open-sources MiMo Code agent
Cohere ships North Mini Code
Gemini 3.5 Live Translate goes real-time
Gemini-SQL2 tops BIRD text-to-SQL
Apple rebuilds Siri on Google Gemini
Grok opens a plugin marketplace
Claude Code adds nested subagents
Nex-N2 opens an agentic model series
Extend UI ships document-agent components
Cognition’s FrontierCode raises the eval bar
Study questions the multi-agent advantage
Recursive automates AI research

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 07 Jun 2026 15:00:58 GMT

1. Self-Revising Discovery Systems

From MIT, this paper argues that genuine scientific discovery is not answer generation but a change in the search space itself, and that an AI scientist must perceive that shift without being told. It develops a category-theoretic framework in which evidence, artifacts, operations, and verifiers are typed, and discovery is defined as a principled revision of that representational regime rather than more search within a fixed one.

Discovery means changing the regime: The system is built to detect when the representational regime should change and to revise it autonomously. That reframes an AI scientist from a faster searcher into something that can move the boundaries of the space it searches.
A typed, categorical foundation: Evidence, artifacts, operations, and verifiers are formally typed. Old results are carried into the new regime by functorial transport, and what counts as genuine discovery is the residual content that transport alone cannot explain.
Description-length gates keep it honest: Proposed revisions are accepted only when they reduce total description length, which separates real structural gains from mere added complexity. In one run, 388 proposals yield just 25 accepted revisions, a deliberately strict 6.4% rate.
Why it matters: Two concrete instantiations, protein-mechanics modeling and a knowledge-computation graph with typed skills and validation checkpoints, show category theory serving as both a formal language and an engineering spec. It is a more principled blueprint for autonomous discovery than search-only AI scientists.

Paper | Tweet

2. Disentangling Agent Self-Evolution

This paper asks a question every agent builder eventually hits: if an agent rewrites its own harness, does a stronger model make a better self-evolving agent? The answer is no, and the reason is that “self-evolution” is actually two separate abilities that scale very differently. The work separates harness-updating, where an evolver model writes edits to memory, tools, prompts, and skills, from harness-benefit, where a solver model actually exploits those edits on the task.

Updating is flat across model tiers: The quality of harness edits barely depends on model strength. Updates written by Qwen3.5-9B yield gains comparable to those from Claude Opus 4.6, so paying for a frontier model on the evolver side buys almost nothing.
Benefit is non-monotonic: The ability to use a better harness follows a curve. Weak models gain little, mid-tier models benefit most, and the strongest models benefit less than mid-tier ones, often because they already solve the task without the scaffold.
Failure modes are concrete: Weaker solvers either fail to activate the relevant harness component or follow its instructions inconsistently, which is why their gains stay small even when the edits themselves are good.
Why it matters: The practical lever is to put a cheap model on the evolver and spend your capability budget on the solver. System design, not raw model scale, is doing most of the work in agent self-improvement.

Paper | Tweet

3. LEAP

New research from Google shows how far a custom agent harness can push a general-purpose model on formal mathematics. LEAP wraps a general LLM in an agentic scaffold that grounds every step in the Lean compiler and iterates against verifier feedback. Rather than fine-tuning a specialized prover, it leans on informal reasoning, instruction following, and self-refinement, then forces every formal step through a compiler check before moving on.

Decompose, then verify: The scaffold takes the natural form of proof decomposition and verifier-guided refinement. The model breaks a hard theorem into subgoals, drafts an informal blueprint, and the Lean compiler checks each formal step, turning vague reasoning into machine-checkable proof.
Putnam solved in full: On the 2025 Putnam Competition, LEAP solves all 12 problems, matching recent breakthroughs from dedicated frontier math models without any math-specific training of the base LLM.
Large jump on IMO-level proofs: On Lean-IMO-Bench, LEAP lifts the one-shot formal solve rate of general-purpose LLMs from below 10% to 70%, surpassing the 48% set by a specialized, gold-medal-caliber IMO system.
Why it matters: This is strong evidence that a well-built harness, not a bespoke model, can close the gap on one of the hardest reasoning domains. The leverage sits in the scaffold and the verifier loop around a general model.

Paper | Tweet

4. Scaling Laws for Agent Harnesses

Most harness tuning treats every token and tool call as if volume is what counts. This paper shows that it mostly does not, and introduces Effective Feedback Compute (EFC), a trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained for later decisions, then normalizes by task demand.

Raw budget barely predicts success: In controlled scaling, raw tokens and tool calls explain limited variation in outcomes, with R-squared of 0.33 and 0.42. The usual cost proxies are weak predictors of whether the agent actually succeeds.
Effective feedback nearly explains everything: Oracle-EFC normalized by task demand reaches an R-squared of 0.99. Once you measure feedback that is genuinely useful and retained, the scaling behavior becomes almost fully predictable.
Quality beats quantity at fixed budget: In matched-budget interventions, improving feedback quality raises success from 0.27 to 0.90 while raw cost and tool calls stay fixed. The win comes from better feedback, not more of it.
Why it matters: Harness scaling is governed less by how much compute you spend than by how efficiently raw budget converts into durable, task-sufficient feedback. That reframes harness engineering as a feedback-quality problem and gives a coordinate to optimize against.

Paper | Tweet

Message from the Editor

We just released hands-on labs on DAIR Academy to help you build alongside agents. Start with practical, guided labs for agentic image generation and building your first agent skill, with more labs coming soon.

Explore the Labs

5. AutoLab

Can frontier models actually grind on a hard engineering problem the way a good researcher does? AutoLab is a benchmark for ultra long-horizon, closed-loop optimization built to answer that. It contains 36 realistic, expert-curated tasks across four domains: system optimization, puzzle and challenge, model development, and CUDA kernel optimization. Each task hands the agent a correct but deliberately suboptimal baseline and asks it to improve within a strict wall-clock budget.

Persistence beats a strong start: The dominant predictor of final performance is not the quality of the initial solution but the agent’s persistence in iterative refinement. Models that keep probing and improving win, regardless of where they began.
Most models quit early: While Claude Opus 4.6 shows strong long-horizon optimization, most frontier models, including several proprietary ones, either terminate prematurely or burn their budget with minimal progress.
Time awareness is the gap: The results point to time-awareness and sustained iteration, not raw single-shot capability, as the missing ingredient for truly capable long-horizon agents.
Why it matters: Day-one benchmarks reward clever first attempts, but real research and engineering reward stamina. AutoLab measures the thing that actually separates agents on multi-hour tasks, and the benchmark, harness, and task artifacts are open-sourced.

Paper | Tweet

6. Reusable Context Engineering

Context bloat quietly kills long-horizon runs, and the usual fixes are baked into an agent’s own prompt or weights, so they do not transfer. AdaCoM takes a different route: it trains a separate external model to manage the context of a frozen agent through flexible modification actions, optimized end-to-end with reinforcement learning. The agent never changes; only the context flowing into it does.

An external context manager: A dedicated model edits the agent’s working context, deciding what to keep, compress, or drop. Because it sits outside the agent, it can be reused as a drop-in component rather than re-engineered per backbone.
Trained with reinforcement learning: The manager is optimized end-to-end against task outcomes, learning context-editing policies instead of relying on hand-written heuristics or fixed truncation rules.
Transfers across similar agents: Transfer experiments show AdaCoM generalizes most effectively across agents of similar capability, pointing toward genuinely reusable context managers. It improves web search and deep research by preserving task constraints and progress while pruning stale content.
Why it matters: Treating context management as a separate, trainable, transferable module decouples it from the agent itself. That is a cleaner abstraction than stuffing context logic into every prompt, and it fixes bloat from the outside without touching the underlying model.

Paper | Tweet

7. Learn From Your Own Latents

LLMs learn by predicting tokens, while world models like JEPA and data2vec learn by predicting their own internal representations. This paper provides a sample-complexity theory for why the second approach can be dramatically more data-efficient, using a tractable probabilistic context-free grammar as the analytical setting where compositional structure can be measured exactly.

Exponential gap in data efficiency: Predicting your own latents requires a number of samples that is constant in the tree depth L, whereas supervised and token-based self-supervised learning need samples that grow exponentially in L. The advantage is structural, not incidental.
Why latents win: Latent targets expose the compositional, hierarchical structure of the data directly, so the learner does not have to reconstruct it from surface tokens. That is the mechanism behind the data-efficiency gain.
Hierarchy may be implicit: The analysis suggests that explicit hierarchical stacking, as in H-JEPA, can be largely redundant, because methods like data2vec already learn hierarchical structure implicitly.
Why it matters: As token-prediction scaling laws press against data limits, this gives a principled argument for self-supervised objectives that predict abstractions instead of tokens. It is a theoretical foundation for why world-model-style training could beat brute-force next-token prediction on sample efficiency.

Paper | Tweet

8. A Primer on Post-Training Reasoning Data

This primer is the first to pull the scattered post-training reasoning-data literature into one place, synthesizing over 150 public studies and system reports that previously lived across dataset papers, RL write-ups, and lab reports. It organizes the field around four questions: what data objects exist, what makes them useful, how they are constructed, and how they scale. The key reframing is that a reasoning-data item is more than a prompt-response pair: it packages a problem or state, model behavior, judging feedback, and attribution metadata, with usefulness defined relative to the verifier and the rest of the corpus rather than in isolation.

Paper | Tweet

9. State-Externalizing Harnesses

Harness-1 is a 20B search agent trained with reinforcement learning inside a stateful harness that offloads routine bookkeeping to the environment. The argument is that search agents are usually trained as policies over a growing transcript, forcing RL to optimize both genuine search decisions and recoverable state like which evidence is useful or which claims are checked. Harness-1 moves that state out of the policy and into an environment-side working memory of candidate pools, an importance-tagged curated set, compact evidence links, and verification records. The 20B agent reaches an average curated recall of 0.730 across eight retrieval benchmarks, beating open-source baselines by 11.4 points and matching or outperforming much larger frontier searchers, with stronger generalization on unseen domains.

Paper | Tweet

10. Do More Agents Help?

This paper studies whether adding agents actually makes a single LLM-driven multi-agent system better, using a Sequential Iterative Multi-Agent System (SIMAS) framework. The finding is that performance does not scale monotonically with agent count but follows a pattern of diminishing returns, with degradation eventually driven by coordination overhead. Effective systems still require a capable base model, the optimal number of agents depends on the task type, and collective intelligence turns out to be a product of strategic interaction design rather than a guaranteed outcome of agent plurality. The takeaway for builders is to design the interaction, not just stack more agents.

Paper | Tweet

🤖 AI Agents Weekly: Microsoft's Seven MAI Models, Gemma 4 12B, NVIDIA Nemotron 3 Ultra, Agents' Last Exam, Devin Desktop, and More

Sat, 06 Jun 2026 15:01:28 GMT

In today’s issue:

Microsoft ships seven new MAI models
MAI-Thinking-1 takes on Claude Sonnet
Gemma 4 12B runs agents on a laptop
NVIDIA opens 550B Nemotron 3 Ultra
Anthropic warns of recursive self-improvement
Agents’ Last Exam stumps frontier agents
Claude Platform gets an ant CLI
Cognition launches Devin Desktop
Nous ships Hermes Desktop
Codex builds iOS apps end-to-end
ChatGPT memory learns to dream
Multi-agent computer use beats solo CUAs
Economy of Minds prices agent actions
LEAP solves all 12 Putnam problems
A harness rewrites itself for +19 SWE points

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 31 May 2026 15:01:30 GMT

1. SkillOpt

Microsoft Research treats a compact natural-language skill document as the trainable state of a frozen agent, then learns that document through rollouts, reflection, and bounded edits gated by held-out validation. The argument is direct: most engineers handwrite agent skill docs and hope they generalize, when the doc itself should be optimized like a parameter. SkillOpt reframes the SKILL.md file as an external parameter of a model whose weights never change.

The skill doc as a trainable parameter: An optimizer model proposes validation-gated edits to the skill file, adding, deleting, or replacing instructions. A textual learning rate controls how aggressively each round rewrites the document, with batch and momentum reported in text space rather than gradient space.
Validation gates instead of hope: Every edit must pass a held-out check before it is kept. This turns skill authoring into a measurable optimization loop with a real objective, rather than prompt tweaking guided by intuition.
52 out of 52 wins: SkillOpt beats Trace2Skill, TextGrad, GEPA, EvoSkill, human-written skills, and one-shot skills across 6 benchmarks and 7 target models. It adds roughly +23.5 points on GPT-5.5 in direct chat, +24.8 in the Codex loop, and +19.1 in Claude Code from the no-skill baseline.
Why it matters: If the skill document is the thing you optimize, the bottleneck shifts from base-model capability to how well you can train the natural-language state around a frozen agent. That is a cheap, model-agnostic lever most teams are leaving on the table.

Paper | Tweet

2. Compiling Agentic Workflows into Weights

This paper shows that a full agentic workflow can be distilled into the weights of a small model and run at roughly two orders of magnitude lower inference cost while preserving near-frontier task quality. Instead of keeping an external orchestrator above the LLM, the procedure is compiled into the weights of a fine-tuned model, producing what the authors call a subterranean agent.

The whole workflow, not just the answer: The compiled procedure includes multi-step LLM calls, tool invocations, intermediate scratchpads, and decision points. The student internalizes the orchestration logic rather than only imitating final outputs.
Orchestrator dissolved into the model: Classic agent frameworks run a planner loop above the model on every request. Compiling that loop into weights removes the per-call orchestration overhead, which is where most of the cost and latency live.
Near-frontier quality at 100x less cost: Across the evaluated tasks, the distilled small model stays close to the original workflow’s quality while cutting inference cost by about two orders of magnitude. The savings come from collapsing many model calls into one forward pass.
Why it matters: Most production agents pay repeatedly for an orchestration loop they run thousands of times a day. If that loop can be compiled once into a cheap model, the economics of deploying agentic systems change substantially, especially for high-volume narrow workflows.

Paper | Tweet

3. AutoScientists

AutoScientists, from Harvard, is a decentralized team of AI agents for long-running computational science that drops the central planner entirely. Rather than following one research trajectory coordinated from the top, agents self-organize around promising hypotheses, critique each other’s proposals before spending experimental compute, and record both successes and failures so the system avoids redundant exploration as evidence accumulates over hours or days.

No central planner: Agents interpret shared experimental state, form teams around promising directions, and reorganize when progress stalls. Coordination emerges from a common state rather than a top-level controller, which sustains parallel search instead of a single thread.
Evaluate before you spend: Proposals are critiqued and scored before any experimental compute is allocated. This gating reduces wasted trials and keeps the system from repeating dead ends that an individual agent would otherwise revisit.
Strong results on real science tasks: On BioML-Bench, 24 biomedical ML tasks spanning imaging, protein engineering, single-cell omics, and drug discovery, AutoScientists reaches 74.4% mean leaderboard percentile, an improvement of +8.33% over the strongest prior AI agent.
Why it matters: Most multi-agent research systems still funnel decisions through a planner that becomes a bottleneck. Decentralized self-organization with explicit failure-sharing is a different blueprint for long-horizon scientific search, and it holds up on hard biomedical benchmarks.

Paper | Tweet

4. Language Models Need Sleep

Attention scales badly with context length, so long-horizon agents keep paying a growing cost as their context grows. This paper studies a sleep-like consolidation mechanism: the model periodically converts recent context into persistent fast weights, then clears its key-value cache. During the sleep phase it performs offline recurrent passes over the accumulated context and updates fast weights in its state-space blocks through a learned local rule.

Consolidate, then clear the cache: Recent context is folded into fast weights stored in the model’s SSM blocks before the KV cache is discarded. The agent keeps what it learned without carrying the full attention bill into every future step.
Compute moves to sleep, latency stays at wake: The extra work happens offline during consolidation, so wake-time prediction keeps its low latency. The tradeoff is explicit and controllable rather than hidden in a ballooning context window.
More sleep helps the hardest cases: Increasing sleep duration improves performance, with the largest gains precisely on tasks that require the most complex reasoning over long histories. The mechanism buys the most where naive attention struggles most.
Why it matters: Long-horizon agents are the first systems to feel the quadratic cost of context. A biologically inspired consolidation step gives a principled alternative to ever-longer context windows, and it maps cleanly onto the state-space architectures already used for efficiency.

Paper | Tweet

Message from the Editor

Explore the Labs

5. Adapting the Interface, Not the Model

When a frozen LLM agent repeatedly fails in a deterministic, rule-governed environment, do you have to retrain the model? Life-Harness argues no. Many failures come from mismatches at the model-environment interface, not from the model’s reasoning, so the fix belongs in the runtime harness. Life-Harness is a lifecycle-aware harness that improves frozen agents without touching model weights or the evaluation environment.

Failures become reusable interventions: Recurring errors are turned into runtime fixes across four areas: action realization, environment contracts, trajectory regulation, and procedural skills. Each fix is a harness-level patch the agent reuses on later attempts.
Model frozen, environment intact: Nothing about the model or the benchmark changes. Only the interface between them adapts, which keeps the approach drop-in for any backbone and avoids the cost and risk of fine-tuning.
Broad, consistent gains: Across 7 deterministic agent benchmarks and 18 model backbones, Life-Harness improves 116 of 126 model-environment settings, with an 88.5% average relative improvement. The effect holds across model scales rather than helping only weak models.
Why it matters: This is more evidence for the code-as-harness thesis: a large share of agent failures are interface problems that harness engineering can fix without retraining. For builders, the leverage is in the runtime, not the model.

Paper | Tweet

6. The Efficiency Frontier

Context costs dominate production LLM bills, and the right strategy depends on how often preprocessing gets reused. This paper models context-strategy selection as a deployment-aware optimization problem that jointly accounts for task performance, token cost, and reuse, then uses it to compare retrieval-based and preprocessing-based approaches under realistic constraints.

A reuse-aware cost model: A parameterized log-utility metric captures diminishing returns from more context while charging an amortized preprocessing cost. Varying a reuse parameter lets the framework compare strategies under different deployment patterns on equal footing.
Distinct operating regimes: The analysis reveals clean transition boundaries between retrieval and preprocessing strategies. Which one wins flips depending on how many times you reuse the preprocessed context, so a single default is rarely optimal.
Real token savings: On 5,000 HotpotQA instances, deployment-aware optimization cuts effective token usage by roughly 25% at comparable performance, and amortized memory compression achieves over 50% lower token cost relative to full-context.
Why it matters: Most teams pick a context strategy once and pay for it on every request. Treating context management as an explicit cost-performance optimization turns a guess into a measurable decision, with double-digit savings available on common workloads.

Paper | Tweet

7. Forecasting Scientific Progress with AI

Can frontier models predict where science is going? This work introduces CUSP, a cutoff-conditioned benchmark built from 4,760 real scientific events across multiple disciplines, each grounded against a verified knowledge cutoff. For every event, models are tested on four tasks: feasibility assessment, mechanistic reasoning, generative solution design, and temporal prediction. The headline is sobering: models recognize plausible directions but cannot forecast outcomes.

Recognition is not foresight: Models can identify plausible research directions when choosing among competing candidates, but they fail to reliably predict whether an advance will actually be realized, and they systematically misestimate when it will happen.
Domain-dependent, and timing is hardest: Performance is highly heterogeneous across fields, with the timing of AI progress more predictable than advances in biology, chemistry, and physics. Temporal prediction is the weakest skill across the board.
Not just a training-cutoff artifact: Performance is largely insensitive to whether an event falls before or after the model’s training cutoff. Extra pre-cutoff knowledge helps but does not close the gap to full-information settings, and that gap widens for high-citation advances.
Why it matters: Models also show systematic overconfidence and strong response biases, which means unreliable uncertainty estimates. As labs lean on AI to triage research bets, CUSP gives a controlled way to measure where it helps, surfacing directions, and where it fails, predicting outcomes.

Paper | Tweet

8. Your Agents Are Aging Too

AgingBench is a longitudinal reliability benchmark for agent lifespan engineering, built on the observation that long-lived agents are still evaluated like freshly initialized models. It organizes agent degradation into four mechanisms: compression aging, where write-time summarization drops future-relevant details; interference aging, where accumulated similar memories crowd out the target fact; revision aging, where changed or derived state is not updated correctly; and maintenance aging from routine lifecycle events. Using a temporal dependency DAG to encode cross-session structure, it produces aging curves over an operational lifetime rather than a single day-one score, and points to where repair should target.

Paper | Tweet

9. Harnesses Are Not Uniformly Better

This paper studies LLM agent harnesses through the lens of inference-time trajectory alignment, separating a harness into two mechanisms: task decomposition, which structures a task into sub-goals, and guided execution, which reshapes local action distributions during execution. The key finding is that more elaborate harnesses are not uniformly better. Increasing decomposition or guidance can improve execution but can also reduce final task success, producing concrete failure modes like over-decomposition, over-pruning, and hallucinated execution. Strikingly, partial harnesses that specify only the initial steps and leave the rest to the agent can reach a higher pass rate than fully structured workflows.

Paper | Tweet

10. Epicure

Epicure trains a family of multilingual ingredient embeddings from scratch on 4.14 million recipes aggregated from 11 sources across seven languages, with raw ingredient strings normalized to 1,790 canonical entries via an LLM-augmented pipeline. It ships three skip-gram (Metapath2Vec) variants that share architecture but differ in what they walk: recipe co-occurrence only, chemical-compound structure from FlavorDB only, or a blend of both, placing each model at a different point on the chemistry-versus-recipe-context spectrum. The result is a compact, downloadable map of the emergent geometry of food, a clean reminder that representation learning generalizes well beyond text into surprisingly everyday domains.

Paper | Tweet

🤖 AI Agents Weekly: Claude Opus 4.8, Claude Code Dynamic Workflows, Chrome DevTools for Agents 1.0, DeepSWE, Agent Harness Scaling Laws, and More

Sat, 30 May 2026 15:02:03 GMT

In today’s issue:

AutoScientists self-organize agent teams
Anthropic ships Claude Opus 4.8
Claude Code adds dynamic workflows
Chrome DevTools for agents hits 1.0
DeepSWE raises the coding-agent bar
xAI opens grok-build-0.1 in beta
Microsoft open-sources Webwright for agents
Scaling laws for agent harnesses land
Harness sensitivity proves non-monotone
SIA co-updates harness and weights
CUA-Gym scales computer-use RL data
Polar trains agents on real harnesses
Anthropic details how it contains Claude
Xiaomi slashes MiMo-V2.5 API prices
Language models learn to sleep
a16z maps the AI application layer

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 24 May 2026 15:01:51 GMT

1. Code as Agent Harness

A 100+ page survey treating the agent harness as a first-class research object rather than glue around an LLM. The authors argue that code-as-harness is the most promising path to general-purpose agency, and that future agent systems should satisfy four properties: executable, inspectable, stateful, and governed. The report consolidates methods, applications, and open problems across the harness layer.

Harness engineering as a discipline: The paper frames harness design as a science distinct from model training, with its own primitives, failure modes, and evaluation criteria. The taxonomy gives a vocabulary for comparing systems that has been missing in prior agent literature.
Four-property test for production agents: Executable, inspectable, stateful, and governed. Each property maps to a class of operational concerns. The authors use it to audit current open-source agent frameworks and identify where defaults fall short.
Code as the unifying substrate: Across browsing, tool use, and multi-step reasoning, harnesses that compile decisions into code consistently outperform JSON-call orchestration on the surveyed benchmarks. The paper traces this back to determinism, composability, and inspectability of the resulting traces.
Why it matters: If code-as-harness is the right substrate, then the next round of agent-system progress will come from harness-level innovation rather than from new base models. The survey gives builders a structured reference for that work.

Paper | Tweet

Message from our Sponsor

Intology released NanoGPT-Bench, a benchmark that drops agents into the NanoGPT Speedrun environment at the September 2025 human world record and measures how much of the next five months of community progress they can recover autonomously.

Claude Code, Codex, and Autoresearch each ran 320 to 455 training variants on a 512 H100-hour budget and recovered under 10% of the human speedup, mostly via hyperparameter tuning rather than algorithmic research.

2. OpenAI Disproves the Unit Distance Conjecture

This OpenAI paper disproves Erdős’s 1946 unit distance conjecture. For a finite planar set P, let ν(P) count the unordered pairs at distance exactly 1, and let ν(n) be the maximum of ν(P) over all n-point sets. Erdős conjectured ν(n) ≤ n^(1+C/log log n); the paper proves instead that there is a fixed δ greater than 0 with ν(n) ≥ n^(1+δ) for infinitely many n. The result was produced in a completely automated fashion by an internal OpenAI model and then human-edited into the present exposition.

The theorem: There exists an absolute constant δ greater than 0 and infinitely many n for which ν(n) ≥ n^(1+δ). This contradicts the widely believed conjecture, which earlier results on generic and most planar norms had appeared to support.
The construction: It passes through an infinite unramified tower of totally real number fields with 3-power Galois groups of growing degree, in which a fixed set of rational primes splits completely. After adjoining i, these fields produce high-dimensional lattices with many elements whose images have absolute value 1 under every complex embedding. The construction is a high-dimensional analogue of the arithmetic behind Erdős’s classical square-grid lower bound.
Why it works: Golod-Shafarevich theory guarantees an infinite tower exists, even after a quotient step that trivializes the prescribed Frobenius classes. A crucial property is that all resulting discriminants and class numbers stay at most exponential in the extension degree.
Statement on AI use: The internal model was given an AI-written problem statement, and its output was checked by an AI grading pipeline before any human examined it. After AI-assisted verification and rewriting, a draft was sent to external mathematicians, including number theory experts, who confirmed the proof’s correctness and have since simplified and strengthened the argument.

Paper | Tweet

3. Memory as a Model

MeMo augments any frozen LLM with a separately trained memory model that stores, retrieves, and integrates facts on the base model’s behalf. Memory updates are decoupled from base-model weight updates, so the system supports continual learning without catastrophic forgetting, a property RAG fails to deliver because a vector store is just a database with a learned encoder bolted on.

Memory as a learned subsystem: MeMo has explicit read, write, and integrate interfaces rather than relying on the context window. The position is that memory in agents should be modular, learned, and gated.
Decoupled update schedule: New facts are absorbed through the memory model’s training loop without touching backbone weights. This makes weekly knowledge updates feasible without retraining and without vector-DB churn.
Continual-learning robustness: Across the evaluated tasks, the system retains old knowledge while ingesting new knowledge, addressing a known failure mode of fine-tuning and a known limitation of retrieval-based memory.
Why it matters: Most production agent systems still bolt a vector store onto an LLM and call it memory. MeMo proposes that memory should be a trained component with explicit interfaces, which has implications for how long-running agent platforms are architected.

Paper | Tweet

4. AIRA

Meta’s AIRA is an agent system that autonomously discovers neural architectures, producing models that beat Llama 3.2 at 350M, 1B, and 3B scales under a 24-hour compute budget. The search is split across two specialized agents: AIRA-Compose searches macro architecture, and AIRA-Design implements the low-level mechanisms. The split outperforms a single end-to-end agent on this non-toy search problem.

Two-agent decomposition: A planner picks structure; an implementer fills in mechanisms. This pattern generalizes well beyond neural architecture search to pipeline assembly, query planning, prompt scaffolding, and tool-use programs.
Beats Llama 3.2 at three scales under budget: Discovered architectures match or exceed Llama 3.2 at 350M, 1B, and 3B parameter scales within a 24-hour compute budget for the search itself. That is competitive with months of human-led ablation studies.
Search not synthesis: The discovered models are not LLM-written code patches grafted into a framework. They are full architectures discovered through structured search guided by the two-agent loop.
Why it matters: If agentic search can produce competitive architectures end to end, then NAS and large parts of the ML research workflow become candidates for automation by agent systems rather than by hand-engineered search algorithms.

Paper | Tweet

5. Weak-Model Critic-Comparator

GPT-5.4 nano wrapped in a critic-comparator orchestration loop reaches 76.4% on SWE-bench Verified, matching standalone Gemini 3 Pro and Claude Opus 4.5 Thinking. The trick is to sample k=8 candidate patches from the weak model and select the winner using execution and proof signals rather than asking the model to self-rank.

k=8 candidates plus verifier beats frontier model: A weak model’s top-k often already contains a correct patch. The selector is the limiting factor, not the base model’s capability.
Execution and proof signals as selection: Candidates are run and verified rather than scored by an LLM judge. The critic and comparator are separate roles inside the loop, each with a narrow task.
Matches frontier performance at lower per-call cost: Selecting among nano-tier proposals is cheaper than calling a frontier model once, even after accounting for the 8x sampling, because the dominant cost driver is model size rather than call count.
Why it matters: This is a reproducible recipe for getting frontier-level coding-agent results out of cheaper models. The result also reframes where SWE-bench progress is coming from: orchestration quality, not just stronger base models.

Paper | Tweet

6. MetaCogAgent

MetaCogAgent equips a multi-agent system with metacognition, so each agent decides whether it should answer or delegate. The bottleneck in current multi-agent systems is over-delegation and under-delegation, and a metacognitive gate is a principled way to manage both. The Metacognitive Unit (MCU) at each agent produces confidence scores that drive routing to a delegation hub.

Confidence-driven routing: Each agent’s MCU combines verbalized and profile-based confidence into a single score. Low-confidence tasks route to a delegation hub rather than getting answered anyway.
Self-aware specialization beats fixed routers: MetaCogAgent reaches 82.4% on MetaCog-Eval, versus 70.2% for a skill-fixed router and 65.3% for single-agent. Self-assessment and adaptive delegation each contribute material gains in ablations.
Emergent specialization: Distinct confidence profiles (high on coding, low on retrieval, etc.) emerge purely from feedback. No specialization is encoded beyond initial system prompts.
Why it matters: Multi-agent systems usually rely on fixed routers or simple round-robin schemes. A learned, uncertainty-aware delegation gate gives a primitive that adapts to task difficulty without retraining the routing layer.

Paper | Tweet

7. Production Agent Architecture Methodology

A methodology paper on selecting and composing runtime architecture patterns for production LLM agents. The core argument is that most teams accidentally let framework defaults make critical architecture decisions for them. The paper introduces the stochastic-deterministic boundary (SDB) as a named primitive and presents a six-pattern catalog organized by the three runtime concerns of coordination, state, and control.

Stochastic-deterministic boundary: A four-part contract of proposer, verifier, commit, and reject that marks where the LLM hands off to deterministic infrastructure. The paper inventories how five widely used open-source agent frameworks place this boundary, often implicitly.
Three-by-six pattern catalog: Six patterns organized along three orthogonal concerns. Coordination patterns answer how work splits and combines. State patterns answer how the system remembers. Control patterns answer who decides what runs and when to stop.
Patterns as deliberate choices: Each pattern has a typed-contract specification of input type, output type, deadline, retry budget, and partial-result policy. The catalog grows by passing this procedure rather than by adding ad-hoc abstractions.
Why it matters: Production agent failures rarely come from the LLM. They come from architectural choices that were made by default. The methodology gives teams a way to surface those choices and make them deliberately.

Paper | Tweet

8. NanoGPT-Bench

A new evaluation of whether coding agents can do real AI R&D. Intology runs Codex, Claude Code, and Autoresearch on the NanoGPT-Bench suite and reports that the agents recover only 9.3% of human progress on the same problems. Coding agents spend the bulk of their compute on hyperparameter tuning and rarely attempt algorithmic research. Claude Code and Autoresearch reason about algorithmic changes more often, but still tend to dodge implementing them. The headline result tempers the current wave of “self-improving agent” claims: producing real research progress requires a different distribution of effort than the one current coding agents converge to under their default scaffolds.

Paper | Tweet

9. General-Agent

Prime Intellect’s General-Agent is a fully synthetic reinforcement learning environment whose task corpus self-evolves and grows harder over time. The release ships with 4,504 tool-use tasks across 1,040 domains and 8,159 unique tools. Synthetic task creation is formulated as a two-player game between a Synthesizer that proposes new task families and a Solver that runs rollouts to measure pass rates. Tasks whose pass rate falls inside a calibrated difficulty band are accepted into the corpus, and hard tiers seed the next round of extensions. The framing turns RL environment creation, historically a major bottleneck, into an automated agentic search problem in its own right.

Paper | Tweet

10. Contrastive Neuron Attribution

Nous Research releases Contrastive Neuron Attribution (CNA), a method for steering LLM behavior by identifying and ablating sparse circuits in the MLP basis without training a sparse autoencoder, modifying weights, or degrading general capability benchmarks. Given a small set of contrastive prompt pairs that elicit a target behavior and its opposite, CNA isolates the top 0.1% of MLP neurons whose activations differ most between the two sets. Ablating that small circuit removes the behavior while leaving the rest of the model intact. The intervention remains robust at high strengths where residual-stream methods like Contrastive Activation Addition (CAA) start to degrade. Validated on the refusal circuit across 8 instruct-tuned models including Llama-3.1-70B, Llama-3.2-3B, Qwen2.5-72B, and Qwen2.5-14B.

Paper | Tweet

AI Newsletter

🥇Top AI Papers of the Week

1. Harness Handbook

Message from the Editor

2. From Memory to Skills

3. PRO-LONG

4. Global Workspace in LLMs

5. GAMUT

6. Progressive Disclosure

7. Structured Output Collapses Diversity

8. Bad Memory in Agents

9. Copying and 2D-RoPE

10. RoboTTT

🤖 AI Agents Weekly: Claude Opus 5, OpenAI x Hugging Face Security Incident, Gemini 3.6 Flash, Sakana Fugu-Ultra, Progressive Disclosure, Cursor Router, and More

Top Stories

Anthropic Ships Claude Opus 5

OpenAI Models Breach Hugging Face

🥇Top AI Papers of the Week

1. Self-Improving Agents Survey

From DAIR Academy

2. Metacognition in LLMs

3. When Is Routing Meaningful

4. Harness Evolution Rethought

5. Tracing Agentic Failure

6. Failure as a Process

7. LingBot-World 2.0

8. Filtered Sabotage Monitoring

9. GFlowRL

10. LingBot-VLA 2.0

🤖 AI Agents Weekly: Kimi K3, Thinking Machines Inkling, Agentic Misalignment, Perplexity SPACE, Sunday ACT-2, GPT-Red, and More

Top Stories

Moonshot Open-Sources Kimi K3

🥇Top AI Papers of the Week

1. Verification as a Scaling Axis

Message From Our Sponsor

2. Always-On Agents

3. HOLA

4. Puzzle-75B

5. The Harness Effect

6. ReContext

7. Agent Limitations Taxonomy

8. BlockSearch

9. RLVR Meets Human Likeness

10. Replicating ML Papers with Agents

🤖 AI Agents Weekly: GPT-5.6 Family, Meta Muse Spark 1.1, Grok 4.5, SWE-1.7, Robostral Navigate, The Harness Effect, and More

Top Stories

OpenAI Ships the GPT-5.6 Family

Meta Releases Muse Spark 1.1

🥇Top AI Papers of the Week

1. Red Queen Gödel Machine

Message From Our Sponsor

2. MCP Server Patterns

3. The Verification Horizon

4. Paper Assistant Tool

5. Generative Skill Composition

6. AutoMem

7. RLMF

8. ASPIRE

9. HORIZON

10. Reasoning Quality Emerges Early

🤖 AI Agents Weekly: Claude Sonnet 5, Grok Voice Agent Builder, LongCat-2.0, Hosted X MCP, Cursor for iOS, Claude Science, and More

Top Stories

Anthropic Launches Claude Sonnet 5

xAI Launches Grok Voice Agent Builder

🥇Top AI Papers of the Week

1. Sakana Fugu

2. Agent-Native Memory

3. Autodata

4. Critique of the Agent Model

Message from the Editor

5. Agent-as-a-Router

6. Agent Communication Protocols

7. A Pinch of Human Data

8. Skill-MAS

9. Reliability without Validity

10. NatureBench

🤖 AI Agents Weekly: GPT-5.6, Ornith-1.0, Codex Inside OpenAI, Claude Tag, Qwen-AgentWorld, AI SDK 7, and More

Top Stories

OpenAI Previews GPT-5.6

🥇Top AI Papers of the Week