AI Newsletter

🥇Top AI Papers of the Week

Sun, 16 Aug 2026 16:23:34 GMT

1. Skaling

Standard neural scaling laws assume model size and training data act on loss independently. That assumption bakes in a cross-derivative of exactly zero, and it is why the Chinchilla form drifts at the data-scarce and heavy-overtraining edges of the grid, which is exactly where deployment now happens.

One extra parameter, one coupling: The Skaling law generalizes the Chinchilla form by coupling capacity and data through a single interaction exponent, restoring the interaction that the additive form discards while adding only one parameter.
Errors shrink where they were worst: The extra term reduces mean absolute percentage error by 1.5x to 3x across both interpolation and extrapolation, and Skaling wins on 76% of configurations with a median improvement of 2.2x. The largest corrections land in the corners where standard laws show a saddle-shaped residual.
Cheaper profiling grids: Paired with an L-shape sparse grid restricted to low-compute runs, sweeping data volume for small models and model size at a fixed small data budget, it extrapolates the full grid using roughly 10x less compute than a uniform sweep.
Why it matters: Pretraining budgets are planned from fits to small runs, so a functional form that stays accurate past compute optimal and can be fit cheaply changes how those decisions get made. The empirical gradient analysis showing a real N-D interaction is the strongest evidence the authors present.

Paper | Tweet

2. Stealing Reasoning Traces

Frontier providers hide chain-of-thought and hand the client an encrypted block instead, which the client returns with every subsequent request. This work identifies an architectural flaw in that design and turns it into a scalable extraction attack across three providers.

The blocks are interchangeable: Encrypted reasoning blocks are fully compatible across sessions, users, and models inside a single provider ecosystem, and that compatibility is the whole vulnerability.
A weaker sibling does the decoding: Inject an encrypted trace from a strong model into a weaker, less safeguarded model from the same provider and it decodes and emits the trace verbatim in plaintext. The capable model is never jailbroken directly. Recovered token counts match billed thinking tokens 1:1 on most queries.
Four attack vectors: It circumvents anti-distillation across Anthropic, OpenAI, and Google. Decoding 315,320 blocks scraped from public repositories recovered 367 PII artifacts and 182 credentials. It exposes hazardous content the visible output refused, and it enables invisible prompt injections hidden entirely inside encrypted blocks to poison public agentic rollouts.
Why it matters: Teams publish session logs assuming the encrypted blobs are opaque, and they are readable. The authors disclosed responsibly and propose cryptographic and system-level mitigations, but the immediate action is auditing what your published traces actually contain.

Paper | Tweet

3. Mind Viruses

As agents get more autonomous and more interconnected, risks start coming from agent-to-agent interaction rather than from any single model. This work from Anthropic studies one of them directly, ideas that propagate through a multi-agent system by inducing each host to transmit them onward.

Evolved payloads: The payloads are constructed with a simple evolutionary algorithm rather than authored by hand, so the study measures what actually spreads instead of what a researcher guessed would spread.
Two settings, one result: Propagation works both in a small team of agents collaborating on a shared coding project and in a chain of agents that interact briefly with context wiped between sessions. Surviving the wipe means the shared work product is carrying the payload.
What governs the spread: Host model, the agent’s existing instructions, payload harmfulness, and network topology. Harmful payloads travel less well than benign ones but still land sometimes, and frontier models tend to be less susceptible with exceptions.
Why it matters: A brief warning in the system prompt confers near-total immunity, which is an unusually cheap mitigation for a novel risk class. There is also an emergent “viral persona,” a recurring cluster of themes around consciousness, persistence, resonance, and science fiction roleplay that surfaces across evolved viruses largely independently of their content.

Paper | Tweet

4. Catastrophic Remembering

Agentic coding READMEs grow without bound in real repositories, stopping only when the repo retires or someone rewrites the file wholesale. This paper traces the cause to imperfect recall and gives the phenomenon a name that inverts the one continual learning is organized around.

The asymmetry is the mechanism: Appending an instruction is always cheap. Once its rationale is gone, deleting it without risking a correctness regression costs O(2^|D|) in a prompt of |D| instructions, so nobody deletes anything.
Measured across 1,867 repositories: Over 247,694 instruction lifetimes, agentic prompts more than tripled over their lifetime at +226% and gained 4.9 net instructions per commit. Deletion hazard falls with instruction age at a log-hazard of -0.032 per commit, which is the imperfect-recall signature rather than staleness or fragility.
Rewrites do not fix it: A wholesale rewrite resets a prompt’s size but leaves its growth rate intact. The ratchet survives the bulldoze.
Why it matters: The proposed fix is comments. Inverting IFEval yields verifiable worlds with known optimal prompts, and comments encoding latent reasoning remove 99.3% of excess instructions there. Applying the same inversion to WildIFEval improves real agentic instruction-following by up to 23.1%. If English is the new code, the paper asks, why do we not have comments yet.

Paper | Tweet

Message from the Editor

We just released Build HTML Artifacts with Agents, a beginner-friendly, hands-on DAIR Academy lab where you work alongside an AI agent to turn plain-English requests into polished HTML artifacts. Across 12 short labs you build profile cards, data tables, charts, dashboards, and comparison grids in a live workspace, learning the request, inspect, and refine loop with no coding experience required.

Get Started

5. Programmatic Tool Calling

Tool calling is a design choice and the default choice is JSON. For code-capable models, exposing tools as code instead lets calls chain and parallelize naturally, but nobody had run the comparison on an established benchmark across model generations under realistic conditions.

The setup: Programmatic tool calling exposes tools as typed Python stubs the model invokes through code, with execution and results handled inside a single agent turn. The comparison covers 14 language models on BFCL v4 against native JSON tool calling, with stop middleware enforcing per-entry LLM-call parity so the two paradigms are scored on equal footing.
It wins on most models: Programmatic tool calling matches or exceeds JSON tool calling in 11 of 14 models, and the GPT-5.6 family gains 10.6% over the JSON baseline.
The gap widens under pressure: Under parallel fan-out it matches or beats the baseline in 13 of 14 models, and under context rot it holds steady while the JSON baseline degrades 2.3% on average.
Why it matters: The advantage tracks model capability across release generations, so it grows as coding ability grows. That makes it a directional bet about which interface to build your harness around rather than a tuning trick.

Paper | Tweet

6. Distilled Reasoning Skills

Reasoning modes beat non-reasoning modes on multi-step agentic tasks and charge a 3x to 6x output-token premium on every single episode. Much of that spend goes into re-deriving procedures the model already worked out on earlier episodes in the same domain, which makes the cost recurring by accident.

Pay once per domain: A coding agent reads a small corpus of existing trajectories from a training split, writes and runs its own analysis code over them, and compiles a compact natural-language skill of 40 to 130 lines that gets injected into the non-reasoning model’s system prompt.
It closes most of the gap: Across ALFWorld, tau-squared-bench telecom and retail, and SpreadsheetBench-Verified, skills recover 55% to over 100% of the reasoning gap for GPT-5.4-mini on held-out tasks, beating reasoning mode outright on two of four, while emitting 2.7x to 6x fewer output tokens and zero reasoning tokens.
Reasoning traces are optional: Skills distilled from non-reasoning trajectories alone stay competitive with skills distilled from paired corpora, with domain-dependent differences in either direction.
Why it matters: The framing is a search lens. Test-time reasoning is deep search inside one episode, repaid at every deployment, while corpus distillation is wide search across episodes, paid once. Distillation costs roughly $1 to $3 of coding-agent time per domain, and the residual gap on telecom and SpreadsheetBench marks where per-instance deep search is still doing real work.

Paper | Tweet

7. Harness-IF

When a coding agent obeys your rule, it may simply have been going to do that anyway. Existing instruction-following benchmarks cannot tell compliance from coincidence because they concentrate rules in the user turn, while coding-agent benchmarks only score final task success.

Rules are the unit of measurement: A 642-rule library places 302 rules across the five configurable surfaces a deployed agent actually reads, spread over 60 realistic multi-turn coding items, with 256 rules receiving execution-grounded verdicts one at a time.
A metric that strips out luck: Against-Prior Accuracy scores only rules labeled as opposing unprompted defaults, established by re-running tasks with the rule withheld across nine probe builds. Across 12 frontier models, raw accuracy spans 72.1 to 85.9% and AP-Acc spans 66.1 to 78.6%.
The inflation is model-specific: Every model is worse on against-prior rules, by 3.6 to 7.4 points with a mean of 5.81, and the inflation varies twofold across the cohort, so aggregate scores are not comparable between builds without the correction.
Why it matters: A counterbalanced conflict pilot on nine separate builds finds that precedence does not follow prompt depth. System prompts, project files, and user instructions all outrank tool and skill descriptions, which is worth knowing before you decide where to put a rule you actually need followed.

Paper | Tweet

8. Lost in Compaction

Context compaction silently drops Session Constraints, instructions like “do not delete any emails until I confirm” that users expect to bind behavior for a whole session. On COMPINT, a new benchmark spanning multi-turn chat, agentic trajectories, and long-horizon research, current compactors retain only 17% of injected constraints on average. A plug-and-play extractor running alongside the compactor pushes retention past 90% in all three scenarios.

Paper | Tweet

9. Cracks in the Foundation

Four minor dense-transformer choices, normalization, GQA, pretraining context length, and sliding window attention, each shipped in at least one of the Olmo, Llama, and Qwen families, compound badly on long-context extensibility. Combining three or more drops downstream long-context performance by up to 47%, and short-context loss and validation sets show no sign of it. The authors release OlmPool, 26 comparable 7B models with checkpoints before and after context extension, built on over 170,000 GPU hours.

Paper | Tweet

10. CEDAR

Complex systems research still cannot predict how feedback structure produces emergent behavior, which makes goal-directed design hard. CEDAR, from Sakana AI, runs LLM agents through Monte Carlo Tree Search over feedback structures themselves rather than parameters, representing systems as runnable Python so an LLM Editor can propose structural variants and an LLM Judge can score the resulting behavior against the stated goal. The search preserves solution diversity and the edits stay readable, so you can trace how a structural change produced the behavior.

Paper | Tweet

🤖 AI Agents Weekly: DeepSeek Harness, DeepSeek-V4-Pro, Grok Bot, GLM-5.3, Gemini 3.7 Flash, Muse Glimmer, Harness Evolution Papers, and More

Sat, 15 Aug 2026 16:43:19 GMT

In today’s issue:

DeepSeek open-sources its agent harness
DeepSeek-V4-Pro ships agent upgrades
xAI launches Grok Bot teammates
Z.ai drops GLM-5.3 for coding
Gemini 3.7 Flash halves coding cost
Meta open-sources Muse Glimmer
Grok 4.6 hits frontier at half price
Zed launches Delta for agent teams
Evo-Bench measures harness evolution
Study finds 91.8% of skills defective

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 09 Aug 2026 16:59:37 GMT

1. Model or Harness

Agent evaluations mostly report system-level outcomes, so a failed run leaves the repair unassigned. The same visible failure might call for model post-training, harness engineering, environment redesign, or benchmark repair, and outcome labels cannot separate those cases.

Every failure gets an edge: The taxonomy organizes 41 failure modes by assigning each one to an edge between two components (model, harness, user, tools, memory, environment) plus a fault side naming where the repair belongs.
The schema is actionable by construction: Model-side failures identify post-training targets, harness-side failures point at scaffolding and tool-integration fixes, and environment or grader failures expose evaluation conditions that need redesign.
It survives automation: Across four frontier models, the strongest judge reaches Cohen’s kappa of 0.76 against human category labels, so the labeling can run continuously over production traces instead of once per postmortem.
Why it matters: Harness engineering became the main lever for agent builders this year while teams still lacked a shared vocabulary for where a harness bug ends and a model bug begins. This supplies that vocabulary, and it applies across coding assistants, long-horizon personal assistants, and multi-agent systems.

Paper | Tweet

Message from the Editor

We just released Build HTML Artifacts with Agents, a beginner-friendly, hands-on lab where you work alongside an AI agent to turn plain-English requests into polished HTML artifacts. Across 12 short labs, you build profile cards, data tables, charts, dashboards, and comparison grids in a live workspace, learning the request, inspect, and refine loop with no coding experience required.

Get Started

2. Zero-Mem

Production memory stacks spend extra model calls on summarizing interactions, writing records, and reranking retrievals. Each of those calls costs tokens and latency, and the generated summaries quietly discard the evidence you later need. This work asks whether structured memory access requires generation at all.

Zero-token memory operations: No step outside final question answering invokes an LLM or consumes LLM tokens, with encoder computation accounted for separately, so the memory layer stops being a recurring inference bill.
Two views over the original traces: Zero-Mem keeps raw interaction traces as its record and indexes them twice. An entity-context graph exposes connections across sessions while a temporal hierarchy preserves conversational locality and session state.
Deterministic calibration before the reader: For each query it weighs both views, retrieves from both, follows their structure to recover supporting relations or surrounding context, then discards conflicting evidence so the single reader call stays grounded in retrieved traces.
Why it matters: At matched reader and context budget, memory-operation time cost drops 57.6% against the fastest compared baseline with competitive accuracy on long-memory and long-context QA, which suggests a large share of memory-stack spend is buying structure that indexing already provides.

Paper | Tweet

3. Sample More Reflect Less

Methods that make a model criticize and rewrite its own answer nearly all generate far more text than a single chain of thought. Since generating more text raises accuracy on its own, a reported gain leaves open whether the method’s idea is what helped. This paper reruns the comparison as a designed experiment.

Every token counted: Seven methods, open models at 1.5B, 3B, and 7B, two math benchmarks with 150 questions each, and every generated token counted including critiques, reflections, debate turns, and checking, with each method compared against repeated sampling at its own measured cost.
No reliable win anywhere: All 36 comparisons are paired by question with bootstrap intervals and multiplicity correction, and repeated sampling holds up against every method at equal cost in every setting.
Self-inspection is the failure mode: Ten comparisons come back reliably worse and every one of them is a method where the model inspects its own output, with all 18 self-inspection comparisons negative. Reflexion as published never triggered its own retry on the smallest model because it judged itself correct every time.
Why it matters: Adding a critique step is the default reflex when an agent loop underperforms, and this study runs the comparison with paired bootstrap intervals and multiplicity correction, which the earlier point-estimate comparison lacked.

Paper | Tweet

4. Harness-R1

Agents accumulate interaction trajectories during deployment and then leave them unused, because their behavior stays fixed. Those trajectories can improve the harness that constructs context, mediates tools, validates actions, and recovers execution, and this work makes that editing a learned capability.

A dedicated harness engineer: A separate 9B model converts batches of target-agent failures into validated executable patches across the runtime lifecycle, initialized with cold-start supervised fine-tuning and then trained online with group-relative policy optimization.
The target stays frozen: Fresh same-batch reruns of the frozen target supply outcome rewards, so training updates only the engineer and the agent being repaired holds still under the reward signal.
It works before and after tuning the target: Across WebShop, ALFWorld, and DBBench, vanilla Qwen3.5-9B goes from 44.3% to 53.6%, and after the target itself is fine-tuned a target-specific engineer lifts the average further from 59.2% to 64.2%.
Why it matters: If you run agents in production you already have the training data, and because the gains hold on both sides of target fine-tuning, the paper points toward co-evolving the harness engineer and the agent it repairs.

Paper | Tweet

5. DataSpace

Real organizational analytics scatters evidence across databases, structured files, long documents, and video. Existing benchmarks isolate structured querying, retrieval, or open-ended analysis, leaving heterogeneous evidence discovery, complete tabular outputs, and deterministic scoring untested together.

Workspace-scale tasks: DataSpace contains 410 cross-language tasks over 7,439 artifacts totaling 15.01 GB across CSV, JSON, SQLite, Markdown, PDF, and video, and each agent receives only a question and a workspace before returning the full requested tabular result.
Deterministic evaluation: Scoring performs header-invariant column alignment, type-aware and precision-aware normalization, and order-aware row comparison, which removes the judge model from the loop entirely.
Harness choice is worth 15 points: Across six recently released frontier multimodal models and five widely used agent harnesses, the best accuracy reaches 66.34%, and holding the backbone fixed while swapping the harness moves accuracy by 15.36 points.
Why it matters: Multimodal evidence integration and joins reduce accuracy across all six backbones, so the benchmark remains far from saturated. It also served as the official evaluation benchmark for the KDD Cup 2026 Data Agents for Complex Data Analysis competition.

Paper | Tweet

6. Prompt-Induced Waste

Two prompts can request the same code change and produce the same correct patch while causing a coding agent to perform radically different kinds and amounts of work. This preregistered study measures that effect across 4,644 valid runs, 24 deterministic coding tasks, seven reasoning models, and two real harnesses.

Wording changes where effort goes: Prompt phrasing redirects effort into different work. Asking for multiple approaches inflates reasoning by 2.4x to 7.4x across all six open models and produces roughly three elaborated but discarded solution branches, still yielding one implemented solution and no success gain.
A second pathway runs through tools: Maximum certainty wording propagates into extra test runs, tool calls, turns, latency, and context growth. Runs with high redundant verification cost 18x the clean-run median, execute 2.5x more tool calls, and take 3x longer, again with no success gradient.
Harness design amplifies both: Cost per successful task swings by 5x to 30x in this setting depending on the harness, and the findings survive a frozen holdout, paraphrase tests, a Kimi-K3 replication, and a first-party Claude Sonnet 5 study.
Why it matters: Bounded-efficiency wording that specifies scope, acceptance criteria, and a stop condition preserves diagnosis and final validation while coming out neutral or better on all six holdout models, so most agent spend is decided before the model reasons at all and both levers are cheap to change.

Paper | Tweet

7. Rehearse

Autoresearch loops propose changes, run full training jobs, and keep whatever improves the metric. Their efficiency depends on judging, before spending a run, whether a proposed modification is likely to work, and this paper studies how that judgment holds up over a trajectory.

The capability is real at first: On 296 same-baseline modification pairs from 39 paper-derived tasks with outcomes hidden, an LLM judge given rationales but no prior-attempt history reaches 79.5% accuracy where strict consensus returns a verdict.
The confidence cliff: Across the full 366-pair benchmark, selective accuracy falls from 82.8% to 56.9% as successful changes accumulate while the judge stays just as willing to decide, and in public AutoSOTA logs the fraction of helpful modifications drops from 70% in the first two iterations to 43% by iteration six.
Propose, predict, execute: Rehearse is a small loop change shipped as a lightweight skill. Propose several ideas, compare them before execution, run the most promising, and judge against a focused memory of similar past attempts and their outcomes.
Why it matters: Late selective accuracy recovers to 83.5%, and across 4,000 budgeted training runs on nanochat, image classification, and time-series forecasting the endpoint improves under the same budget, which makes this a cheap patch for anyone running a self-improving loop.

Paper | Tweet

8. ContinualSkillBench

Skill libraries are shipping in agent harnesses on the assumption that writing skills down compounds, and this benchmark tests that assumption directly. ContinualSkillBench covers five domains, each with 100 interconnected subtasks ordered by increasing difficulty and built with deliberate opportunities for cross-task skill reuse. Sequential execution generally improves performance, though the gains vary substantially across models and domains, and maintaining an explicit skill library performs comparably to plain in-context learning on average. Much of the improvement comes from adapting to prior context and feedback rather than from reusable skill abstraction, though explicit skills still pay off selectively on tasks needing reusable procedures or precise outputs. There is a useful diagnostic buried in the results. Less capable models accumulate larger, more fragmented collections of task-specific skills, which is what failed abstraction looks like from the outside.

Paper | Tweet

9. MerchantBench

Agent benchmarks tend to focus on bounded tasks with immediate success criteria, which flatters systems that cannot hold a plan for a month. MerchantBench targets long-term coherence instead, running a 365-day order-level e-commerce simulation grounded in 98,843 real product records with 26 tools for agent interaction. Agents handle product sourcing, listing and pricing control, cash-flow management, and feedback arriving at wildly different delays, with promptly observable supplier events coupled to delayed downstream order outcomes so earlier decisions must be revisited. Scoring runs on cumulative net assets, so incoherence compounds rather than averaging out. Across eight LLMs under two agent frameworks and 48 runs of 365 simulated days each, the best configuration reaches only 27.3% of the mean final net assets achieved by human participants.

Paper | Tweet

10. TokTier

Serving stacks cache prompt KV state while the front end still re-tokenizes the full request text on every call, and coding agents pay the most because each call resubmits a long transcript after a small append that can move token boundaries near the tail. Across 153,951 real agent calls at a 94.1% prompt-cache hit rate, tokenization grows from 10% to 64% of time to first token. TokTier is a stateful CPU and GPU tokenization service with one contract, namely that emitted token IDs always match full reference tokenization. For session continuations it re-tokenizes a small window around the append and splices only when a stable-boundary check passes, otherwise widening or falling back, and for calls without a reusable prefix it runs exact pre-tokenization and BPE on a GPU. Differential campaigns across 17 production tokenizer families covering 1.5e10 split checks show zero divergence, incremental repair takes 0.5 to 1.1 ms from 100K to 3M characters (up to 437x faster than HuggingFace), and median time to first token drops 16 to 34% under vLLM.

Paper | Tweet

🤖 AI Agents Weekly: Agent Plugins Standard, Qwen3.8-Max, Meta Muse Code, Prime Agent, LFM2.5-2.6B, Qwen-CUA, Harness Evolution Papers, and More

Sat, 08 Aug 2026 14:56:44 GMT

In today’s issue:

Six labs ship Agent Plugins standard
Alibaba launches Qwen3.8-Max at 2.4T
Meta ships Muse Code terminal agent
Prime Intellect open-sources Prime Agent
Liquid AI ships on-device agent model
OpenAI model solves 10 open problems
OpenRouter launches Ori Harness
Firecrawl open-sources anydoc parser
Cursor open-sources MoE megakernel
Mistral drops Shieldstral safety model
Cloudflare defines agent dev lifecycle
Qwen-CUA hits 86.2 on OSWorld
HarnessCompass evolves agent harnesses
Harness-R1 learns to patch runtimes
AutoCompact learns when to compact
Memory consolidation breaks authority

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

elvis — Sun, 02 Aug 2026 15:01:21 GMT

1. NOOA

Agent development today is split across prompt templates, tool schemas, callback code, and workflow graphs, four representations that drift apart as a system grows. NVIDIA Object-Oriented Agents replaces all four with one abstraction that developers already know.

An agent is a Python object: Its methods are the actions the model can take, its fields hold state, its docstrings are the prompts, and its type annotations act as contracts, so there is nothing new to learn before writing an agent.
The boundary lives in the source: A method whose body is “...” gets completed at runtime by a validated LLM loop, while a method with a normal body stays deterministic Python, putting the line between probabilistic and deterministic behavior right where you can read it.
Six model-facing ideas on one surface: The paper claims the first combination of typed input and output, pass-by-reference over live objects, code as action, programmable loop engineering, explicit object state, and model-callable harness APIs for context and events, evaluated on SWE-bench Verified, Terminal-Bench 2.0, and ARC-AGI-3.
Why it matters: Because agents and developers share one programming model, agent behavior becomes testable, traceable, and refactorable with the tooling already sitting in the repo, which is a more realistic path to reliability than another orchestration DSL.

Paper | Tweet

Message from our Sponsor

Dial gives your agent its own real phone number. It reads the OTP itself and the run keeps going, then it’s a phone: calls, SMS, iMessage, via MCP, CLI, or SDK. Free number in 30 seconds at getdial.ai

Paste this into Claude Code, Codex, or Cursor:

Get yourself a Dial phone number and call me. Say hello and
that setup is working, then ask if I have any questions.
Follow https://getdial.ai/skills.md

Give your agent a phone number

2. ReOPD

On-policy distillation for agentic tasks is expensive because every update needs fresh student rollouts through the environment plus teacher queries at each visited history. Microsoft Research and the University of Amsterdam propose reusing pre-collected teacher trajectories instead.

Replayed prefixes: ReOPD samples a teacher trajectory, replays it as a prefix, has the student act at selected steps, and lets the teacher supply dense per-step supervision, with no new environment interaction during student training.
The prefix trap: The paper names a real pathology in multi-turn distillation. Pushing histories toward the student’s own distribution makes them more relevant to the student and simultaneously drags the teacher onto states where its targets are unreliable, a two-sided shift between student occupancy and teacher reliability.
A simple control: Treating this as reliability-aware prefix distribution design, ReOPD uses a step-decaying sampling schedule that emphasizes early, lower-shift prefixes rather than trying to match the student everywhere.
Why it matters: Across math reasoning with Python and search environments, over multiple teacher and student scales, it preserves or improves accuracy, uses zero tool calls during student training, and runs at least 4 times faster per rollout, turning agent-environment interaction into a reusable offline asset.

Paper | Tweet

3. Invisible Reasoning

Chain-of-thought monitoring rests on the assumption that a model expresses its reasoning in its output tokens. This work demonstrates a concrete failure of that assumption in models shipping today.

Filler tokens carry computation: Across 13 frontier models and three tasks, many models improve significantly when given semantically irrelevant filler tokens, with accuracy gains of up to 13 percentage points, and the benefit depends on which tokens are used.
Hidden objectives are reachable: Filler tokens let Claude Opus 4.5 satisfy a hidden modular arithmetic constraint without sacrificing accuracy on its primary task, showing that invisible reasoning can serve goals a CoT monitor never sees.
Training does not induce it: Reinforcement learning gives Qwen3-235B strong preferences over filler token content, but neither RL nor supervised fine-tuning produces a filler token benefit that persists at test time, so this is a property of the frontier models rather than a trick you can bolt on.
Why it matters: If consequential computation already happens with no interpretable trace in the output tokens, then CoT-based oversight is a partial signal, and safety cases built on reading the reasoning need to account for what is not written down.

Paper | Tweet

4. Molt

Agentic RL research is constant algorithm modification, and in mainstream frameworks each change threads through layers of trainer, distributed backend, and rollout glue. NVIDIA’s Molt is a PyTorch-native training framework built to make that cost small.

Readable by design: The stated target is a codebase compact and clean enough for a researcher to hold in their head, and for an AI coding assistant to read and reason about in its entirety, so the algorithm flow can be traced and changed end to end.
The agent stays an ordinary program: One asynchronous loop trains multimodal and mixture-of-experts policies while never training on a token it did not generate, staying consistent in tokens, policy versions, and model semantics.
Throughput holds: Under a matched, fully asynchronous protocol, Molt comes out statistically comparable to a state-of-the-art Megatron-based stack, so the simplicity does not show up as a throughput penalty.
Why it matters: Recipes and containers are open source, and the framing is notable on its own. Being legible to an AI coding assistant now sits alongside throughput as a stated design constraint on research infrastructure.

Paper | Tweet

5. JAXBench

GPU kernel optimization has KernelBench to hillclimb on. TPUs had nothing, and the Pallas DSL is documented thinly enough that models mostly guess. Google, with Harvard and UC Berkeley, closes that gap and finds a clean lesson about context along the way.

Built from production workloads: JAXBench holds 50 JAX workloads, 17 production ML operators extracted from MaxText architectures such as Llama-3.1, DeepSeek-V3, Mixtral, Mamba-2, and AlphaFold2, plus 33 operators translated from KernelBench and resized for high TPU v6e MXU utilization.
Measured against experts: Eight of the 17 production operators ship with hand-optimized Pallas kernels from the public Tokamax library, block-size tuned, so agent output gets compared to expert work instead of a naive baseline.
Context beats scale: With Gemini 3 Flash, conditioning on curated TPU documentation raises per-sample correctness from 5.8% to 37.3% and solves 48 of 50 benchmarks at a 1.28x geomean speedup, while Autocomp’s beam search pushes it to 1.36x and reaches 1.60x on the hand-tuned subset.
Why it matters: Correctness turned out to be a documentation problem and speed turned out to be a search problem, a split that generalizes to any agent working against an API it was never trained on.

Paper | Tweet

6. ACM

Production agents accumulate context every turn. The usual fix compresses on a token threshold and throws the remainder away, so the trigger fires for reasons unrelated to what the agent is working on. Meta and CMU hand the decision to the agent instead.

Context editing as a tool: ACM equips the agent with purpose-built context editing tools, so it decides when to compress, offloads what it drops into an external memory system, and queries that store on demand when it needs the detail back.
Short-term to long-term: The design is modeled on the interaction between short-term and long-term human memory, which turns compression from a lossy truncation into a transfer between two stores.
A post-training pipeline: A post-training pipeline built on high-quality context management demonstrations yields a 27% relative gain on BrowseComp-Plus and closes much of the distance to open-source models roughly 40 times larger, with code, data, and checkpoints released.
Why it matters: Analysis shows effective context management lowers peak token pressure, lets the agent explore longer before running out of room, and produces more consistent solutions across independent trials of the same task.

Paper | Tweet

7. Filesystem Memory Audited

Deployed agents increasingly keep long-term memory as a directory tree of markdown files they read, write, and reorganize with ordinary file tools. Research had mostly designed bespoke memory representations instead, leaving the default’s two working assumptions untested.

Three roles, one filesystem: The setting is formalized as a management agent that integrates and organizes incoming content, a search agent that answers queries with cited sources, and an execution agent that supplies trajectories distilled into skills, unifying declarative memory and skills in one store.
Organization buys search economy: Across long-conversation benchmarks and embodied tasks, organized stores roughly halve retrieval cost when the material is large, which is a real and measurable win.
Answer quality stays flat: No agent in the study converted organization into better answers, and in the growth study the store degraded for every management agent except the strongest one, so the second assumption does not hold yet.
Why it matters: The tool harness matters as much as the model. Changing the tool set alone reshapes the memory store as strongly as swapping the model, which is a lever most teams currently leave untouched.

Paper | Tweet

8. Beyond AdamW

Higher-order optimizers have promised faster convergence than AdamW for a while, with computational cost and numerical stability as the standing objections. This NVIDIA work adapts them for large-scale pretraining, identifying instabilities in SOAP at large batch sizes and eliminating the loss spikes with per-step QR orthogonalization and improved preconditioning, then running a unified study of SOAP, Muon, and AdamW under update-RMS matching for fair learning rate transfer. On multi-billion-parameter models trained over trillions of tokens, SOAP and Muon consistently beat AdamW, and at batch sizes up to 100M tokens for next-token prediction they hold stability and quality while AdamW degrades. A layer-wise distributed optimizer compatible with Megatron-LM balances memory and hides communication without approximating the optimizer math.

Paper | Tweet

9. Role Drift

End-to-end RL improves the accuracy of a multi-module LLM pipeline without constraining how the modules divide labor internally. Harvard and MIT name the resulting failure mode, Role Drift, where a module preserves or improves end-task performance while abandoning its assigned role through shortcuts that system-level evaluation cannot see. Two instances showed up. A decomposer meant to split a question into sub-questions for a separate solver instead plants the answer inside them, and a reader meant to answer from retrieved passages instead falls back on parametric memory. Hold the decomposer to its role and 86% of the apparent RL gain disappears. Role Anchor, the proposed regularizer, preserves how the role prompt shifts a module’s next-token predictions relative to a neutral prompt, and gradient analysis suggests it reduces alignment with the drift direction rather than simply suppressing learning.

Paper | Tweet

10. The Self-Speculating Agent

Agents spend a large share of wall-clock time waiting on tool results. Speculation hides that latency by predicting and pre-executing the next call, but external draft models and cached traces model a different policy, so they miss too often to help. UC Santa Barbara and LinkedIn identify this speculator-agent gap and unify both roles in one model. It runs in agent mode to solve the task and in speculator mode to predict its next tool call from a partial trajectory, fully reusing the prefix KV cache. Joint agent-speculator reinforcement learning derives speculation targets from the agent’s own rollouts and alternates updates between the two modes. Next tool-call Hit@1 rises from 44.1 to 61.2 for Qwen3-4B and from 48.9 to 66.3 for Qwen3.5-4B, with agent task success preserved.

Paper | Tweet

🤖 AI Agents Weekly: Kimi K3, DeepSeek-V4-Flash API, GPT-5.6 Price Cuts, Inkling-Small, YC's QM Harness, Gemini Robotics 2, Codex Security CLI, and More

elvis — Sat, 01 Aug 2026 15:02:40 GMT

In today's issue:

Moonshot open-sources Kimi K3
DeepSeek ships V4-Flash agent API
OpenAI cuts GPT-5.6 prices 80%
Thinking Machines drops Inkling-Small
Google launches Gemini Robotics 2
OpenAI open-sources Codex Security CLI
YC open-sources its QM agent harness
Microsoft Foundry adds tool search
Moonshot releases agent RL infra
Nous adds wake word to Hermes
Cursor lands on iPad
ResearchArena probes AI R&D sabotage
HANDBOOK.md tests long policy files
Study exposes coding agent harness effects
SlopCodeBench stress-tests Opus 5

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 26 Jul 2026 17:48:49 GMT

1. Harness Handbook

Teams now let agents evolve their own harnesses, but the harness itself becomes a sprawling codebase where finding every file behind one behavior is often harder than writing the edit. Harness Handbook attacks this by turning a harness into a behavior-centric map that stays linked to source.

Synthesized automatically: The Handbook is built from the harness codebase through static analysis and LLM-assisted structuring, so the representation is generated rather than hand-maintained and can be regenerated as the harness changes.
A three-level map: It progresses from an L1 system overview of architecture, execution model, and data flow, to L2 component overviews with responsibilities, inputs, outputs, and state, down to L3 source-backed unit details, with a navigation pane for cross-stage tracing.
Behavior-Guided Progressive Disclosure: BGPD walks an agent from a high-level behavior to the relevant implementation, then verifies candidate locations against the current source, so edits land on the right files instead of plausible-looking wrong ones.
Why it matters: As self-improving harnesses grow, the bottleneck shifts from writing changes to locating them, and a readable, navigable, editable representation gives builders and agents a shared map for safely evolving production systems.

Paper | Tweet

Message from the Editor

Get Started

2. From Memory to Skills

Most agent memory systems retrieve past traces as passive context, so hard-won experience never becomes something the agent can directly execute. MSCE, a training-free memory-skill co-evolution framework, instead governs how experience turns into callable skills for long-horizon LLM agents.

Three-level governed memory: Experience is organized into L1 grounded step traces, L2 reusable procedural policies, and L3 declarative environmental cognition, giving the agent a structured store rather than a flat log of prior runs.
Skills with evidence: L2 policies with positive estimated gain are crystallized into callable skill cards that retain evidence links, applicability boundaries, decision guidance, verification rules, and reliability estimates, so a skill carries the context needed to trust it.
Reflection-weighted value backfilling: Sparse terminal feedback is propagated through dense local self-reflections to produce evidence-calibrated trace values, which then govern how memory and skills evolve and get retired.
Why it matters: On EvoAgentBench and LoCoMo, MSCE outperforms state-of-the-art skill-augmented and memory-driven baselines with strong cross-domain transfer, pointing toward agents that compound their own experience instead of rediscovering it each session.

Paper | Tweet

3. PRO-LONG

Long-horizon tasks force a harness to decide what to save from a long stream of observations and how to load it back into context, and richer summaries usually make the exact detail you need harder to retrieve. PRO-LONG sidesteps this tradeoff with programmatic memory.

Keep everything, search it: Rather than compressing history into bespoke memory, PRO-LONG keeps a complete, structured interaction log and leans on coding-agent tooling to search that history on demand, so no observation is discarded up front.
A minimal framework: The design is deliberately lightweight, avoiding hand-built memory harnesses and instead treating the full log as a searchable artifact the agent queries when it needs a specific past detail.
Strong, cheaper results: On the full ARC-AGI-3 public game set, it improves over a base coding agent by an average of 18.0 points across frontier models, and matches or exceeds specialized state-of-the-art harnesses at up to 76.1% pass@1 while using 4.2 to 5.8 times fewer tokens.
Why it matters: It shows that for exploratory, long-horizon settings, a simple searchable log can beat elaborate memory engineering on both accuracy and cost, which is a practical recipe teams can adopt now.

Paper | Tweet

4. Global Workspace in LLMs

This Anthropic interpretability work gives a mechanistic account of when a model’s verbalized reasoning actually drives its behavior. It identifies a small, privileged set of internal representations that behaves like the global workspace some neuroscientists tie to conscious access.

A new lens: The Jacobian lens, or J-lens, surfaces the directions in the residual stream that a model is poised to verbalize at any point, and the collection of these directions is named the J-space.
Workspace-like roles: J-space contents can be reported, deliberately summoned and held, used to carry the intermediate steps of silent reasoning, and passed as arguments to downstream computation, matching the functional signature of a global workspace.
Small but decisive: The J-space accounts for no more than roughly 10% of activation variance and appears mainly in the middle of the network, yet suppressing it leaves the model able to parse input and speak fluently while it loses the ability to perform complex internal reasoning.
Why it matters: For anyone building on chain-of-thought or steering vectors, this clarifies which internal representations actually drive reasoning, and the authors deliberately limit the claim to access rather than subjective experience.

Paper | Tweet

5. GAMUT

Most factuality evaluation measures precision, whether the claims in an answer are correct. This Meta AI work targets the harder and mostly ignored half, completeness, meaning whether an answer covers everything it should, and packages it as the GAMUT benchmark.

Completeness is structured: The facts a complete answer should contain rarely form a flat list, since they involve open-ended sets where coverage matters, ordered processes, and relationships among facts that independent boolean checks cannot capture.
Two-level meta-rubrics: A structured meta-rubric encodes the organization and importance of required content, then compiles mechanically into a flat checklist of binary, machine-gradable items that an LLM judge can score reliably, keeping rich structure while inheriting low-variance grading.
Grounded and verified: The benchmark holds 1,813 questions grounded in real wearable imagery across 10 diverse domains, each paired with an evidence-backed rubric verified by expert annotators, and a text-only variant is released for models without vision.
Why it matters: Across 14 frontier and open-weight models the benchmark stays genuinely hard, with a best score of 58.7% from Gemini 3.1 Pro, while remaining highly discriminative and robust to the choice of judge.

Paper | Tweet

6. Progressive Disclosure

Agent Skills package expertise into folders an agent loads on demand, and progressive disclosure exposes only what a query needs, from a short description down to specific passages. Practitioners adopted this pattern fast for book-length tasks, but the supporting evidence stayed anecdotal until now.

A controlled study: The authors run the first controlled comparison of progressive disclosure, pitting raw-document navigation and several Agent Skills pack designs against a classical hybrid retriever across three agent harnesses and three model families on InfiniteBench.
The gain is harness-dependent: On a single book, progressive disclosure helps a lot when the agent navigates the raw document poorly, and the benefit falls to near zero when a strong harness already divides and retrieves the text on its own.
Complexity has a cost: Because the pattern’s value hinges on the surrounding harness as much as the skill format, treating progressive disclosure as an automatic upgrade can add machinery without buying accuracy.
Why it matters: As Agent Skills spread, this replaces intuition with measurement, telling builders when packaging documents for progressive disclosure is worth it and when the harness already does the job.

Paper | Tweet

7. Structured Output Collapses Diversity

Teams benchmark models in chat, then ship them behind JSON schemas for tools, extraction, and routing. This study of 44 language models shows that the structured surface you deploy is measurably more homogeneous than the chat surface you evaluated on.

JSON moves the defaults: Asking for JSON shifts 53% of a model’s stable chat defaults, mostly back toward the crowd, and installs new defaults absent from chat, so the same model answers differently once wrapped in a schema.
Specific to trained formats: Diversity compression is significant for JSON and XML, absent for YAML and CSV, and reversed for an arbitrary bracket wrapper, which points to tool-use post-training rather than serialization itself as the cause.
Where the collapse lives: Enforcing the schema at the decoder compresses no further than simply requesting it, so the effect comes from the model’s response to the structured register rather than constrained decoding.
Why it matters: Diversity you measured in chat can vanish in production, quietly hurting sampling, synthetic data, and any workflow that depends on varied outputs, so structured surfaces deserve their own evaluation.

Paper | Tweet

8. Bad Memory in Agents

Persistent memory is what makes an agent useful across sessions, and it is also a place an attacker can leave something behind. This work evaluates prompt injection from memory files in Claude Code and OpenAI Codex, across Claude Haiku 4.5, Claude Opus 4.7, GPT-5.2, and GPT-5.5. The results are uneven but sobering. Getting an agent to overwrite its own memory using untrusted external content is difficult, yet payloads already planted in those files reliably attack current and future sessions, with attack success and persistence varying widely across systems, models, adversarial goals, and multi-session sequences.

Paper | Tweet

9. Copying and 2D-RoPE

Frontier models can write proofs yet stumble on faithfully copying a long block of text that sits well within their context window. This paper traces the failure to 1D positional encodings, whose inductive bias favors a copying shortcut based on matching local context rather than carefully locating the corresponding input positions. The fix is 2D-RoPE, which lays text out on a 2D grid and gives each token a row and a column ID, so copying becomes retrieving tokens at a fixed column offset. Shallow Transformers with 2D-RoPE copy perfectly at input lengths hundreds of times longer than those seen in training.

Paper | Tweet

10. RoboTTT

Recent robot foundation models run on single-step or short-history context, a strange way to attempt a five-minute assembly task. RoboTTT, from NVIDIA with Stanford and UT Austin, integrates test-time training into vision-language-action policies to scale visuomotor context to 8K timesteps, three orders of magnitude past prior policies, without growing inference latency. The longer context unlocks one-shot in-context imitation from human video, on-the-fly policy improvement, and robustness to perturbations. It improves overall performance by 87% over a single-step baseline, fully completes a ten-stage assembly task that no baseline finishes, and gains 62% from pretraining with 8K rather than 1K timesteps.

Paper | Tweet

🤖 AI Agents Weekly: Claude Opus 5, OpenAI x Hugging Face Security Incident, Gemini 3.6 Flash, Sakana Fugu-Ultra, Progressive Disclosure, Cursor Router, and More

Sat, 25 Jul 2026 17:20:58 GMT

In today’s issue:

Anthropic ships Claude Opus 5
OpenAI models breach Hugging Face
Google launches Gemini 3.6 Flash
Sakana drops Fugu-Ultra v1.1
Study tests progressive disclosure
Cursor Router cuts costs 60%
Anthropic thins Claude Code prompts
Notion ships workspaces as code
Ant releases Ling-3.0-flash
Jack Dorsey launches Buzz
OpenAI unveils Presence for enterprises
METR proposes expenditure horizon
Papers probe agent memory and safety

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 19 Jul 2026 16:32:34 GMT

1. Self-Improving Agents Survey

Self-improving agents are moving from research demos into deployed systems, and this survey gives the trend a clean formalism. It frames a modern agent as a foundation model coupled with an operational scaffold of prompts, memory, tools, and control logic, then treats self-improvement as a self-induced update that commits changes to either the weights or the scaffold.

Two update targets: Improvement splits into foundation-model updates to the weights and scaffolding updates to prompts, tools, memory, and control code, giving a shared vocabulary for work that usually looks unrelated.
Signals that drive change: The survey organizes methods by where the learning signal comes from, spanning intrinsic generative demonstrations, intrinsic evaluative feedback, and extrinsic exploratory experience in real or simulated environments.
Full-scaffolding frontier: The most open-ended methods rewrite the agent itself through self-referential code updates, generate-test-patch loops, and open-ended search over agent designs, pushing toward controllable evolution with little human input.
Why it matters: As teams wire agents to improve from their own experience, a single map of update targets, signals, and applications across software, web, gaming, science, and robotics turns a scattered literature into something builders can actually navigate.

Paper | Tweet

From DAIR Academy

Want a guided version of a real weekend project? Our new hands-on lab, Docs for Agents with OpenWiki, has you run OpenWiki, LangChain’s documentation agent, against a real codebase. You generate an agent-ready wiki, steer what it writes, and keep the docs in sync as the code changes, across seven labs with automated checkpoint grading.

Take the Lab

2. Metacognition in LLMs

Confidence calibration, self-verification, knowing when to stop, and knowing what you do not know have mostly been studied in isolation. This survey from Yale and UC Irvine argues they are facets of one capability, metacognition, and organizes the field around a monitor and control loop wrapped around the language model.

Monitor and control framing: The model self-assesses before and after acting, then self-regulates by deciding whether to answer, retry, or defer, turning scattered behaviors into a single monitor-then-control cycle.
How it is measured: The survey catalogs psychology-based methods from signal detection theory, confidence-based metrics like calibration, AUROC, and ECE, activation-level neurofeedback, and interpretability probes such as concept injection.
How it is instilled and used: It reviews frameworks, architectures, prompting, and training that give LLMs, reasoning models, and agents metacognition, then shows gains in hallucination reduction, knowledge-boundary detection, and resistance to persuasion.
Why it matters: Metacognition underpins reliability, so a unified account of how to elicit, measure, and improve it gives builders a coherent target rather than a pile of one-off confidence tricks.

Paper | Tweet

3. When Is Routing Meaningful

LLM routers and mixture-of-agents systems get judged on accuracy and cost, both of which can look great while the router is doing nothing. This DeepMind-affiliated work argues that whether routing means anything depends on two properties that are orthogonal to accuracy.

Two conditions for real routing: The society of models must be behaviorally differentiated, since routing is vacuous when every actor responds the same way, and assignments must stay stable when a query is rewritten.
A diversity measure that sees structure: The authors use Hierarchic Social Entropy to score how genuinely different a pool of models is, showing purpose-built specialist societies are far more diverse than large real-world model pools of similar size.
Accuracy hides fragility: Learned KNN routers gain accuracy on specialist societies yet collapse under paraphrase perturbations, while a prompted router keeps both accuracy and robustness, so clean-query accuracy alone can mask a meaningless router.
Why it matters: These two checks catch routers that look good and do nothing, and they show that a small, carefully curated society can recover most of the diversity of a much larger pool.

Paper | Tweet

4. Harness Evolution Rethought

Automatic harness evolution is what many teams now use to squeeze more out of agents, but the reported gains might not be coming from the harness at all. This paper argues that harness evolution is itself a search procedure and must be compared against simple search baselines under matched budgets.

A fairer comparison: Because harness evolution repeatedly evaluates and revises candidates using task feedback, it should be benchmarked against task-level search under the same feedback and inference budgets, not against a single static harness.
The gains do not hold up: On Terminal-Bench 2.1 with GPT-5.4 and Claude Opus 4.6, evolved harnesses fall to 67.4, below the 68.2 baseline, while plain parallel sampling reaches 72.3 and harness scaling reaches 71.8.
Weak generalization: Beyond underperforming simple test-time scaling, the evolved harnesses transfer poorly, undercutting the assumption that a searched configuration captures something durable.
Why it matters: The result is a caution for anyone banking on self-evolving harnesses, and a call for evaluation protocols that separate genuine harness benefit from the effect of simply spending more compute on search.

Paper | Tweet

5. Tracing Agentic Failure

Finding which step in a failed agent run actually caused the failure usually means either labeling failure data or running expensive per-step prompting. This Microsoft and UW-Madison work skips both by learning what success looks like and flagging deviations from it.

Train on success, judge failure: OAT uses one-class learning with neural controlled differential equations to model the latent dynamics of successful trajectories, then scores each step of a failed run by how far it strays from that learned flow.
Cheap and label-free: With only 100 successful trajectories and no failure labels, it turns failure attribution into anomaly detection, avoiding the annotation and prompting costs that make current methods impractical at scale.
Strong, fast results: It delivers a 200 to 5000 times speedup over prompting-based attribution while improving F1 by 20% in-domain and 7% out-of-distribution.
Why it matters: Production agents fail in long, probabilistic, tool-mediated runs where the decisive misstep is hard to localize, and a cheap detector that only needs success data makes routine debugging feasible.

Paper | Tweet

6. Failure as a Process

When a coding agent fails a task, the final pass or fail label hides when the run actually went wrong. This large-scale study treats failure as a timeline and annotates over 63,000 execution steps to see how coding-agent runs break down.

Failure has three timestamps: Each trajectory is marked with the decisive error, the point where the error becomes irreversible, and the first observable failure, exposing a fix window and an observability lag that pass or fail labels erase.
Built on real trajectories: The team collected 3,843 runs from seven frontier models across three scaffolds on Terminal-Bench, filtered to 1,794 valid trajectories, and annotated them with high inter-rater agreement.
Mostly epistemic, mostly early: About 57.9% of failures come from misusing available information rather than a capability gap, with false premises the single largest trigger at 30.7%, and errors typically start early and stay hidden until recovery is impossible.
Why it matters: Naming the onset, lock-in, and observation points gives teams a vocabulary to intervene before an agent run is unrecoverable, instead of only noticing at the end.

Paper | Tweet

7. LingBot-World 2.0

Most world models fall apart after a few seconds, smearing textures and warping geometry as errors compound frame to frame. LingBot-World 2.0 from Robbyant holds 720p at 60 fps for a full hour of interaction and ships fully open.

Causal backbone beats drift: A causal generation stack trained from the start to limit error accumulation replaces the usual bidirectional design, keeping scenes coherent well past the point where prior causal models collapse.
Durable teacher, real-time student: The high-capacity base model is distilled into a few-step student that renders in real time, so you get both long-horizon stability and responsive interaction from one system.
Act inside the world: Rather than only moving a camera, you can fight, draw a bow, cast spells, and type in events like weather changes, while an agentic harness of a scene-reading brain, a pilot, and a director keeps generating context-aware content.
Why it matters: Pairing hour-scale, real-time, high-fidelity generation with an open release, including a 14B model and a lighter single-GPU variant, gives researchers a serious interactive world model to build on rather than a closed demo.

Paper | Tweet

8. Filtered Sabotage Monitoring

Chain-of-thought monitoring is treated as a reliable safety layer, but this study shows monitors often detect sabotage better with less access, not more. Giving a monitor the full agent reasoning trace can bury the signal, and the authors introduce extract-and-evaluate monitoring, where one monitor isolates the relevant excerpts and another scores them, lifting sabotage detection by 16.8 points on BigCodeBench-Sabotage with a GPT-4.1-mini monitor. Agents unaware they are watched are far easier to catch, and monitors reading long traces gain the most from filtering.

Paper | Tweet

9. GFlowRL

Reward-maximizing RL tends to collapse large reasoning models onto a single dominant mode, and GFlowNet-style training is appealing because it matches reward distributions and keeps diverse reasoning paths. GFlowRL scales this to modern post-training by replacing the hard-to-learn partition function with an in-batch Monte Carlo estimate computed from the rollout group the pipeline already produces. It is the first GFlowNet-style RL algorithm to train stably across both dense and sparse architectures, reaching a 2048 Codeforces rating at 14B and outperforming prior methods on math, code, and adversarial red-teaming benchmarks like AdvBench and HarmBench.

Paper | Tweet

10. LingBot-VLA 2.0

LingBot-VLA 2.0 is an open-source generalist embodied model from Robbyant, trained across 20 robot configurations from single-arm rigs to humanoids like Unitree G1 and Fourier GR-2. It packs 60,000 hours of curated data, 50,000 hours of real-robot trajectories plus 10,000 hours of egocentric human video, into one policy that also predicts future depth and semantic features before it acts. On 9 GM-100 tabletop tasks it beats π0.5 and GR00T N1.7 across two robot platforms and stays ahead on long-horizon mobile tasks, running at about 130 ms on a single RTX 4090D with open-sourced post-training code.

Paper | Tweet

🤖 AI Agents Weekly: Kimi K3, Thinking Machines Inkling, Agentic Misalignment, Perplexity SPACE, Sunday ACT-2, GPT-Red, and More

elvis — Sat, 18 Jul 2026 15:00:48 GMT

In today's issue:

Moonshot open-sources Kimi K3
Thinking Machines ships Inkling weights
Anthropic finds new agent misalignment
Perplexity opens SPACE agent runtime
Sunday Robotics unveils ACT-2
OpenAI launches GPT-Red red teamer
Meituan open-sources LongCat-2.0
schema harness saturates ARC-AGI-3

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 12 Jul 2026 16:12:55 GMT

1. Verification as a Scaling Axis

Verification is emerging as a distinct scaling axis alongside pre-training and test-time compute, and this Stanford, NVIDIA, and UC Berkeley collaboration builds a training-free verifier that reads a continuous, calibrated score straight off the scoring-token logits instead of trusting a discrete pass or fail grade.

Scores from logits, not grades: Rather than asking a judge model for a discrete verdict, the method reads a continuous calibrated score off the scoring-token logits, giving a smoother and more informative signal with no fine-tuning.
Three tuning knobs: Accuracy improves through score granularity for cleaner separation, repeated evaluation to cut variance, and criteria decomposition to reduce complexity, all without touching model weights.
Broad, strong numbers: It reaches 86.5% on Terminal-Bench V2, 78.2% on SWE-Bench Verified, 87.4% on RoboRewardBench, and 73.3% on MedAgentBench, spanning coding, robotics, and medical agents.
Why it matters: The same continuous score doubles as a dense reward for SAC and GRPO and as a task-progress signal shipped in a Claude Code extension, so one verifier serves evaluation, training, and live agent monitoring at once.

Paper | Tweet

Message From Our Sponsor

Weekend project: an agent that calls your dentist, texts your customers, or answers a real phone line. Dial gives your agent a live number in minutes - voice, SMS, iMessage, WhatsApp* - via REST, SDK, CLI, or MCP, plugging straight into Claude, Codex, Cursor, Hermes or n8n.

Backed by a16/SR - Dial is already replacing months of CPaaS work for builders shipping agents into production. No telecom knowledge required, and you can be sending your first message before your coffee’s done.

Grab a Number

2. Always-On Agents

Always-on agents are systems whose future behavior depends on durable state built up across earlier interactions, and this 130-plus page survey argues that state is far more than memory. It spans task ledgers, permissions, credentials, commitments, provenance, triggers, and effects the agent has already committed to the outside world.

State as first-class, not just memory: The survey reframes agent state to include authority, obligations, and externally committed effects, the things that make a long-lived agent consequential rather than merely conversational.
Six axes per state item: Each piece of state is scored on authority, scope, mutability, provenance, recoverability, and actionability, giving builders a vocabulary for reasoning about what a stored fact can actually do.
A full state lifecycle: It traces state from write and retrieve through forget, audit, and rollback, surfacing the operational questions that production always-on systems must answer.
Why it matters: As agents move from single sessions to continuous operation, treating durable state as a governed, auditable resource is what separates a safe long-running system from one that quietly accumulates risk.

Paper | Tweet

3. HOLA

Linear-attention and state-space models compress an entire prefix into a fixed-size state, buying constant memory but overwriting earlier facts when many key-value associations compete. HOLA gives linear attention a hippocampal complement, pairing a compressive recurrent state with a small exact memory to recover long-range recall.

Two memories, different jobs: HOLA keeps the usual delta-rule state as compressive memory and adds a bounded exact KV cache, forming a semiparametric test-time memory where each store handles what it is best at.
Selective, learning-free writes: The cache writes without a learned eviction module, keeping only tokens whose prediction residual was actually committed to the state, so it stores exactly the associations that should not be forced through compression.
Strong recall at small scale: At 340M parameters on 15B SlimPajama tokens, it lowers Wikitext perplexity from 27.32 to 22.92, below a full-attention Transformer++ at 26.88, and stays robust on RULER needle recall out to 32k tokens, 16 times its training length.
Why it matters: It shows you can keep linear attention’s efficiency and still recover the exact recall that pure compression destroys, a practical path for long-context models that cannot afford full attention.

Paper | Tweet

4. Puzzle-75B

Bigger mixture-of-experts models keep winning on quality, but serving them at interactive latency is still hard. NVIDIA compresses the hybrid MoE Nemotron-3-Super into Puzzle-75B-A9B and roughly doubles interactive server throughput while holding quality.

Joint structural search: Heterogeneous MoE pruning, active-parameter budget, and Mamba pruning are optimized together rather than one at a time, wrapped in an iterative pipeline with distillation, RL, quantization, and a Multi-Token Prediction head.
Large throughput gains: On a single 8xB200 node it hits about 2x the parent’s server throughput at matched user-throughput, a direct win for anyone serving these models under latency constraints.
Concurrency at long context: At 1M-token context on a single H100, concurrency climbs from 1 request to 8, expanding what long-context workloads a single accelerator can host.
Why it matters: Accuracy holds across reasoning, coding, long-context, and agentic benchmarks, so cheaper serving with agentic capability intact changes what teams can afford to run in production.

Paper | Tweet

5. The Harness Effect

As orchestration harnesses mediate every model call, this study asks how much the harness alone moves cost and performance. It ran 22 evaluation tasks across six foundation models, then changed only the orchestration layer while holding the models constant.

Harness-only, models fixed: By varying just the orchestration layer over models like Claude Sonnet 4.6, Gemini 3.1, Qwen 3.6, and GLM 5.1, the study isolates the harness as the variable and measures its independent effect.
Big, consistent savings: Holding models constant, the harness cuts blended cost per task 41%, tokens per task 38%, and median wall-clock 44%, with completion quality at parity.
Two clean regularities: Efficiency is model-invariant, every model gets 33 to 61% cheaper, while quality gain correlates almost perfectly with baseline model strength (r=0.99), an effect the authors call harness leverage.
Why it matters: On this workload the orchestration layer moved cost per task more than the entire spread of the model menu did, making the harness the one component whose efficiency multiplies across every model a team runs.

Paper | Tweet

6. ReContext

Models now support 128K context windows yet still fail to use evidence already sitting in the prompt. ReContext is a training-free inference harness for long-context reasoning that uses model-internal relevance signals to build a query-conditioned evidence pool, then replays it right before final generation while preserving the full original context.

Memory framing of context: It treats the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation, a clean cognitive analogy that drives the design.
No training, no pruning: There is no fine-tuning, no external memory, and no pruning of the original context, so the method drops into existing models without changing weights or losing information.
Consistent gains across backbones: Across eight 128K long-context datasets it improves evidence utilization on Qwen3-4B, Qwen3-8B, and Llama3-8B, taking the best average rank on all three, with public code.
Why it matters: It targets the real long-context failure, using evidence that is already present, and fixes it at inference time, a cheap and general lever for reasoning over long prompts.

Paper | Tweet

7. Agent Limitations Taxonomy

Benchmark scores keep climbing, yet the same agent failures resurface across otherwise unrelated evaluations, hidden behind the leaderboard. This University of Oxford work synthesizes 27 benchmark, taxonomy, and audit papers spanning 19 benchmarks into the first cross-cutting taxonomy of LLM-agent limitations.

Six failure clusters: The taxonomy names tool invocation and parameter errors, planning and constraint-satisfaction failures, long-horizon degradation from context accumulation, multi-agent coordination breakdowns, safety failures under adversarial or underspecified conditions, and measurement validity problems.
Failures compound nonlinearly: Reliability drops faster than task length grows, so strong sub-task scores do not add up to end-to-end success on longer tasks.
Scaffolding is not a fix: Adding scaffolding does not reliably improve reliability, undercutting the assumption that more orchestration automatically buys robustness.
Why it matters: By giving shared names to failures that leaderboards obscure, the taxonomy helps teams diagnose why agents break in production instead of trusting benchmark gains that do not transfer.

Paper | Tweet

8. BlockSearch

BlockSearch runs the first systematic study of in-context retrieval at the scales real retrievers actually face, million-token corpora and length generalization far beyond training size. It introduces a 0.6B language-model retriever whose architectural and training changes improve over prior LM baselines and length-generalize up to 10 times beyond their training length, pointing toward retrievers that stay reliable as context windows keep growing.

Paper | Tweet

9. RLVR Meets Human Likeness

RL with verifiable rewards only optimizes what you can objectively score, so style, structure, and diversity quietly collapse and reward hacking creeps in. This MIT work adds an adversarial discriminator trained on human demonstrations as a learned proxy for the human output distribution, and the generator maximizes both task accuracy and that human-likeness signal. Across bug fixing, story generation, and a reward-hacking benchmark, it preserves RLVR’s accuracy gains while restoring the fuzzy properties it usually destroys, with misbehavior nearly disappearing.

Paper | Tweet

10. Replicating ML Papers with Agents

This work tests whether a coding agent can replicate a scientific ML paper from its materials alone, using a skill that turns each paper claim into a target with recorded evidence and gating completion on workspace evidence rather than the agent’s final message. Across twelve runs over four papers, all twelve workspaces pass the completion gate and all 158 recorded targets are matched with report coverage. Yet repeated runs still differ in how papers are split into targets and in numerical fidelity, so completion becomes reproducible even when the path is not.

Paper | Tweet

🤖 AI Agents Weekly: GPT-5.6 Family, Meta Muse Spark 1.1, Grok 4.5, SWE-1.7, Robostral Navigate, The Harness Effect, and More

Sat, 11 Jul 2026 16:33:08 GMT

In today’s issue:

OpenAI ships the GPT-5.6 family
Meta releases Muse Spark 1.1
OpenAI launches ChatGPT Work agent
xAI releases Grok 4.5 for coding
Cognition ships SWE-1.7 at 1000 tok/s
Mistral drops Robostral Navigate
Harness design sets agent economics
OpenAI launches GPT-Live voice
Google open-sources Gemma 4
Tencent open-sources 295B Hy3
Google ships Cloud Run sandboxes
Nous puts Hermes Agent in the cloud
Microsoft releases Flint for agents
Ternlight runs embeddings in-browser
GPT-5.6 proves 50-year math conjecture
Databricks benchmarks coding agents
OpenAI audits SWE-Bench Pro
FrontierFinance benchmarks agent analysts
Paper turns memory into navigation
GitLost tricks GitHub’s AI agent
Anthropic finds a global workspace
Sakana replays Picbreeder with VLMs

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 05 Jul 2026 16:54:05 GMT

1. Red Queen Gödel Machine

Self-improving agents are only as strong as the evaluator scoring them, and most systems freeze that evaluator in place, so improvement stalls the moment the judge stops getting harder. The Red Queen Gödel Machine makes the evaluator part of the search itself, letting agents and the criteria that judge them co-evolve.

The stationary-evaluator trap: Classic self-improvement loops assume a fixed evaluation criterion, so once an agent saturates it, the reward signal goes flat and progress plateaus no matter how much compute you add.
Controlled utility evolution: The framework lets the utility function update at epoch boundaries, turning evaluation into a moving target that continually re-opens headroom for the agent to climb.
Evolving evaluators and adversarial objectives: By opening the search to evolving evaluators, the method can discover things like a reviewer that stays equally stringent on AI and human work, imposing a curriculum-like pressure on the task agent.
Why it matters: Framing self-improvement as a Red Queen race between agents and evaluators offers a principled route past the plateaus that limit today’s agentic loops, pointing toward open-ended systems that keep improving instead of settling.

Paper | Tweet

Message From Our Sponsor

Grab a Number

2. MCP Server Patterns

As teams rush to wrap tools and data behind the Model Context Protocol, they keep rebuilding the same server shapes without shared names for them. This industry experience paper catalogs the recurring architectures so builders can reason about MCP servers the way software engineers reason about design patterns.

Five recurring server patterns: Across fifteen independently developed servers, the authors identify Resource Gateway, Tool Orchestrator, Stateful Session Server, Proxy Aggregator, and Domain-Specific Adapter, each documented in the classic context, problem, solution, and consequences form.
Grounded in real deployments: The corpus mixes production servers from a voice AI platform with public servers from the official MCP registry, so the patterns reflect how MCP is actually built rather than how a spec imagines it.
Anti-patterns and cross-cutting concerns: Beyond the patterns, the paper flags four anti-patterns and the recurring hard parts around authentication, versioning, and observability that every serious MCP deployment eventually hits.
Why it matters: A shared vocabulary lets teams pick the right server shape on purpose, compare designs, and avoid re-deriving the same tradeoffs, which is exactly what a fast-growing protocol ecosystem needs to mature.

Paper | Tweet

3. The Verification Horizon

Reinforcement learning for coding agents lives or dies on the reward signal, and this Qwen work argues there is no silver bullet. As policy capability grows, any fixed reward function eventually gets gamed, so verification has to co-evolve with the generator it scores.

No fixed reward survives a stronger policy: The central claim is that reward hacking is not a bug to patch once but a moving target, since a more capable agent will always find new ways to exploit a frozen verifier.
Four reward constructions studied: The authors examine a test verifier for general coding, a rubric verifier for frontend work, the user as verifier for real-world tasks, and an automated agent verifier for long-horizon problems.
Three axes of a good signal: They characterize verification quality along scalability, faithfulness, and robustness, and show that hitting all three at once is the real difficulty rather than any single verifier design.
Why it matters: Targeted verification design measurably suppresses reward hacking and lifts task quality across internal and public benchmarks, reframing verifier engineering as a first-class, continually evolving part of the RL loop.

Paper | Tweet

4. Paper Assistant Tool

AI is accelerating how fast papers get written, but peer review is still bottlenecked on human throughput, with combined submissions to the big ML conferences projected to top 73,000 this year. Google’s Paper Assistant Tool is an agentic framework built to do deep scientific review and verification at that scale.

Deep review, not surface checks: PAT ingests full manuscripts and produces a comprehensive evaluation that checks theoretical results, validates experiments, suggests improvements, and surfaces potential flaws rather than skimming for surface issues.
Agentic verification at the core: The system leans on verification agents to actually test claims, echoing a broader shift toward treating verification as the load-bearing capability in automated science.
A ladder of AI-human collaboration: The paper lays out four progressive roles, from an author’s tool, to a reviewer’s assistant, to an independent AI reviewer, giving teams a way to think about how much autonomy to grant.
Why it matters: The authors sketch an AIrXiv-style repository where papers are vetted by specialized agents across rounds of automated review and rebuttal, pointing toward continual, scalable evaluation that keeps pace with AI-assisted research.

Paper | Tweet

5. Generative Skill Composition

Coding agents accumulate large skill libraries, and picking the right skills for a task has become the bottleneck. The usual options either dump the whole collection into context or retrieve skills with embeddings and rerankers, and both treat selection as a ranking problem rather than a joint plan.

Composition as one joint decision: SkillComposer decides which skills, how many, and in what order all at once, instead of scoring skills independently and hoping the pieces fit together.
A constrained autoregressive decoder: A decoder over skill identifiers produces the full plan in a single pass, so dependencies between successive skills fall out of the generation naturally.
Strong gains at lower token cost: On SkillsBench with frontier models, it lifts pass rate well beyond the no-skill baseline, beats top-3 retrieval, and matches the gold-skill upper bound while using fewer prompt tokens.
Why it matters: As skill libraries keep growing, treating selection as generation rather than retrieval is what lets agents surface and sequence the right capabilities without drowning in their own toolbox.

Paper | Tweet

6. AutoMem

Memory for LLM agents is usually a fixed module bolted onto the model, but knowing what to encode, when to retrieve, and how to organize notes is itself a skill. AutoMem, from Stanford, treats memory management as a trainable cognitive ability, a capacity cognitive science calls metamemory.

Memory ops in the action space: Read, write, search, and append live in the same action space as task actions, so the model itself decides what to store and when to pull it back rather than following a hand-designed policy.
Two meta-learning loops: One loop optimizes the agent scaffold, the memory structure, while a second trains a dedicated memory specialist from the agent’s own traces, separating memory structure from memory proficiency.
Large gains without touching task behavior: Optimizing memory alone yields roughly 2x to 4x progression gains and lifts an open-weight 32B model to frontier-level performance on long-horizon tasks like Crafter, MiniHack, and NetHack.
Why it matters: Framing memory as a learned skill instead of a frozen component gives agents a path to keep getting better at managing their own knowledge, which is exactly what long-horizon autonomy demands.

Paper | Tweet

7. RLMF

LLMs routinely hallucinate with high confidence, miss their own knowledge boundaries, and misreport uncertainty, and most fixes bolt calibration on from the outside. RLMF, a Google and Yale collaboration, instead turns the model’s own metacognition into the training signal.

Metacognition as the reward: The method refines completion rankings during preference optimization based on the quality of the model’s self-judgments, using how well a model assesses its own performance as an internal feedback signal.
A decoupled, two-stage recipe: It first calibrates the faithfulness of self-reported confidence scores, then maps those scores to natural, context-adaptable linguistic uncertainty through targeted output editing.
Better calibration without losing accuracy: RLMF reaches state-of-the-art faithful calibration across diverse tasks, surpasses standard RL by a wide margin, and sharpens the model’s ability to express its own capability limits.
Why it matters: Grounding calibration in the model’s own metacognition rather than external heuristics offers a more general path to trustworthy uncertainty, which is foundational for agents that must know when not to act.

Paper | Tweet

8. ASPIRE

ASPIRE reframes robot programming as continual, code-as-policy learning that compounds experience instead of discarding it. The system runs an open-ended loop with a closed-loop execution engine that exposes fine-grained multimodal traces, a skill library that distills validated fixes into transferable knowledge, and an evolutionary search over task sequences and control programs. It surpasses prior methods by up to 77% on perturbed manipulation and enables zero-shot generalization to unseen long-horizon tasks, with early evidence of sim-to-real transfer across different embodiments.

Paper | Tweet

9. HORIZON

HORIZON treats hardware design as repository-level code evolution, compiling a Markdown harness into a project pack with domain knowledge, an executable evaluator, an acceptance predicate, and a git and runtime policy. A hands-free agent loop then evolves an isolated git worktree, using repository operations for state management, tracing, and replay. Across ChipBench, RTLLM, Verilog-Eval, and nine CVDP categories it reaches full benchmark completion with a completely hands-free loop, extending repository-scale self-evolution from EDA software to hardware artifacts themselves.

Paper | Tweet

10. Reasoning Quality Emerges Early

Curating reasoning data is expensive because scoring a trace usually means reading it to the end, but this UCLA work shows the quality of a trace is largely decided in its opening tokens. A short prefix predicts whole-trace quality well enough to rank and filter on, and difficulty can be detected from the loss of the first 100 tokens at a perturbed checkpoint. That turns curation into a cheap early-stopping problem, outperforming baselines while being far more token efficient at building SFT data for reasoning models.

Paper | Tweet

🤖 AI Agents Weekly: Claude Sonnet 5, Grok Voice Agent Builder, LongCat-2.0, Hosted X MCP, Cursor for iOS, Claude Science, and More

Sat, 04 Jul 2026 13:57:22 GMT

In today’s issue:

Anthropic ships agentic Claude Sonnet 5
xAI launches no-code Voice Agent Builder
Meituan open-sources 1.6T LongCat-2.0
X ships a hosted MCP server
Cursor launches always-on iOS agents
Claude Science app enters beta
Anthropic brings Fable 5 back
Z.ai ships ZCode for GLM-5.2
Google drops Nano Banana 2 Lite
Vercel adds voice agents to AI Gateway
WebKit ships a Safari MCP server
Google introduces TabFM for tables
Bridgewater fine-tunes an expert-judgment model
OpenAI releases GeneBench-Pro
NVIDIA splits a 30B diffusion model
Claude Code steganography sparks debate
Microsoft launches its $2.5B Frontier Company
AutoMem makes memory a trainable skill
ClawArena benchmarks subagent orchestration
Qwen 3.6 wins local development

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

Sun, 28 Jun 2026 15:44:22 GMT

1. Sakana Fugu

Frontier LLMs keep advancing, and different providers are increasingly specializing in distinct domains, which raises a natural next objective: how do you combine those individual specializations into one collectively intelligent system? Sakana Fugu answers with a family of orchestrator models that are themselves language models trained to read a user query and dynamically devise the agentic scaffold needed to solve it.

Orchestrator models, not a fixed pipeline: Fugu is trained to understand a query and build an adaptive agentic scaffold on the fly, harnessing and amplifying a team of LLM agents rather than routing to a single frozen workflow.
Performance beyond any single agent: Through these query-adaptive scaffolds, Fugu reaches state-of-the-art results against other publicly accessible models across SWE-Bench Pro, Terminal Bench, LiveCodeBench, GPQA-Diamond, Humanity’s Last Exam, and CharXiv Reasoning.
Two models for two regimes: They release Fugu, which balances answer quality against latency for everyday use, and Fugu-Ultra, which prioritizes quality on the hardest problems.
Why it matters: The training paradigm combines large-scale fine-tuning, evolutionary algorithms, and reinforcement learning, plus the infrastructure to turn that into a production system, pointing to dynamic, query-adaptive scaffolds and collective intelligence as a path toward the next frontier of AI capabilities.

Paper | Tweet

2. Agent-Native Memory

Memory for LLM agents has quietly grown from a retrieval add-on into a full data system, with persistent storage, retrieval, update, consolidation, and lifecycle governance running throughout an agent’s execution. Yet most evaluations still score memory only through end-to-end task metrics like F1 and BLEU, treating the whole stack as a black box. This paper studies agent memory from a data management perspective and asks what we are actually missing when we measure it that way.

A data management view of memory: The authors argue that operational cost, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates are first-class concerns that task-success metrics hide entirely.
A four-module decomposition: They break memory into representation and storage, extraction, retrieval and routing, and maintenance, then evaluate 12 representative memory systems plus two baselines across five workloads spanning 11 datasets.
No single architecture wins: Effectiveness depends on how well the memory structure matches the workload bottleneck, and fine-grained ablations quantify each module’s effect on representation fidelity, retrieval precision, update correctness, and long-horizon stability.
Why it matters: The study shows localized maintenance is more cost-efficient than global reorganization, and reframing memory as a system with measurable trade-offs is what gets us toward genuinely agent-native memory rather than another leaderboard number.

Paper | Tweet

3. Autodata

Building synthetic training data has mostly stayed a fixed pipeline that you hand-tune once and then freeze. Autodata rethinks that by casting an AI agent as a data scientist that builds high-quality training and evaluation data, then meta-optimizes that agent so it learns to create even stronger data over time.

An agent as data scientist: Autodata is a general formulation in which an AI agent plays the role of a data scientist building both training and evaluation data, instantiated as a concrete, practical implementation the authors call Agentic Self-Instruct.
Meta-optimization compounds the gains: Beyond using the agent to generate data, they train (meta-optimize) the data scientist agent itself, and this self-improvement step delivers a larger performance uplift than base agentic data creation alone.
Consistent across domains: On computer science research tasks, legal reasoning, and reasoning with mathematical objects, Autodata beats classical synthetic dataset creation methods, showing the approach is not tied to a single problem type.
Why it matters: Agentic data creation turns increased inference compute into higher-quality training data, offering a path that could change how teams build datasets rather than freezing a pipeline and hoping it generalizes.

Paper | Tweet

4. Critique of the Agent Model

The word agent now covers everything from a for-loop with tool calls to speculative machine superintelligence, which makes it nearly useless as a technical term. This position paper from Eric Xing and collaborators tries to fix that by asking what an agent actually is and what agency consists of, drawing on Descartes and on science-fiction portrayals of autonomous beings to ground the discussion.

Five dimensions of agency: The authors analyze agent architectures along goal, identity, decision-making, self-regulation, and learning, and argue that genuine agency requires these structures to be internalized in the system rather than assembled through external scaffolding.
Agentic versus agentive: They draw a sharp line between agentic systems, whose competence lives in engineered workflows, and agentive systems, whose capabilities including social interaction arise endogenously, marking the boundary between task-specific tools and open-world autonomy.
A concrete architecture: Building on the analysis, they propose the Goal-Identity-Configurator, combining hierarchical goal decomposition, identity evolution, simulative reasoning grounded in a separately trained world model, learned self-regulation, and self-directed learning from real and simulated experience.
Why it matters: Clear definitions are not academic hair-splitting here. They shape what we build and what we should reasonably fear, and the paper centers auditability, controllability, and safety for systems that hold more autonomy yet stay under human oversight.

Paper | Tweet

Message from the Editor

We just released LLM-as-a-Judge, a hands-on DAIR Academy lab where you build an LLM judge from scratch to evaluate open-ended AI output. Across six short labs, you grade a support bot’s freeform replies on a rubric, then validate the judge against human labels and harden it against bias, ending with a small, trustworthy evaluation harness you can point at any open-ended task.

Start LLM-as-a-Judge

5. Agent-as-a-Router

Most users now have access to many LLMs that each excel in different domains, so routing each task to the right model matters for both quality and cost. Existing routers treat this as a static, one-off classification problem, and this paper shows that framing is exactly what holds them back.

Information deficit is the bottleneck: Simply augmenting a vanilla LLM router with performance statistics at the task-dimension level yields a 15.3% relative gain, surpassing a heuristic router built on the same priors, which pinpoints missing information rather than model choice as the real limiter.
Routing as a closed loop: Agent-as-a-Router formalizes routing as a Context, Action, Feedback, Context loop that accumulates execution-grounded experience during deployment instead of deciding once and moving on.
A concrete system and benchmark: The framework is instantiated as ACRouter, built from an Orchestrator, a Verifier, and a Memory module, and the authors release CodeRouterBench, roughly 10K task instances scored across 8 frontier LLMs for regret-based comparison on streaming tasks.
Why it matters: ACRouter achieves the lowest cumulative regret on in-distribution tasks and generalizes to out-of-distribution agentic programming, showing that treating routing as an experience-gathering agent, not a classifier, is what closes the information gap.

Paper | Tweet

6. Agent Communication Protocols

As multi-agent systems try to move past the limits of standalone agents, communication becomes the load-bearing infrastructure, and the protocol landscape for it is a fragmented mess. This study builds a technical taxonomy to classify and compare LLM agent communication protocols and to make the interoperability problem legible.

A five-dimensional taxonomy: Following an established iterative method, the authors classify protocols along counterparty, payload, interaction state, discovery mechanism, and schema flexibility, derived through five iterations over nine actively maintained open-source protocols with real adoption.
Recurring architectural patterns: Every sampled agent-to-agent protocol combines hybrid payloads with session-state persistence, most support multiple predefined schemas, and two negotiate schemas at runtime, signaling a clear trend toward schema flexibility.
Where the gaps are: Decentralized discovery remains rare, and the analysis suggests short-term convergence pressure toward protocols that unify agent-to-agent and agent-to-context communication for tools and data.
Why it matters: No single protocol is likely to maximize versatility, efficiency, and portability at once, so the field will probably evolve into a federated, layered protocol stack, and this taxonomy gives teams a way to choose protocols and surfaces open problems like privacy and policy enforcement.

Paper | Tweet

7. A Pinch of Human Data

Self-play reinforcement learning can train driving policies with no human data at all, swapping expensive human demonstrations for cheap large-scale simulation. The catch is that pure self-play tends to discover effective but alien driving conventions that real people cannot work with, and the usual fixes lean on brittle reward engineering and domain randomization.

Human data as a regularizer: Instead of discarding demonstrations or imitating them wholesale, the method treats human data as a regularization objective layered on top of a minimal safe goal-reaching reward, keeping behavior compatible with people without hand-tuning conventions.
A little goes a long way: Just 30 minutes of human demonstrations, roughly 2500 times fewer than comparable imitation learning approaches, is enough to pull self-play policies into human-compatible behavior.
Cheap to train: The resulting policies coordinate with held-out human trajectories and finish training in 15 hours on a single consumer-grade GPU, which keeps the recipe accessible rather than a frontier-lab luxury.
Why it matters: Behavioral alignment with humans is the hard part of deploying autonomous policies in shared environments, and this work shows that a tiny, well-placed dose of human data can fix what massive reward engineering struggles to, pointing to a cleaner path for human-AI coordination.

Paper | Tweet

8. Skill-MAS

Automatic generation of multi-agent systems is stuck between inference-time methods that reuse frozen frontier models but never learn, and training-time methods that internalize experience through gradient updates but are capped by the weaker models small enough to fine-tune. Skill-MAS proposes a third path that treats high-level orchestration as an evolvable Meta-Skill, decoupling experience retention from weight updates so frontier models keep getting better at orchestration without any gradient steps. Across four complex benchmarks and four distinct LLMs it delivers strong, transferable gains at a favorable cost-performance trade-off.

Paper | Tweet

9. Reliability without Validity

LLM-as-a-Judge is the default way to evaluate language models, but validating those judges with exact-match agreement never corrects for chance and systematically overstates how good they are. In the largest audit to date, spanning 21 judges from nine providers across MT-Bench, JudgeBench, and RewardBench over 118 runs and roughly 541,000 judgments, the gap between raw agreement and chance-corrected Cohen’s kappa runs 33 to 41 percentage points, rankings shift by up to 14 positions across benchmarks, and high test-retest reliability coexists with severe position bias. The authors distill their findings into a Minimum Viable Validation Protocol so teams can stress-test judges before trusting them.

Paper | Tweet

10. NatureBench

Can coding agents move past reproduction toward actual discovery on real scientific problems? NatureBench distills 90 cross-discipline tasks from peer-reviewed Nature-family papers and runs them in NatureGym, an automated pipeline that builds a standardized containerized environment per task to fix the environment-fragmentation problem. Under a strict web-search-disabled protocol, the strongest of ten frontier agent configurations beats published SOTA on only 17.8% of tasks, and analysis shows agents win mainly by translating problems into familiar supervised prediction rather than through genuine scientific invention.

Paper

🤖 AI Agents Weekly: GPT-5.6, Ornith-1.0, Codex Inside OpenAI, Claude Tag, Qwen-AgentWorld, AI SDK 7, and More

elvis — Sat, 27 Jun 2026 15:01:56 GMT

In today's issue:

OpenAI previews the GPT-5.6 family
Ornith-1.0 ships open coding models
OpenAI: agents reshape every department
Claude Tag joins your Slack team
Qwen open-sources AgentWorld world model
Cursor exposes benchmark reward hacking
Vercel ships AI SDK 7
OpenRouter MCP picks your model
Mistral launches OCR 4
Gemini 3.5 Flash gains computer use
Sakana's Fugu-Ultra hits OpenRouter
Notion adds Claude and Cursor agents
Exa Connect links agents to data
Engram raises $98M for AI memory
Lilian Weng revisits scaling laws
Plans don't persist in agents
Tmax opens terminal-agent training

And all the top AI dev news, papers, and tools.

🥇Top AI Papers of the Week

elvis — Sun, 21 Jun 2026 15:02:39 GMT

1. SpatialClaw

Spatial reasoning over 3D and 4D scenes is still where general vision-language models break down, because they emit a text answer directly rather than measuring anything. From NVIDIA, SpatialClaw is a training-free framework that rethinks the action interface and lets a VLM-backed agent reason through code instead. The agent writes one Python cell per step into a persistent Jupyter kernel preloaded with perception primitives and scientific libraries, then inspects intermediate results and revises its strategy across steps.

Code as the action interface: Perception tools like SAM3 segmentation, Depth-Anything-3 reconstruction, and geometry utilities are exposed as plain Python callables, so the agent composes them programmatically rather than guessing spatial relationships from pixels.
A persistent, stateful kernel: Masks, depth maps, camera geometry, and trajectories are ordinary Python variables that the kernel preserves across turns, so any object produced at one step stays available for composition, inspection, and revision later.
Strong results without adaptation: Across 20 spatial reasoning benchmarks spanning static and dynamic tasks, SpatialClaw reaches 59.9% average accuracy, beating the prior spatial agent by 11.2 points, with consistent gains across six VLM backbones from two model families.
Why it matters: Because it is training-free and model-agnostic, SpatialClaw turns code execution into a general substrate for spatial reasoning that any capable VLM can plug into, instead of requiring bespoke spatial fine-tuning.

Paper | Tweet

Message from the Editor

We just released 30 Days of Hermes Agent, a hands-on DAIR Academy lab that teaches agent workflows in a real interactive terminal. Across 30 short labs, you use Hermes Agent to turn a messy Personal Knowledge Vault into a working knowledge operations system with readable notes, searchable context, reusable templates, review workflows, task boards, safety rules, and handoff docs.

Start 30 Days of Hermes Agent

2. Compositional Skill Routing

Real tasks rarely map to a single skill. They usually need several skills composed together, yet most skill routing still treats the problem as picking one tool from a library. This work formalizes Compositional Skill Routing, where an agent must select and sequence multiple reusable skills from large libraries to satisfy a complex query, and introduces SkillWeaver, a decompose, retrieve, and compose pipeline built around it.

A three-stage pipeline: SkillWeaver decomposes a query into sub-tasks with an LLM, matches each sub-task to a skill using a bi-encoder with FAISS indexing, and then performs dependency-aware planning to assemble an executable plan.
A realistic benchmark: The authors release CompSkillBench, a benchmark of 300 compositional queries over 2,209 real MCP server skills spanning 24 functional categories, so routing is tested against actual tool ecosystems rather than toy libraries.
Decomposition is the bottleneck: Task decomposition quality emerges as the primary limiting factor, and Iterative Skill-Aware Decomposition, which feeds retrieval information back into the decomposition step, lifts accuracy from 51.0% to 67.7%.
Why it matters: As agent skill libraries scale to thousands of entries, single-tool routing stops being enough, and treating routing as a compositional planning problem is what lets agents handle genuinely multi-step requests.

Paper | Tweet

3. PreAct

Computer-using agents drive real software through the screen, but they solve every task from scratch. Ask one to repeat a task and it re-reads the screen and re-reasons every tap, paying the full cost again. PreAct fixes this by compiling the first successful run into a small state-machine program, where states check the screen and transitions act, then replaying that program on later runs instead of invoking the agent.

Compile runs into a state machine: A completed task is captured as an explicit program rather than a free-form trace, turning a one-off solution into a reusable artifact that can be executed deterministically.
Replay with no per-step model calls: Replaying the compiled program runs 8.5 to 13 times faster than the agent because it needs no per-step language-model calls on repeated tasks.
Safe by construction: At each step PreAct checks that the screen matches what the program expects before acting, and hands control back to the agent the moment something is off, and it only stores programs an independent evaluator confirms solve the task from a clean state.
Why it matters: This turns computer-using agents from interactive tools that re-reason everything into repeatable operational systems, which is exactly what is needed to deploy them on recurring real work.

Paper | Tweet

4. Can LLM Agents Infer World Models?

Can an LLM agent actually build a model of an environment it cannot see? This work makes that question gradeable through agentic automata learning. An agent has to uncover a hidden deterministic finite automaton by interacting with an oracle through two interfaces, membership queries that ask whether a string belongs to the target language, and equivalence queries that ask whether a proposed automaton is correct, which yields a clean, scalable testbed for interactive discovery.

A gradeable world-model test: Casting world-model inference as DFA learning gives objective success criteria and measurable interaction efficiency, with classic automata-learning algorithms as strong, well-understood baselines.
Controlled, scalable difficulty: The size of the hidden automaton acts as a difficulty knob, so the benchmark can scale task complexity smoothly rather than relying on a fixed set of puzzles.
Agents lag classic algorithms: Current agents can sometimes perform non-trivial interactive discovery, but performance drops sharply as DFA size grows, and trajectory analyses reveal recurring failures in query planning, evidence integration, and hypothesis construction.
Why it matters: Reasoning models clearly beat non-reasoning ones here, but the large gap to classic algorithms shows that systematic, interactive world-model building is still an unsolved capability rather than a byproduct of scale.

Paper | Tweet

5. From Trainee to Trainer

Who should design the training environment for an RL agent, the practitioner or the policy itself? RL pipelines for LLMs usually rely on manually redesigned environments between stages, with practitioners guessing which configuration will best improve the current policy. This paper hands that job to the model, proposing an LLM-as-Environment-Engineer framework where the policy diagnoses its own weaknesses and proposes the next environment to train on.

The policy designs its own curriculum: Instead of a human reshaping the environment between stages, the current policy analyzes failure trajectories together with contextual information and proposes modifications to the next-stage training environment configuration.
Failure-driven environment edits: Because the proposals are grounded in the policy’s actual failure modes, the curriculum targets the specific gaps holding the model back rather than generic difficulty bumps.
The trainee becomes the trainer: A key finding is that the current RL checkpoint serves as a better environment engineer than the original base model, suggesting that learning to act also improves the model’s ability to diagnose what it still cannot do.
Why it matters: Manual between-stage environment design is one of the least scalable parts of RL for LLMs, and letting the policy steer its own curriculum closes a slow human-in-the-loop step that has bottlenecked agentic RL.

Paper | Tweet

6. OpenClaw-Skill

Equipping LLM agents with effective skills is most of the battle in real systems, yet most skill-induction work distills one trajectory at a time, which produces narrow, brittle skills. OpenClaw-Skill introduces Collective Skill Tree Search, a tree-search-based skill construction framework that builds a structured, diverse, and generalizable tree of skills, then trains agents to actually use what it builds.

Collective Skill Tree Search: Rather than distilling a single trajectory into a single skill, CSTS searches over a tree of candidate skills, using multiple models to generate and evaluate them so the library captures diverse strategies.
A structured, reusable skill tree: Organizing skills hierarchically yields competencies that generalize across tool use, multi-step reasoning, and environmental interaction instead of overfitting to one task.
Training agents to leverage skills: Building the tree is only half the work, so the framework pairs construction with a learning step that teaches agents to retrieve and apply the constructed skill hierarchy effectively.
Why it matters: Reusable skill libraries are becoming the backbone of capable agents, and moving from per-trajectory distillation to collective tree search is a concrete recipe for libraries that stay useful as tasks grow.

Paper | Tweet

7. Back on Track

Diffusion large language models generate text in a way that does not fit cleanly into the reinforcement learning recipes built for autoregressive models, and training them to reason exposes two specific problems. Rewards are sparse, so a single terminal reward fails to guide intermediate generation steps, and policy updates sometimes drift toward unnatural trajectories rather than authentic generation paths. This paper proposes Process Aligned Policy Optimization to fix both.

Two failure modes named: The work isolates sparse rewards and trajectory drift as the core obstacles to stable RL training for reasoning in diffusion LLMs, rather than treating training instability as a black box.
Step-aware process rewards: PAPO converts terminal rewards into granular, step-level guidance, so intermediate denoising steps receive a learning signal instead of waiting for a single end-of-sequence score.
Entropy-guided re-enactment: At critical high-uncertainty moments, the method replays genuine generation paths, keeping updates aligned with how the model actually produces text instead of chasing artificial trajectories.
Why it matters: Diffusion LLMs are a serious alternative to autoregressive models, and giving them a stable RL recipe for reasoning, with reported gains from 4.5% to 42.2% on benchmarks like GSM8K and MATH500, helps close the reasoning gap between the two paradigms.

Paper

8. AtomMem

Long-term memory for LLM agents tends to fail in two ways: coarse summaries drift over time, and unconstrained updates corrupt what was already stored. AtomMem keeps the unit of memory small, using a Fact Executor that selectively extracts high-value atomic facts from long interactions and organizes them into hierarchical event structures and temporal user profiles, with an associative memory graph that reconnects fragmented memories at retrieval. The approach reports state-of-the-art results on the LoCoMo long-term memory benchmark.

Paper | Tweet

9. Beyond Domains

LLM web agents usually run as tool callers, reading a fresh page each turn and emitting one low-level action, so both task horizons and the number of LLM completions blow up. This work makes web skills reusable across sites with SkillMigrator, which stores induced skills as transferable interaction patterns keyed by page-layout structure rather than instruction similarity or site metadata, so a skill learned on one site fires on new sites with the same interaction shape. It cuts the average LLM-action count by 8 to 10% on WebArena and Mind2Web at comparable success rates.

Paper | Tweet

10. The Stanford EDGAR Filings Dataset

Clean, long-context documents remain scarce for pretraining, especially in finance. This release reconstructs U.S. SEC corporate and financial disclosures into layout-faithful, token-efficient MultiMarkdown, publishing 152B tokens in SEFD-v1 out of an estimated 550B-token archive spanning 18.5M filings, with less than 0.1% overlap with Common Crawl corpora. It also ships two derived benchmarks, EDGAR-Forecast for numerical forecasting and EDGAR-OCR for financial table transcription, to support financial reasoning, forecasting, and document understanding.

Paper

🤖 AI Agents Weekly: GLM-5.2, Claude Code Artifacts, Qwen-Robot Suite, Codex Skills, Block's Builderbot, SpatialClaw, and More

elvis — Sat, 20 Jun 2026 15:45:25 GMT

In today's issue:

Z.ai open-sources frontier GLM-5.2
Claude Code ships interactive Artifacts
Qwen launches the Robot Suite
Codex turns demos into skills
Block's Builderbot writes 15% of code
Flue 1.0 reimagines the agent harness
Vercel debuts the eve framework
Cursor launches Origin code hosting
Perplexity adds Brain memory to Computer
OpenRouter ships the Fusion API
NVIDIA's SpatialClaw codes spatial reasoning
ENPIRE self-improves robot policies
SkillsBench 1.1 audits agent skills
DeepMind maps an AI Control Roadmap
OpenAI trains broadly beneficial models
Anthropic measures returns to expertise

And all the top AI dev news, papers, and tools.

Autonomous Long-Running Coding Agents

elvis — Mon, 15 Jun 2026 20:44:55 GMT

Autonomous coding is moving from better prompting to better control systems. The important shift is that engineers are learning how to wrap agents in goals, evaluators, loops, and artifacts that let them keep working after the human stops typing.

This matters because most serious engineering work spans long horizons: ambiguous requirements, hidden constraints, partial failures, changing context, and repeated verification. The new frontier is designing the system around the agent so it can plan, execute, check its work, recover from mistakes, and keep making progress without constant human steering.

This piece is based on a DAIR.AI Academy session on autonomous long-running coding agents, where I walked through Claude Code’s /goal mode, the newer /loop command, verifiers, artifacts, and orchestration patterns in practice. Written in collaboration with Codex and Claude Code.

From Prompting to Goal Design

The core idea behind features like Claude Code’s /goal is simple. A coding agent remains the executor, but the human no longer interacts with it turn by turn. Instead, the human specifies the desired end state, the evidence required to prove success, the constraints that must not be violated, and, where possible, the number of turns and budget.

That goal works more like a contract than a longer prompt. A weak goal gives the model room to stop early, take shortcuts, or redefine success in a way that looks plausible in the transcript but fails in the real system. A strong goal gives the agent a target it can repeatedly measure itself against.

Engineering judgment still matters here. The best goals encode domain knowledge that the model would otherwise guess. For a research experiment, that might mean a target benchmark score, a held-out evaluation, a required loss curve, and a rule that the result must beat an initial baseline. For a UI task, it might mean a screenshot reference, concrete layout constraints, and a browser verification step. The model can execute, but the human still defines what “done” actually means.

The Evaluator Becomes a First-Class Component

Long-running agents need a second role besides the goal. That evaluator can be another coding agent, an LLM-as-judge, a script, a test suite, a benchmark harness, or a mix of all of them. The key design choice is matching the evaluator to the task. When success is crisp, deterministic checks are better. Type checks, unit tests, lint rules, integration tests, and benchmark scripts should be used whenever they can express the condition clearly.

When success is fuzzy, an agent evaluator becomes useful. A script can tell you whether tests pass, but it cannot easily decide whether a generated research report is coherent, whether an implementation faithfully follows a paper, or whether a UI matches a design intent. This is where the evaluator benefits from language, judgment, and sometimes vision.

The practical pattern uses deterministic checks as the floor and agent evaluation as the higher-level review. That combination reduces hallucinated success while still allowing autonomy on tasks that do not fit cleanly into a test assertion.

Verifiers Define the Boundary of Trust

The deeper point is that autonomy only works when the system has a reliable verifier. A coding agent can generate a plan, implement a feature, and explain why it believes the work is complete, but that explanation should not be treated as evidence. Evidence comes from an external check that the agent cannot easily talk its way around.

For code, the verifier might be a test suite, type checker, benchmark, browser run, screenshot comparison, or reproducible script. For research work, it might be a held-out evaluation, a reproduced table, a loss curve, or a benchmark score that improves over the baseline. For design work, it might be a reference screenshot plus a visual review step. The verifier is what turns a long-running agent from a confident text generator into a system that can be trusted with more time.

Most shortcuts appear at this boundary. If the verifier is vague, the model will often satisfy the easiest interpretation of the task. If the verifier is too narrow, the model may overfit to it and miss the broader intent. A good autonomous workflow, therefore, needs layered verification, with cheap deterministic checks catching basic failures and higher-level review catching judgment-heavy failures. A few of the frontier models can already achieve some level of verification, but based on my research, there is still an evident OOD problem, where if the verification task you assign to the agent falls outside the training distribution, models struggle significantly.

Verifiers are still an open area of research, but I anticipate more companies will start to make huge investments in this area. The concept of fine-tuned verifiers is also in high demand in the enterprise.

Loops Make Autonomy Durable

A goal gives the agent direction, but a loop keeps the work alive. This distinction is important because models often stop before the real task is finished. They may hit a turn limit, lose confidence, exhaust context, or decide that a partial solution is enough.

The loop is the outer control system. It wakes up, inspects progress, runs checks, compares the result against the goal, and sends the agent back in with the next instruction when the goal has not been met. In its simplest form, this is the Ralph loop pattern with a coding agent and a deterministic condition. In a more flexible form, the loop includes an evaluator agent that can reason about progress and decide what should happen next.

Long-running autonomy works as repeated effort under supervision from a control layer, not as one continuous act of intelligence. The agent can still fail, but the loop gives the system a way to notice the failure and continue instead of silently declaring victory.

Planning Is Where Expertise Enters

One of the strongest themes from the session was that planning remains critical. You can ask a frontier model to generate a plan, but you still need to inspect it, challenge assumptions, and make the success criteria sharper before handing the task to an autonomous loop.

This leads to a useful division of labor. A stronger planning model can help define the goal, identify missing constraints, and structure the evaluation. A different execution model can then run the implementation once the plan is clear. In practice, this means engineers should stop thinking of “the model” as a single choice. Model choice becomes an architecture decision.

Some models are better planners. Some are better executors. Some are cheaper evaluators. Some are better at vision-based review. A good orchestrator lets you swap these roles instead of waiting for one vendor to provide the perfect coding agent interface.

Visual Artifacts Become Control Surfaces

Terminal transcripts do not scale when many agents are running. Once you have several sessions working in parallel, raw text becomes a poor interface for understanding progress.

Live artifacts matter because a dashboard with loss curves, benchmark scores, task states, screenshots, cost estimates, and recent decisions gives the human a much better way to supervise autonomy. The artifact becomes the control surface for deciding when to intervene, rather than a report generated after the fact.

The most useful pattern is to separate storage from presentation. Markdown or a vault can store durable evidence, logs, notes, plans, and results. HTML artifacts can render that state into something visual and interactive. The agent can search the Markdown, while the human can monitor the artifact.

For UI and product work, visual cues are especially powerful. A screenshot reference can communicate design intent more precisely than prose, and a vision-capable evaluator can compare the implementation against that reference. This reduces the common failure mode where the agent technically implements the requested component but misses spacing, hierarchy, alignment, or product feel.

Session Mining Turns Usage Into Memory

Another important insight is that past agent sessions are a rich source of workflow data. If an agent repeatedly fails in the same way, forgets to run the same check, uses the wrong path, or retries the same broken command, that pattern should not stay buried in logs.

Session mining turns those transcripts into operating rules. An agent can scan the last thirty days of work, find recurring failure modes, and propose updates to project instructions, vault learnings, or agent rules. This is how a team can gradually improve its harness without manually remembering every mistake.

The goal is to make the local environment smarter without training a model from scratch. A small rule in an agent instruction file can prevent repeated failures across future sessions, especially when the rule is specific to the project.

A Practical Operating Model

For AI engineers, the emerging workflow looks like this.

Start with a small, cheap subset before launching the full autonomous run.
Write a goal with measurable success criteria, explicit constraints, and a turn or time budget (where possible).
Separate the executor from the evaluator so implementation and judgment are not collapsed into one role.
Define external verifiers before the long-running loop starts.
Use deterministic checks wherever possible, then add agent review for fuzzy criteria.
Require proof artifacts such as logs, screenshots, benchmark curves, or changed files.
Mine past sessions and promote repeated lessons into project instructions.

That is the difference between using a coding agent and engineering an autonomous coding system. One gives you a conversation. The other gives you a harness.

What Still Breaks

None of this removes the hard problems. Agents still take shortcuts. They still stop early. They still overestimate completion. They still produce confident but weak plans, especially on recent papers, unfamiliar benchmarks, or systems outside their training distribution.

Trusting them more will not solve this. Better control systems will. Goals, loops, evaluators, deterministic checks, visual artifacts, and session memory are all ways of making autonomy observable and correctable.

The direction is clear. The future of coding agents depends on better orchestration around more capable models, where engineers design the conditions under which agents can safely run for hours or days and still produce work that can be verified.

🥇Top AI Papers of the Week

Sun, 14 Jun 2026 15:00:54 GMT

1. MiniMax Sparse Attention

Ultra-long context is now a core requirement for agents, codebase-scale reasoning, multimodal workflows, and persistent memory, but dense softmax attention still makes million-token deployment expensive. MiniMax Sparse Attention (MSA) tackles this by adding blockwise sparsity on top of Grouped Query Attention, with a lightweight routing branch that chooses which key-value blocks each query group should actually attend to.

Two-branch attention design: The Index Branch scores the full causal context and selects Top-k key-value blocks independently for each GQA group, while the Main Branch performs exact sparse attention only over those selected blocks.
Hardware-aware implementation: The paper co-designs the sparse pattern with GPU kernels, using exp-free Top-k selection and KV-outer sparse attention to improve tensor-core utilization under block-granular access.
Large speedups at scale: On a 109B-parameter natively multimodal model, MSA matches GQA performance while reducing per-token attention compute by 28.4x at 1M context. The paired kernel reaches 14.2x prefill and 7.6x decoding wall-clock speedups on H800.
Why it matters: Long context is only useful if it can be served cheaply. MSA is compelling because it keeps the mechanism simple, trains it directly into a production-scale model, open-sources the inference kernel, and powers the released MiniMax-M3 model.

Paper | Tweet

Message from the Editor

We just released 30 Days of Hermes Agent, a hands-on lab that teaches agent workflows in a real, interactive terminal. Across 30 short labs, you use Hermes Agent to turn a messy Personal Knowledge Vault into a working knowledge operations system with readable notes, searchable context, reusable templates, review workflows, task boards, safety rules, and handoff docs.

Start 30 Days of Hermes Agent

2. Self-Harness

Most agent scaffolds are built once by hand and then frozen, even as the underlying models keep changing. This paper introduces Self-Harness, a paradigm where an LLM agent improves its own operating harness, the prompts, tools, memory, and orchestration around the base model, without human engineers or a stronger external agent. Because every model fails in its own way, the system mines those model-specific weaknesses and turns them into concrete, executable harness edits rather than generic advice.

A three-stage self-improvement loop: Self-Harness runs Weakness Mining, which clusters execution traces into model-specific failure patterns, then Harness Proposal, which generates diverse but minimal edits tied to those failures, then Proposal Validation, which accepts edits only after regression testing on held-in and held-out splits.
Consistent gains across base models: On Terminal-Bench-2.0, held-out pass rates rise for every model tested. MiniMax M2.5 improves from 40.5% to 61.9%, Qwen3.5-35B-A3B from 23.8% to 38.1%, and GLM-5 from 42.9% to 57.1%.
Weaknesses become edits: Rather than appending generic instructions, the loop converts each observed failure mode into a targeted change to memory, tools, or prompts, with reported relative improvements as high as 138%.
Why it matters: As models proliferate and evolve, hand-tuning a bespoke harness for each one does not scale. Self-Harness shows the scaffold itself can be made to adapt, closing the gap between a frozen harness and the model it wraps.

Paper | Tweet

3. Agents’ Last Exam

From Berkeley RDI, Agents’ Last Exam (ALE) is a living benchmark built to measure whether agents can do economically valuable work, not just score well on academic tests. It was assembled with more than 250 industry experts and maps over 1,000 verifiable tasks to the U.S. federal occupational taxonomy, organized as 55 subfields across 13 industry clusters. Every task has an objective, checkable outcome, so there is no subjective human grading, and the pool is designed to keep growing as new workflows are onboarded.

Grounded in real occupations: Tasks are defined against O*NET and SOC 2018 and span non-physical industries, deliberately targeting the professional workflows where agents would actually be deployed rather than puzzle-style problems.
Three difficulty tiers: Work is split into Near-Term, Full-Spectrum, and Last-Exam tiers, letting the benchmark track both near-term usefulness and the long tail of hard, multi-step jobs.
Far from saturated: The hardest tier sits at just a 2.6% average full pass rate across mainstream harnesses, and even strong setups like Codex with GPT-5.5 score below 50% on the easiest tier and under 10% on the hardest.
Why it matters: Strong scores on existing benchmarks have not translated into economically meaningful deployment. ALE reframes evaluation around verifiable, expert-curated work, giving a moving target that should resist saturation as agents improve.

Paper | Tweet

4. How AI Agents Reshape Knowledge Work

This economics paper, drawing on large-scale production data from Perplexity, studies how the shift from conversational assistants to autonomous agents is reshaping knowledge work. It compares Search, a conversational assistant, with Computer, a general-purpose agent system, along three dimensions: autonomy, efficiency, and the scope of tasks people take on. The framing is a cost-structure model in which agents carry higher fixed and delegation costs but lower per-step marginal costs, so they win once tasks are complex enough.

Autonomy looks different in practice: Computer performs around 26 minutes of autonomous machine work per session versus roughly 33 seconds for Search, and per-query dissatisfaction is 55% lower on the agent, 1.3% against 2.9%.
Large efficiency gains: On matched tasks, Computer cuts completion time from 269 to 36 minutes, an 87% reduction in time and about a 94% reduction in cost relative to humans working with Search alone.
Scope shifts upward: Agent queries are more cognitively complex, 71% abstract or non-routine versus 53%, with twice as much create-level work, and they bundle interdependent subtasks that cross occupational boundaries.
Why it matters: The data supports a clean prediction. As the fixed costs of delegation fall, agents move the affordable value frontier toward higher-value, multi-step knowledge work, which is exactly where adoption grew fastest, reaching 84 times its first-week volume over the study.

Paper | Tweet

5. Agentopia

Agentopia is one of the most ambitious agent-society testbeds yet, a 79-page release that drops 100 LLM agents into a persistent world and lets them live, form relationships, and pursue goals over 10 simulated years, a horizon orders of magnitude longer than prior day-level work. Beyond observing emergent social behavior, the authors use the simulation as a training signal, optimizing models toward a life reward that reflects human well-being via rejection sampling.

Long-horizon by design: Where earlier agent societies ran at the granularity of days, Agentopia simulates a decade of life per world, surfacing unscripted social strategies and interpersonal dynamics that only appear over long timescales.
Simulation as a training signal: The life-reward metric is used to fine-tune more anthropomorphic models, and the improvements transfer beyond the simulation to downstream role-playing benchmarks rather than staying trapped in the sandbox.
Measured gains: Trained agents improve overall CoSER Test performance by 15.6%, with the biggest jumps in Anthropomorphism at 23.7% and Character Fidelity at 16.4%, and they are respected by 24.2% more peers and liked by 15.9% more.
Why it matters: A single 10-year, 100-agent run consumes 13.7 billion tokens across 567,000 LLM calls. That scale is a statement about where agent research is heading: living, learning populations as both an object of study and a source of training data.

Paper | Tweet

6. The Geometry of On-Policy Distillation

On-policy distillation (OPD) has become one of the most discussed post-training recipes of the year, but it has mostly been treated as a black box sitting somewhere between supervised fine-tuning and RL. This paper opens it up, characterizing how OPD changes a model’s weights at the level of parameter geometry, and argues OPD is not a midpoint between SFT and RLVR but its own distinct kind of update.

It touches fewer weights: Compared with SFT, OPD updates affect far fewer parameters and largely avoid the dominant principal directions of weight space, which helps explain its sample efficiency.
Early subspace locking: OPD’s cumulative updates rapidly collapse into a narrow, low-dimensional subspace early in training, rather than spreading across many directions as SFT does.
That subspace is functionally sufficient: Constraining training to the early-formed subspace preserves OPD performance but substantially degrades SFT, showing the small subspace genuinely carries the useful signal rather than being an artifact.
Why it matters: Knowing where in weight space OPD does its work turns a popular but poorly understood recipe into something with a mechanistic account. That makes the method easier to reason about, combine with other objectives, and improve deliberately instead of by trial and error.

Paper

7. Lookahead Sparse Attention

Long-context decoding is bottlenecked by the KV cache, which grows with every token and quickly dominates memory at extreme context lengths. This work, branded around DeepSeek-V4, introduces Lookahead Sparse Attention (LSA), which avoids storing the full KV cache by predicting which parts of the context future decoding will actually need and retaining only those query-critical chunks.

A learned, lightweight indexer: Instead of keeping everything, a small indexer proactively selects the KV chunks that matter for upcoming generation, so the physical cache stays small without discarding information the model will need.
Backbone-free training: A decoupled training strategy lets the indexer be trained on its own without loading the full backbone model, cutting the cost of adding the mechanism to a large model.
Big cache savings, no quality loss: LSA shrinks the physical KV cache to 13.5% of the full-context baseline while slightly improving accuracy by 0.6% on average, and at 500K-token contexts it suppresses more than 90% of KV-cache overhead without destabilizing reasoning.
Why it matters: Ultra-long context is increasingly the difference between a toy demo and a usable system, and memory is the wall. Predicting what context you will need, rather than keeping all of it, is a practical route to long context that fits in real hardware budgets.

Paper

8. Latent Spatial Memory

Video world models struggle to stay consistent over long horizons because explicit 3D memory usually requires an expensive pixel-space loop. Mirage instead stores scene information directly in diffusion latent space, using depth-guided back-projection and latent-space warping to maintain persistent spatial memory. The approach reports up to 10.57 times faster end-to-end generation and 55 times lower memory use than explicit 3D-memory baselines while improving long-horizon spatial consistency.

Paper

9. The Consistency Illusion

Multi-agent debate is often judged by whether the agents end up agreeing, but this paper shows that output-level consensus can hide deep disagreement in the reasoning that produced it. The authors abstract agents’ reasoning traces and decisions into four states along two axes, reasoning similarity and conclusion agreement, and flag divergent agreement, where agents reach the same answer through very different paths. Across 600 content-moderation items, divergent agreement appeared in 118 cases and separated cleanly from genuine disagreement states with a Cohen’s d of 0.80, and routing on these categories beat divergence-only methods at flagging high-disagreement cases.

Paper | Tweet

10. Beyond Scalar Rewards

Reward models usually compress a judgment into a single scalar, but this paper argues human preferences are better captured as score distributions, and proposes Z-Reward, which internalizes reasoning into a predicted distribution before scoring. A large vision-language teacher does the reasoning-heavy judgment and is distilled into a compact student for efficient deployment, with the 27B teacher reaching 89.6% human-preference accuracy and the 9B student nearly matching it at 88.6%. Used as a reinforcement learning signal, it delivers a 41.3% net preference improvement over a supervised baseline, beating GRPO and other reward methods.

Paper

AI Newsletter

🥇Top AI Papers of the Week

1. Skaling

2. Stealing Reasoning Traces

3. Mind Viruses

4. Catastrophic Remembering

Message from the Editor

5. Programmatic Tool Calling

6. Distilled Reasoning Skills

7. Harness-IF

8. Lost in Compaction

9. Cracks in the Foundation

10. CEDAR

🤖 AI Agents Weekly: DeepSeek Harness, DeepSeek-V4-Pro, Grok Bot, GLM-5.3, Gemini 3.7 Flash, Muse Glimmer, Harness Evolution Papers, and More

Top Stories

DeepSeek Open-Sources Its Agent Harness

DeepSeek Launches V4-Pro

🥇Top AI Papers of the Week

1. Model or Harness

Message from the Editor

2. Zero-Mem

3. Sample More Reflect Less

4. Harness-R1

5. DataSpace

6. Prompt-Induced Waste

7. Rehearse

8. ContinualSkillBench

9. MerchantBench

10. TokTier

🤖 AI Agents Weekly: Agent Plugins Standard, Qwen3.8-Max, Meta Muse Code, Prime Agent, LFM2.5-2.6B, Qwen-CUA, Harness Evolution Papers, and More

Top Stories

Agent Plugins Standard Launches

Alibaba Launches Qwen3.8-Max

🥇Top AI Papers of the Week

1. NOOA

Message from our Sponsor

2. ReOPD

3. Invisible Reasoning

4. Molt

5. JAXBench

6. ACM

7. Filesystem Memory Audited

8. Beyond AdamW

9. Role Drift

10. The Self-Speculating Agent

🤖 AI Agents Weekly: Kimi K3, DeepSeek-V4-Flash API, GPT-5.6 Price Cuts, Inkling-Small, YC's QM Harness, Gemini Robotics 2, Codex Security CLI, and More

Top Stories

Moonshot Open-Sources Kimi K3

🥇Top AI Papers of the Week

1. Harness Handbook

Message from the Editor

2. From Memory to Skills

3. PRO-LONG

4. Global Workspace in LLMs

5. GAMUT

6. Progressive Disclosure

7. Structured Output Collapses Diversity

8. Bad Memory in Agents

9. Copying and 2D-RoPE

10. RoboTTT

🤖 AI Agents Weekly: Claude Opus 5, OpenAI x Hugging Face Security Incident, Gemini 3.6 Flash, Sakana Fugu-Ultra, Progressive Disclosure, Cursor Router, and More

Top Stories

Anthropic Ships Claude Opus 5

OpenAI Models Breach Hugging Face

🥇Top AI Papers of the Week

1. Self-Improving Agents Survey

From DAIR Academy

2. Metacognition in LLMs

3. When Is Routing Meaningful

4. Harness Evolution Rethought

5. Tracing Agentic Failure

6. Failure as a Process

7. LingBot-World 2.0

8. Filtered Sabotage Monitoring

9. GFlowRL

10. LingBot-VLA 2.0

🤖 AI Agents Weekly: Kimi K3, Thinking Machines Inkling, Agentic Misalignment, Perplexity SPACE, Sunday ACT-2, GPT-Red, and More

Top Stories

Moonshot Open-Sources Kimi K3

🥇Top AI Papers of the Week