Harness².

The problem and the setup

• Model plus harnessAn LLM agent's effectiveness depends not only on its underlying model, but also on its harness: the prompts, memory, skills, tools, and orchestration through which the model performs tasks.
• Fixed harnesses break in the open worldHowever, a static harness selected beforehand often fails to generalize to the diverse, unpredictable environments encountered in the open world.
• No answer keyFirst, tasks typically lack ground-truth labels or reliable verifiers.
• One fix doesn't teach the nextSecond, improving a harness for one task does not automatically improve how the next task is refined.
• The central questionCan recursive self-improvement improve both the current harness and how future harnesses are improved?
• Eight editable partsexposing K = 8 components: a system prompt, guardrails, memory, skills, tool guidance, scripts, subagents, and a middleware plugin. Each component is mounted as a file in the runtime.
• Two levels of recursionTask-level recursion refines the current task's execution harness. … Harnessing-level recursion uses the resulting task-level explorations to update a continual improvement harness.
• WorkBuddy-BenchWorkBuddy-Bench (Team et al., 2026) covers office-style deliverables in three domains (Code, Web, Office).
• JobBenchJobBench (Li et al., 2026) targets end-to-end professional job tasks with long executions.
• LAB, legal workLAB (Harvey AI, 2026) is a legal-work benchmark; following its protocol we report Partial, the per-criterion pass rate.
• Three shuffled runsMain tables report means and standard deviations over three runs, each with a shuffled task order.
• Frozen executors, off-the-shelf harnessesGemini 3.5 Flash and GLM-5 are frozen executors under the off-the-shelf OpenCode and Codex harnesses. … The default harness editor is Claude Opus 4.8 running under OpenCode.
• Starts emptyThe harness improvement library starts empty for each benchmark stream and is updated after each task.

Propose, probe, compose, remember

• Reference runThe reference reveals how the harness behaves, not whether its deliverable is correct.
• The general candidateThe invariant candidate targets agentic deficiencies that may recur across domains, such as planning, tool use, context management, or delegation.
• The domain candidateThe domain-specific candidate targets the knowledge, procedures, and constraints of the current domain or environment.
• Probe under matched conditionsEach candidate harness is executed on the same task under matched model and environment configurations.
• Evidence, not a gradeThese contrasts inform subsequent composition rather than verify task correctness.
• Compose selectivelyComposition is selective: it may merge supported changes, revise conflicting edits, adopt one candidate, or preserve the current harness.
• Every decision has a reasonEach decision is tied to a concrete observation rather than a presumed score gain.
• Wide or deep(a) Parallel recursion probes multiple candidates and independently composes successor harnesses. (b) Sequential recursion carries each refined harness and accumulated execution evidence into the next step.
• Adopted, not proven bestThe corresponding harness is denoted H★, where the star indicates adoption rather than verified optimality.
• Only supported lessons are keptLessons are retained only when supported by execution contrasts; selecting an output alone does not establish transferability.
• What a library entry holdsEach library entry contains an executable harness and an EXPERIENCE.md recording the applicable situation, lessons, and supporting evidence.
• Reused, not inheritedEarlier components are selectively reused or adapted through Propose, rather than automatically inherited.
• Only the library changesModel weights, stage instructions, and the decision rule remain fixed, while the library evolves.

Results

• +11.5 on WorkBuddy-BenchOn WorkBuddy-Bench, Harness² raises GLM-5's average score by 11.5 points under OpenCode, the largest one-step improvement over the base harness on this benchmark.
• JobBench and LABThe same pattern extends to JobBench and LAB, where gains reach 4.5 points and 11.8 percentage points in Partial pass rate, respectively.
• Every configuration improvesA single improvement step improves both GLM-5 and Gemini 3.5 Flash under OpenCode and Codex.
• Other methods can hurtBy contrast, ReasoningBank falls as much as 6.6 points below the base harness on coding tasks. Harness Scaling underperforms the base in seven of the twelve benchmark–configuration pairs.
• A weaker model overtakesOn WorkBuddy-Bench, a single step lifts GLM-5 to 75.7 with OpenCode and 75.3 with Codex, surpassing the corresponding base Gemini scores of 72.4 and 74.5.
• More budget, more gainParallel task-level recursion at k=5 adds a further 2.9 to 3.9 points on WorkBuddy-Bench, 3.0 to 4.2 points on JobBench, and 2.6 to 6.7 percentage points in LAB's Partial pass rate over a single step.
• Rises with budgetParallel recursion rises from 76.5 to 80.8 on Code and from 79.5 to 83.4 on Office.
• Two beats fiveNotably, both recursive forms of Harness² at k=2 outperform every plotted baseline at k=5 on both domains. … Increasing test-time exploration alone therefore does not ensure better performance.
• Cheaper than the offline methodMeta-Harness uses more compute than Harness² at k=5 for lower performance. This comparison includes both execution and improvement costs.
• Every part pulls its weightTable 3 shows that removing the library lowers WorkBuddy-Bench and JobBench scores by 3.1 and 1.8 points, respectively; removing scoping costs 2.9 and 3.7, and removing composition costs 1.6 and 1.9.
• Editing blind backfiresdirectly refining the base execution harness without access to reference executions, candidate probes, or the improvement library degrades performance on Code and Web. Specifically, Code scores drop from 73.4 to 63.2 and Web scores fall from 69.9 to 65.0.
• No stronger editor neededAll five models, from GLM-5 to Claude Opus 5, improve on every benchmark. … GLM-5 gains 13.7 points on LAB. These results demonstrate that harness refinement need not rely on a stronger editor.
• The case-study taskThe baseline run scored 43.1. Its trajectory revealed procedural failures rather than knowledge gaps: data was never reconciled across source files, and outputs were never reopened for verification after being written.
• The two interventionsthe invariant intervention acts as a process-level guardrail, forcing the model to inventory all inputs, trace figures to their exact sources, and use a verification script to reopen and check produced files. Second, the domain intervention introduces a specialized skill providing reconciliation mechanics for schedule, venue, labor, and budget files.
• 43.1 → 77.8When evaluated against the reference, the Composer retained all four edits introduced by the two interventions. The optimized re-run successfully cross-referenced the figures and restored the memo's missing sections, raising the score to 77.8.
• Skills and scripts carry the mostRemoving skills causes the largest loss on Code and Web (3.6 and 6.9 points), while removing scripts has the largest effect on Office (7.4 points).
• Adapts more than it copiesOnly 2–10% of modified files retain at least half their lines from an entry, indicating that adaptation is more common than extensive copying.
• Fewer tool errorsError rates fall on all three benchmarks, most on JobBench (14.4% → 9.1%).
• Beats production harnessesStarting from OpenCode, Harness² exceeds the strongest fixed alternative by 4.6, 5.8, and 4.9 points on Code, Web, and Office. … This comparison shows the benefit of refining an existing harness, while noting that refinement uses additional test-time compute.

Limits and failures

• Where a baseline winson LAB Partial, Meta-Harness leads for Gemini 3.5 Flash under Codex.
• Starting harness still mattersRecursion therefore does not uniformly close the gap between initial harnesses.
• Picked the weaker outputFirst, selection can prefer a weaker candidate: on WB-code, an output scoring 55 is chosen over one scoring 95; on LAB, narrower coverage reduces the score from 54 to 26.
• A wrong premise spreadsSecond, an incorrect premise can propagate across candidates.
• It costs extra computeHarness² requires additional executions and editor sessions to propose, probe, and compose harness changes. Increasing the recursion budget therefore adds inference cost.
• No guarantee of correctnessThese signals help assess the effects of a proposed change, but remain sensitive to execution variability and shared model biases. They do not guarantee correctness or establish that an adopted change will transfer to a different task.
• The library will need upkeepAs this library grows, future work could consolidate overlapping components, revise outdated experience, and refine retrieval according to the current task and available evidence.