Raven.

Harnesses and the problem

• Model plus harnessAn agent's practical capability therefore depends on both its model and its harness: the tool interfaces, context management, skills, execution policies, and recovery mechanisms surrounding the model.
• Manual harnesses don't scaleSupporting more tools and execution conditions increases the number of interacting design choices that must be configured and validated, making manual harness development difficult to scale across models and domains.
• Multi-agent failuresAnalyses of multi-agent failures identify inter-agent misalignment and inadequate task verification as recurring problems.
• The question shiftsThe central question thus shifts from how to engineer a stronger harness for one domain to how to autonomously construct specialized harnesses, improve them through experience, and orchestrate them across domains.

How Raven works

• The harness of harnessesWe introduce Raven, The Harness of Harnesses, an open-source multi-agent ecosystem that automatically constructs and evolves modular harnesses for specific models and domains, treating each executable model–harness pair as a composable unit of intelligence.
• The Host AgentIts Host Agent decomposes goals, matches subtasks to specialized agents, coordinates execution dependencies, and integrates results.
• Plans as a graphThe Host Agent organizes a collaboration by matching subtasks to registered capabilities and representing their dependencies as a directed acyclic graph (DAG). Each node invokes an agent, and edges identify the dependencies between their outputs and subsequent work.
• Memory and skillsA host archive and EverOS preserve experience across tasks and Skill Forge makes that experience available as reusable procedures.
• Diagnose, propose, evaluateThe evolution process diagnoses failures, proposes candidate harnesses, and evaluates their behavior with a frozen task model.
• When composition helpsWe establish sufficient conditions under which complementary local capabilities, compatible handoffs, and bounded planning and execution errors permit the composed system to solve tasks that the available individual agents cannot reliably solve alone under the same budget.

What it achieved

• First on all four graph metricsRaven ranks first among the compared systems on all four graph metrics under both tested backbones, with Exact Match gains of 10.4 and 10.5 percentage points over the strongest baseline for the two backbones.
• 0.923 vs 0.776 Node F1With Qwen3.8-27B, Raven reaches 0.923 Node F1 against 0.776 for the strongest baseline, a margin of 14.7 points on a percentage scale.
• 0.711 vs 0.607 exact matchOn Qwen3.8-27B, Raven achieves an Exact Match rate of 0.711, compared with 0.607 for the strongest baseline, a 10.4-percentage-point improvement.
• Evolution improves 7 of 7The evolved harness improves held-out Pass@1 on all seven benchmarks. The largest gains are 15.4 percentage points on AppWorld, 13.9 on BrowseComp+, and 13.7 on LiveCodeBench.
• Research: +6.6 / +3.3 / +7.6 ppRaven-Research achieves the highest pooled accuracy with all three shared backbones, exceeding the strongest baseline by 6.6, 3.3, and 7.6 percentage points, respectively.
• On-call: 14 of 17 tasksRaven-Oncall solves 14 of the 17 scientific tasks with Claude Opus 5, compared with 11 for Claude Code, while reducing average runtime, token use, and cost.
• Skills help more in RavenOn SkillsBench, Raven gains 6.5 and 13.4 points with Qwen3.5-27B and Qwen3.5-397B-A17B, compared with 4.2 and 5.8 points for OpenClaw.

Limits

• Six pass the significance barSix comparisons pass the source's paired-gain criterion. SWE-bench Verified also improves, although its 26-task test split does not pass that criterion.
• Not best everywhereDeepSeek-Harness leads Raven-Research by 3.4 percentage points on xBench-DeepSearch.
• Judged manuallySuccess is judged manually against task-specific reference values and supporting evidence withheld from the agent.