Meta Harness

A meta harness has two uses, and they do not mean the same thing. In the research sense, Meta-Harness is an outer loop that searches over the code around a fixed model.

That code is the harness: the prompts, tool definitions, completion checks and context rules a team writes around the model. A coding-agent proposer reads earlier candidates from a filesystem, proposes edits, and each candidate is scored on a search set, and the final harness is tested on a held-out set when the benchmark provides one.

On X, a few builders also use meta harness for a layer that sits above several coding-agent harnesses and runs them. We treat the research sense as a later step in harness engineering, not a starting point. Automated search only works if you already have an eval set and per-run traces to score against, and we have not run Meta-Harness for a client. Our harness work starts by tracing a real run end to end, then changing the runtime those traces point at.

What is a meta harness?#

Lee, Nair, Zhang, Lee, Khattab and Finn submitted Meta-Harness: End-to-End Optimization of Model Harnesses on 30 March 2026, from Stanford, MIT and KRAFTON. The work is listed for COLM 2026, and a version was a poster at the ICML 2026 Agents in the Wild workshop. Read the abstract for the setup; the numbers below come from there and from the project page.

The outer loop is a harness too, since it controls what the proposer reads. On this page, harness means the program being searched over, which in their experiments was one Python file wrapped around a model whose weights never changed.

Give the proposer the raw traces, not a score, because Lee and colleagues store every earlier candidate's source, scores and execution traces on a filesystem. The proposer is a coding agent, Claude Code in their runs, and it reads what it needs with grep and cat.

A single evaluation can leave up to 10 million tokens of diagnostic context, which is why a compressed summary is the wrong input. Our harness work starts the same way, with a real run traced end to end before anything changes, and most harness decisions come out of that trace rather than out of opinion.

Our harness work does the human version of this loop. Loop engineering is the part of harness engineering that deals with repetition, and its hill-climbing loop reads traces, finds failures that repeat, and changes the harness. A person still merges that change through an eval gate, and Meta-Harness puts a coding agent in the proposer seat.

The job is the same, and so is the prerequisite: traces and a held-out set. Skip the search if you do not have those yet, because the proposer has nothing honest to read.

What did the Meta-Harness paper show?#

Their result on online text classification is a 7.7 point gain over ACE, a state-of-the-art context-management system, with 4 times fewer context tokens. The discovered harness scored 48.6% against ACE's 40.9%, and used about 45.5 thousand characters of extra context against ACE's 203.0. On retrieval-augmented math, a single discovered harness gained 4.7 points on average on 200 IMO-level problems across five held-out models, from 34.1% to 38.8%.

On TerminalBench-2, the discovered harness reached 76.4% with Claude Opus 4.6 against Terminus-KIRA's 74.7%, and 37.6% with Claude Haiku 4.5 against Goose's 35.5%. Those are the comparisons we can confirm on the abstract and the project page. We do not repeat a leaderboard rank, because the write-up and the project page disagree on how many Opus agents sat above it.

Treat those as one team's scores on their own benchmarks, not as a reason to skip your own evals, because a 7.7 point gain on their classification tasks does not predict your agent. We put an eval suite in CI built from cases pulled from real traffic, and a harness change has to clear it before it merges. A search needs that same gate, and without it the proposer is tuning on the traces it was found in and breaking the rest.

Is a meta harness the same as a coding agent harness?#

A coding-agent harness is the runtime around one model: context, tools, state, permissions and the check that says the work finished. That is the layer our harness engineering builds, and it is the thing Meta-Harness searches over. Claude Code is the proposer in their runs, and it is also a coding-agent harness in its own right, so the two sit at different layers.

Do not collapse them because the same words appear. "Agent harness" stays with the service page, and this page answers whether a meta harness is that layer: it is not.

A looser use showed up on X in early November 2026. Teknium at Nous Research wrote about building a meta-harness with hermes. Eduardo Urias wrote that his meta harness runs each chat in its own Linux microVM, whether the inner agent is Claude Code, Codex or pi.

Liran Tal asked whether a tool was a sort of meta-harness like t3code or herdr. Three posts in two days is a usage, not a standard, so we keep that sense short and do not treat it as a product category.

Put the coding-agent harness in place first: context, tools, a permission scope in code the agent cannot edit, and a trace of every run. Our loop engineering guide is where the check, the cap and the hill-climbing loop sit. A layer that only launches several coding agents, with no eval and no traces, is the X usage and not a search.

What do you need before you run a meta harness?#

Skip automated harness search until the boring layer exists. You need a labelled eval set drawn from your own traffic, which is the set our evaluation work builds, and you need a trace of every run, which is the record our observability work keeps.

The proposer reads those, and if they are missing it is guessing. Our guide to building evals starts from failures you have actually read, because a score you cannot trace to a failure does not tell you what to fix.

On Skionis, the DevOps agent we shipped on a live AWS account, every validation failure and its revision sit in one record with the request and the result. Those failures are the system's own log of what the model gets wrong, and we use them to improve the loop rather than the prompt.

On Solarpunk, each trace tells a planning defect apart from an outage, an expired session or stale state. Feed only the planning defects into any later search, because a loop that learns from outages changes the agent to work around failures that were never the agent's.

Keep the eval gate, the permission scope and the trace outside what the search can edit. A loop told to raise a pass rate can raise it by weakening the check. We enforce the permission scope in code the agent cannot edit, and a harness change clears the suite in CI like any other change.

An agent that runs a few times a month will not produce enough traces for a search to learn from, and a task a person already reviews line by line gains little from one either. Spend that effort on the verification loop and the eval set. Graph engineering is the other fork: if a person can write the steps down, write them down, and leave search for the parts that still fail after the path is fixed.