A preprint posted to arXiv on September 17, "How Do Agent Harnesses Create Value?" (2609.20474, by Yukun Zhang, Kemu Xu and Yishen Chen), tries to answer a question agent engineers argue about constantly but rarely test: how much is an agent's harness — planning guidance, execution organization, completion checking — actually worth? The answer: it depends on how expensive you make erroneous acceptance.
The method is clean. The study runs on two Retail experiments and an Airline pilot in τ²-bench. The central comparison pairs prewritt
[1]en task-specific plans (Fixed) against shuffled policy text matched in word count (Sham), isolating the contribution of guidance content rather than framework effects. Across 265 matched cells, Fixed improves oracle-verified success by 7.17 percentage points (90% task-clustered bootstrap interval 1.15–13.36), with gains concentrated in higher-complexity tasks. 7.17 points is not small in agent evaluation — but note the interval: the lower bound is 1.15, meaning the effect is unstable in some tasks.
The verifier results are more interesting. A read-only terminal verifier rejects 61% of Retail oracle-invalid episodes while withholding 17% of correct ones, at less than one cent of additional cost per episode. Put those numbers together and the value judgment appears: the verifier is cheap and catches most bad outcomes, but it also blocks some good ones. The trade-off between "rather kill a good run" and "rather let a bad run pass" depends on how seriously the product treats erroneous acceptance.
The paper's sharpest conclusion: which component matters more depends on the loss assigned to erroneous acceptance. At low liability — an internal research tool, say — the planning gain dominates. At high liability — customer-facing agents that generate downstream costs — the verifier's avoided false passes dominate, and a standalone verifier captures nearly all the false-pass benefit of the full planning-plus-verification stack at a fraction of its cost. Translated: if your fear is not "it fails" but "it fails and pretends it succeeded," the verifier is worth more than any planning prompt.
The limitations deserve stating. This is a single benchmark (τ²-bench) in a simulated environment; oracle verification is a hand-designed approximation; the 90% confidence interval is wide and the Airline pilot is small; the erroneous-acceptance loss is a scenario parameter the authors chose, and another industry might flip it. The paper does not claim planning is useless. It claims that when false passes get expensive enough, completion checking outweighs planning — a judgment testable with a cost function, not with slogans.
For teams deploying agents at scale, the paper offers an operational audit framework: write the cost of one erroneous acceptance into your model first, then decide whether money belongs in better planning or in a cheap read-only verifier. Most teams spend on prompts. This paper's evidence points the other way.
A final note on why the framing matters. The agent industry currently sells planning: longer reasoning, better prompts, fancier frameworks. This paper is a quiet argument that execution-checking infrastructure — cheap, read-only, external — is systematically under-invested relative to its return, at least whenever a wrong completion carries real consequences. That is a testable claim about where engineering budgets should go, and it deserves replication on more benchmarks and more liability regimes before teams reallocate budgets on its strength.
[1]