Harness Engineering 101: How Coding Agents Actually Work
What is an agent harness?
An agent harness is all the software around a language model that turns it into a working agent. Birgitta Böckeler of Thoughtworks put it in four words, in an article on Martin Fowler's site: Agent = Model + Harness.
The model is the part you rent. Everything else is harness: the loop that keeps it working, the tools it can call, what goes into its context window, what it is allowed to do, and how its work gets checked before anyone accepts it.
A harness is also different from a framework, and the two get mixed up. LangChain, Microsoft Agent Framework and the OpenAI Agents SDK are frameworks: boxes of parts for building an agent. A harness is the finished runtime those parts add up to. Claude Code is a harness. So is Codex CLI. The line is blurring, because several frameworks now ship a ready-made harness of their own.
Why the harness matters
Here is an experiment I wish more people knew about. Take one model and give it 169 real bug-fixing tasks from SWE-bench Verified. Keep the model weights, the tasks and the context window exactly the same. Change only the code that runs around the model. Fully solved tasks go from 43 to 72.
That result is from a paper that went up on arXiv in August, and it is the shortest answer I have to what this post is about. The model was identical in both runs. The harness was not.
I spend most of my working day around agents now, building them and using them to write code, and the first question people still ask is which model to pick. That question matters less every quarter. The frontier models sit close enough together that the software wrapped around them decides most of the outcome: what a task costs, whether the agent finishes it, and whether you can trust what it hands back. That software is the harness. Designing it is what people have started calling harness engineering.
Prompt, context, harness
The field got here in three steps, and each one wrapped the step before it.
Prompt engineering was about the words. Context engineering was about what else goes in front of the model with them: retrieved documents, memory, a summary of what happened ten turns ago. Harness engineering takes both and adds everything a model needs to act rather than talk.
The whole loop in thirty lines
The easiest way to see a harness is to write one. Below is the core of a coding agent in pseudocode. It is simplified, but every real harness I have read has this shape.
def run_agent(task, model, tools, limits):
context = [system_prompt(), project_memory(), task]
for step in range(limits.max_steps):
if count_tokens(context) > limits.window * 0.8:
context = compact(context) # summarize old turns
reply = model.generate(context, tools=tools.schemas())
if reply.is_done:
report = verify(reply) # run tests, linters, a reviewer
if report.passed:
return reply
context.append(report.as_feedback())
continue
for call in reply.tool_calls:
if not policy.allows(call):
result = ask_human(call) or "denied by policy"
else:
result = tools.run(call) # most of the time: a shell command
context.append(trim(result)) # keep the lines that matter
if same_command_failed(context, times=3):
context.append("That failed three times. Try a different approach.")
return stop_and_report(context)Count the lines that involve the model. There is one: model.generate. Everything else is harness, and every other line is a decision somebody had to make. When do you compact? How much of a 4,000-line test log does the model get to see? What does policy.allows say about git push --force? Change any of those answers and the same model behaves like a different agent.
The parts, one at a time
The loop
Plan, act, observe, repeat. The core really is that small: Mario Zechner's Pi agent keeps its whole system prompt and tool definitions under 1,000 tokens. The hard part sits around the loop. When to stop. When to ask a human. What to do when the agent runs the same failing command for the fifth time, which is exactly the case the same_command_failed line above exists for.
Tools
Tools are how the agent touches anything. For a coding agent that means reading and editing files, running commands and searching the repo. For a support agent it means the ticket system and the knowledge base.
Tool design matters more than people expect. A tool that dumps 10,000 lines of log output floods the context. A tool that returns the 20 lines around the error lets the model think. MCP has made plugging tools in easy, so the work now is choosing fewer of them and shaping what they return. In practice, for coding agents, one tool does most of the work, and it gets a section of its own below.
Context
This is the most underrated layer. Every long task eventually runs into the context limit, and what the harness does at that moment decides whether the agent finishes.
The August paper I mentioned above, "Same Model, Different Harness", is the cleanest evidence for this I have seen. Both runs used the same model weights, the same 169 SWE-bench Verified tasks, the same context capacity and the same run protocol. The new harness did two things differently. It shortened older tool results in stages as the window filled, and when it caught the agent repeating failed commands it told it to try something else.
The usual techniques:
| Technique | What it does |
|---|---|
| Compaction | Summarizes old turns once the token count runs high |
| Truncation | Trims old tool output and keeps recent output whole |
| Memory files | Notes loaded at the start of every session, like a project's CLAUDE.md or AGENTS.md |
| Sub-agents | Give a side task its own fresh context and return only the answer |
| Context reset | Clears the window completely and starts a fresh session from a written handoff file |
That last row is newer than the others, and it exists for a reason you would not guess. Anthropic's team building long-running apps found that "compaction alone wasn't sufficient". As the window filled, models started showing what they call context anxiety: wrapping the work up early because they sensed the limit coming. A clean reset with a structured handoff worked better than a summary the model knew it was running out of room behind.
Guardrails
An agent that can run commands can also delete things. Every harness picks a spot on a spectrum, and the spots are more varied than I expected when I lined them up:
- Ask before everything. Cline's default: every action waits for your approval.
- Let a classifier decide. Claude Code's auto mode (the starting mode on Pro, Max and Team plans) and Cursor's Auto-review both have a second model review actions instead of asking you each time.
- Wall it off. Codex CLI runs in an OS-level sandbox by default, limited to the workspace with the network turned off.
- Trust the user. Pi has no sandbox and no permission prompts. Its author calls it "full YOLO mode" and recommends running it in a container.
None of these is wrong. The right choice depends on how much a mistake can cost, which is a question about your machine and your data rather than about the tool.
Verification
This is the layer that lets you trust the agent without reading every line it writes. Böckeler splits it into two kinds of control. Guides steer the agent before it acts: instructions, conventions, examples. Sensors watch the result afterwards and help it correct itself: tests, linters, type checkers, review agents. In the pseudocode, project_memory() is a guide and verify() is a sensor.
OpenAI's post "Harness engineering: leveraging Codex in an agent-first world" is the far end of this. A team that started at three engineers shipped an internal beta with zero lines of hand-written code, around 1,500 merged pull requests in. Codex wrote the application, the tests, the CI configuration and the docs. The humans built the harness: the checks, the structure and the feedback loops that kept the agent on track.
The catch is that an agent is a poor judge of its own work. The same Anthropic post puts it bluntly: asked to evaluate what they have produced, agents "tend to respond by confidently praising the work", even when a human can see it is mediocre. Their answer was to split the job across three agents. A planner writes the spec, a generator builds, and a separate evaluator tests the running app with Playwright against criteria agreed before any code was written. A solo agent took 20 minutes and $9. The full harness took six hours and $200, and the result was far better. Verification does not happen by itself. Someone has to build it into the harness, sometimes as a whole second agent whose only job is to be hard to please.
Extensibility
A good harness lets you change its behaviour without forking it. Claude Code exposes more than 30 lifecycle hook events you can attach scripts to, plus skills, plugins, sub-agents and MCP servers. This is where a team's own conventions go.
Here is a real hook, the kind I would add on day one. After every file edit it runs the formatter on the file that changed:
{
"hooks": {
"PostToolUse": [
{
"matcher": "Edit|Write",
"hooks": [
{
"type": "command",
"command": "jq -r '.tool_input.file_path' | xargs npx prettier --write"
}
]
}
]
}
}Look at the command. The harness passes the event in as JSON on standard input, jq pulls out one field, and xargs hands it to another program. That is a Unix pipeline, and it is not a coincidence.
Why the shell does most of the work
My first job was Linux server administration, and what hooked me back then was how much one line could do. Chain a few small programs together and a job that sounds like an afternoon of work is finished before you have stopped typing. This is the kind of line that got me:
grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | headThat is every IP address that tried to brute-force SSH on the box, counted and ranked, from five programs that know nothing about each other. Watching a coding agent work gives me the same feeling. It reaches for the same kind of tools, in roughly the order I would:
$ rg -n "InvoiceTotal" src/
$ sed -n '118,160p' src/billing/invoice.ts
$ npm test -- invoice
$ git diff --statMy view is that this is where most of a coding agent's ability to act actually comes from. Give a model one tool, a shell, and it gets every program on the machine along with it. Nobody had to build a search_code tool or a run_tests tool. rg and npm test already existed, with decades of documentation behind them.
I think four things make the shell fit a language model so well.
First, it speaks text. Doug McIlroy wrote the rule down in the Bell System Technical Journal in 1978: "Expect the output of every program to become the input to another, as yet unknown, program." Nobody at Bell Labs was thinking about language models, but an unknown program reading your output is a fair description of one.
Second, models already know it. Fifty years of man pages, shell scripts, READMEs and forum answers are in the training data. Zechner said it plainly when he explained why Pi ships only four tools (read, write, edit and bash): "Models know how to use bash."
Third, every command reports back. An exit code of 0 or not 0 is a free sensor. The agent knows whether the test passed without anyone writing a verification layer for it.
And programs compose. A pipe turns two small tools into a third one on the spot, so the harness does not need a dedicated tool for every job.
There is one more piece of Unix doing quiet work here, and LangChain's write-up on harness anatomy calls it "arguably the most foundational harness primitive": the filesystem. A model forgets everything the moment its context ends. A file does not. A progress note, a feature list, a git log: that is how a long task survives from one session to the next, and it is files and git doing the job they have done for fifty years.
The vendors reached the same conclusion from the other direction. Anthropic describes the design principle behind its Claude Agent SDK as giving "your agents a computer, allowing them to work like humans do". Boris Cherny, who created Claude Code, has said early versions used RAG with a local vector database, and the team switched to plain agentic search (the model running grep and friends) because it worked better. By his own account that was mostly internal vibes rather than a benchmark, but it is what they shipped. Vercel cut an internal data agent down to little more than a single bash tool and reported it running 3.5x faster on 37% fewer tokens. That was on five test queries, so treat it as a direction rather than a measurement.
The shell does have limits, and they are worth knowing. It cannot click through a web app, so GUI work still needs browser or computer-use tools. A SaaS product with no CLI comes in through an API or an MCP server. And the same shell that runs npm test can run rm -rf, which is why the guardrails section exists at all.
Coding agents compared
Here are the harnesses I get asked about most, as they stand in September 2026. Defaults change fast, so check the docs before relying on any cell.
| Agent | Open source | Models | Default safety | Built-in tools |
|---|---|---|---|---|
 Claude Code | No | Claude only | Classifier on Pro, Max and Team, otherwise asks | 40+, core is Read, Edit, Grep, Glob, Bash |
 Codex CLI | Apache 2.0 | OpenAI by default, others via config | OS sandbox, workspace only, network off | Mostly shell, plus apply_patch |
 Gemini CLI | Apache 2.0 | Gemini only | No sandbox, confirms shell and writes | About 20, shell and grep among them |
 Cursor | No | Many providers | Sandboxed shell, classifier reviews the rest | Search, read, edit, shell, browser |
 OpenHands | MIT | Almost any, through LiteLLM | Docker sandbox in the web app, asks first in the CLI | Terminal, file editor, task tracker |
 Aider | Apache 2.0 | Almost any, local too | No sandbox, commits each edit to git, asks before commands | No tool loop: edit formats and a repo map |
 Cline | Apache 2.0 | Many, local too | Asks before every action | 7, with ripgrep for search |
 Pi | MIT | 15+ providers | No sandbox, no prompts | 4: read, write, edit, bash |
Read the last column top to bottom. The harnesses that lean hardest on the shell ship the fewest tools, and Codex, the most shell-centric of the big three, is also the strictest about sandboxing it. That pairing is deliberate. Aider is the odd one out: it predates tool calling and still works through edit formats and a map of the repo, which is a reminder that the loop in my pseudocode is one design among several.
Outside of code
Nothing in the first diagram is specific to software. Microsoft Agent Framework reached 1.0 in April 2026, and at Build in June it shipped a built-in harness with shell and file access, tool approval, file-based memory and automatic context compaction. LangChain Deep Agents and the OpenAI Agents SDK offer similar parts.
What changes when you leave code is mostly one row:
| Coding agent | General agent | |
|---|---|---|
| Loop | the same | the same |
| Tools | shell, git, files | APIs, browser, email, CRM |
| Context | repo, diffs | docs, tickets, chat history |
| Guardrails | sandbox | approval for sends, payments, deletes |
| Verification | tests, built in | evals, rubrics, human review |
A research agent has no test suite. A support agent cannot run npm test on a reply. The exit code that makes coding agents so easy to check does not exist, so a general harness has to build its own sensors: evaluation sets drawn from real past cases, rubric-based review agents, and points where a human signs off. If you are building an agent outside code, this is where your time should go, because it is where those agents fail, and they tend to fail quietly. I wrote a separate post on what those quiet failures look like in production.
How to judge a harness
Whether you are picking one or building your own, measure the whole stack rather than the model alone:
- Cost per completed task, not cost per token.
- Success rate on your own tasks, not on a public leaderboard.
- Long tasks: does it finish them, or stall halfway?
- Safety model: what can it break, and who approves?
- Fit: does it work with your tools and your conventions?
Cost is the one people underestimate. When Artificial Analysis launched its Coding Agent Index in May 2026, cost per task across the model and harness pairs it tested ran from $0.07 to $2.26. Most of that spread is the model, but the harness decides how many tokens the model burns on the way.
The machine underneath moves the numbers too. Anthropic's engineering team ran the same Claude model through the same harness on the same Terminal-Bench 2.0 tasks and saw a 6 percentage point gap between the tightest and the most generous container resources. So run your comparison on the infrastructure you will actually use.
Where this is going
A layer above harnesses is starting to appear. In June 2026 Databricks open-sourced Omnigent, which it calls a "meta-harness": it sits above Claude Code, Codex, Pi or your own agent and lets you combine and govern them from one place. If that idea sticks, the harness becomes a component you swap, the way you might swap one database for another.
Some of today's harness will also move into the model. The best line I have read on this is from that Anthropic post: "Every component in a harness encodes an assumption about what the model can't do on its own." The same_command_failed check in my pseudocode assumes the model will not notice it is going in circles. Compaction assumes it cannot hold a long task in one window. As models get better, some of those assumptions stop being true, and the parts built on them can go.
The part I don't expect to move is the permission boundary. A model can learn to catch its own mistakes. What it is allowed to delete on your production server stays your decision, written down in a harness, the same way it was in a sudoers file long before any of this.
If you are working out what that harness should look like for your own team, my calendar is on the contact page.
Sources
- Birgitta Böckeler, Harness engineering for coding agent users, martinfowler.com, April 2026
- Sydney Lewis, Same Model, Different Harness: Different Coding-Agent Results, arXiv, August 2026
- OpenAI, Harness engineering: leveraging Codex in an agent-first world
- Anthropic, Building agents with the Claude Agent SDK
- M. D. McIlroy, E. N. Pinson and B. A. Tague, UNIX Time-Sharing System: Foreword, Bell System Technical Journal, 1978
- Mario Zechner, What I learned building an opinionated and minimal coding agent, November 2025
- Boris Cherny on agentic search replacing RAG in Claude Code, February 2026
- Vercel, We removed 80% of our agent's tools, December 2025
- Artificial Analysis, Coding Agent Index and its May 2026 launch post
- Anthropic, Quantifying infrastructure noise in agentic coding evals
- Anthropic, Harness design for long-running application development, March 2026
- LangChain, The anatomy of an agent harness, March 2026
- Microsoft, Microsoft Agent Framework at BUILD 2026
- Databricks, Introducing Omnigent
- Tool docs for the comparison table: Claude Code tools, permission modes and hooks; Codex and its approvals and security; Gemini CLI tools; Cursor run modes; OpenHands; Aider; Cline tools; Pi