AI harness

What is an AI harness?

An AI harness is the software layer around a language model that lets it do real work. The model itself receives text and returns text. The harness handles everything around that exchange: calling tools, passing results back to the model, managing context, asking for approval, retrying failures, and deciding when the task is finished.

In simple terms: an agent is a model running inside a harness. Without the harness, the model can answer. With the harness, it can search files, call APIs, run code, update systems, and complete a multi-step task.

Microsoft's Agent Framework documentation defines a harness as "the scaffolding that turns a language model into an agent that can actually do things", and Anthropic's Agent SDK documentation points developers at its own writing on agent harness design. No standard says what a harness must contain, but the word has moved from shop talk into vendor documentation, and the shape it describes is consistent across vendors.

What a harness does

The problems are the same whether you are wrapping a coding agent or an internal data assistant. Microsoft now ships a ready-made harness inside Agent Framework, and the list of capabilities it turns on by default is a fair picture of the job.

• The loop. The harness calls the model, receives the model's next action, executes it, returns the result, and repeats until the work is done or an iteration limit is reached.

• Tool integration. It exposes tools to the model: file reads, search, SQL queries, API calls, browser actions, code execution, or workflow steps. It also validates the tool inputs before anything runs.

• Context management. It decides what goes into the context window, and what happens when the window fills up during a long tool-calling run. Context compaction is the usual answer. Anything the agent is meant to keep between sessions is a separate problem, covered under agent memory.

• Permissions. It decides which actions run automatically and which need human approval. Reading a file is different from deleting a record or sending an email. Standing "don't ask again" rules are common, so that approval fatigue does not push people into waving everything through.

• Progress tracking. Long tasks drift. Harnesses built for multi-step work keep a todo list the model updates as it goes, so the plan outlives any single model call.

• Stop conditions. It prevents runaway loops with limits on steps, time, cost, retries, or tool calls.

• Logging and traceability. It records what the model saw, which tools it called, and what changed. Agent Framework emits this as OpenTelemetry traces. Without it, debugging an agent is guesswork.

• Error handling. It turns failed tool calls, empty search results, permission errors, and malformed outputs into something the model or user can recover from.

The tool layer is becoming standardised. Both Agent Framework and the Claude Agent SDK reach outside systems over MCP, so a harness can pick up a database or browser integration without bespoke plumbing.

Harness, framework, and agent

The terms overlap, but they are not identical.

Model
The language model predicts the next response or action based on the input it receives.

Framework
A framework is a toolkit for developers. LangGraph describes itself as a low-level orchestration framework and runtime for long-running, stateful agents. The OpenAI Agents SDK is built around agents, handoffs and guardrails, with an agent loop that calls tools and feeds results back until the task is complete. Microsoft's Agent Framework is the successor to both Semantic Kernel and AutoGen, and adds graph-based workflows on top of agents.

Harness
The harness is the running wrapper around the model in a specific product or workflow. It may be built with a framework, but it includes the concrete prompts, tools, permissions, logging, limits, and error behaviour.

Agent
The agent is the user-facing system: model plus harness plus tools, aimed at a goal.

This is why the same model can feel sharp in one product and clumsy in another. The difference is often not the model. It is the harness: better tools, cleaner context, clearer stop rules, safer permissions.

Harnesses in practice

A coding assistant in the terminal is the clearest example. Claude Code and Codex do not just send your question to a model. They inspect files, run commands, keep a task plan, apply patches, ask for approval where needed, and decide when enough verification has run. Anthropic sells that surrounding runtime as a product in its own right: the Claude Agent SDK is described as the same tools, agent loop and context management that power Claude Code, made programmable. Codex runs the same idea across a terminal, an editor extension and isolated cloud environments you can hand a task to and review later.

A BI assistant can have a different harness. It may expose only approved semantic models, a SQL runner with read-only access, a charting tool, and a policy that requires a human to approve any scheduled report change.

A customer-service agent has another shape again: knowledge-base search, CRM lookup, draft replies, escalation rules, tone instructions, and a hard stop before refunds or account changes.

Where the work splits into chunks that do not need to share a context window, a harness can hand pieces off to a subagent and keep only the answer. Where it needs to run code it does not trust, it can do that inside an agent sandbox.

Build your own or use an existing harness?

Using an existing harness gets you started quickly. Vendor products usually handle tool calling, approvals, tracing, authentication, and updates better than a weekend prototype. Agent Framework's harness is deliberately opinionated for this reason: every default is on until you switch it off.

Building your own gives more control. You can decide exactly which tools exist, how context is assembled, where logs go, how permissions work, and how the agent fits internal systems. That control comes with maintenance: model changes, API changes, prompt regressions, security reviews, and production support.

A sensible path is to start narrow. Use a simple model call or an existing agent product for the first version. Build a custom harness only when you need behaviour the product cannot provide: a specific permission model, strict audit logs, unusual tools, or tight integration with internal workflows.

What to watch out for with an AI harness

Hidden autonomy
If the harness quietly lets the model take actions, users may think they are only chatting while the system is changing data. Make action boundaries explicit.

Tool descriptions
Models choose tools based on names, descriptions, and examples. Vague tool descriptions produce wrong calls.

Context clutter
More context is not better. Anthropic's guidance on context engineering treats the context window as a finite resource with diminishing returns, and aims for the smallest set of high-signal tokens that gets the outcome. A harness that stuffs every document and every prior message into the prompt buries the thing that mattered.

Runaway loops
Every agent needs hard limits. A loop with no step cap can keep spending money while making no progress.

Approval theatre
An approval prompt that fires on every action trains people to click yes without reading. Auto-approve the safe operations properly so the prompts that remain still get attention.

Auditability
If the agent touches customer data, finance data, or production systems, log the tool calls and decisions in a way a human can inspect later.