Human-on-the-loop and autonomy levels

What is human-on-the-loop?

Human-on-the-loop is the middle rung of a three-rung ladder. The system acts on its own, a person watches during the run, samples the results afterwards, and can correct a case or stop the whole thing. Nobody signs off on each action before it happens.

The rungs on either side are easier to picture. In the loop, the human acts before every action: the agent proposes, you approve, then it goes out. Out of the loop, the system acts and nobody looks at the individual cases, either because you review a monthly total instead, or because you review nothing.

Human-in-the-loop already has its own entry here, so this one covers the rest of the ladder: how you pick a rung, what on-the-loop supervision needs in order to be real, and how it stops working without anyone noticing.

That last part is worth saying up front. On the loop is where most agent projects land by default, because approving everything is too slow and approving nothing feels reckless. It is also the rung that fails quietly. In the loop, when the human stops paying attention, the process stops. On the loop, when the human stops paying attention, the process carries on and looks fine.

Choose the level per action, not per system

The usual mistake is to set autonomy once, for the agent as a whole. An agent that reads your ERP, drafts a mail and books a credit note does three things with very different consequences, and one setting for all three is either too tight to be useful or too loose to be safe.

Singapore's Model AI Governance Framework for Agentic AI, published by the IMDA in January 2026, contains a helpdesk case study that scores each type of action on three questions: if it goes wrong, how bad is it, can the action be undone, and is it realistic for a person to review this at each step. The low-severity, fully reversible tickets, about 60 percent of the volume, run fully automatically. A middle tier gets an engineer's sign-off before anything executes. The top tier, production deployments and permission changes, the agent does not touch at all.

Risk on its own is not enough, because volume changes the arithmetic. Say an agent books 140 small credit notes a month and gets 2 percent of them wrong. That is about 34 bad credit notes a year, and at 60 euro each you are out roughly 2,000 euro. Annoying, and cheap to catch in a monthly report. A single wrong credit note of 15,000 euro costs more than that whole year of small ones, and it happens twice. The first case argues for a sample and a report, the second for approval on every single one. Same agent, same process, opposite answers.

The IMDA framework names four categories where approval should be required whatever the volume says: high-stakes decisions, irreversible actions such as deleting data, sending communications or making payments, outlier behaviour such as an agent reaching into a system outside its normal scope, and thresholds the user sets for themselves.

The autonomy scales people are actually using

Several scales are in circulation and none of them is a standard, so name whose you are quoting.

The most cited one was published by the Knight First Amendment Institute at Columbia University in 2025, written by researchers at the University of Washington. It defines five levels by the role you take next to the agent: operator, collaborator, consultant, approver and observer. Autonomy is a design decision, separate from how capable the agent is. A model that could run the whole process does not have to be allowed to.

The IMDA framework compresses that into four levels of human involvement, and those are the easiest to explain to a team. Agent proposes and human operates. Agent and human collaborate, with approval at significant steps and the option to take over at any moment. Agent operates and human approves, with approval only at critical steps such as a payment over a set amount. Agent operates and human observes, with actions audited after the fact. In the loop is the first level; on the loop covers the third and the fourth.

A third scale comes from the security side. The Cloud Security Alliance published a draft agentic profile for the NIST AI Risk Management Framework in March 2026 with four autonomy tiers: fully supervised with approval before any action, constrained autonomy inside a predefined scope with escalation outside it, broad autonomy inside an operational boundary under continuous monitoring, and full autonomy in a constrained environment. Each tier pulls in more governance. Do not quote those tier numbers as though NIST had blessed them: NIST announced its own agent standards work in February 2026 and its guidance is still to come.

Then there are the dials your tools already ship. Claude Code has named permission modes, from Manual, which prompts on first use of each tool, through Plan and Auto, to a mode that skips prompts altogether, plus allow, ask and deny rules per tool. Microsoft's guidance for autonomous agents in Copilot Studio tells you to configure the agent to request approval or confirmation from a person before executing actions that could be sensitive. Whatever your platform calls that setting, it is your autonomy level, and it is set per action.

A ladder for one process: credit notes at a wholesaler

A distributor issues about 200 credit notes a month for damaged goods, short deliveries and pricing errors. Each one starts as a customer mail and ends as a booking in the accounting package. Here is the same process on three rungs.

1. In the loop. The agent reads the mail, pulls up the delivery note and the invoice, and prepares a credit note with the reason filled in. A person opens each one, checks it against the delivery note and books it. At three minutes each that is ten hours a month, and every mistake is caught before it reaches the customer's account.

2. On the loop. The agent books a credit note by itself when three conditions hold: the amount is under 250 euro, the quantity matches the delivery note, and the customer has no open dispute. About 140 a month clear that bar. The other 60 go into an approval queue, each with the reason the agent could not decide. Someone opens the queue every morning, works through the exceptions and reads a random five of the automatic ones. That is a bit over three hours a month, and the customer gets their money back the same day instead of next week.

3. Out of the loop. The under-250 bracket is booked and never opened again. The control moves up a level: a monthly report of credited amounts per customer and per reason, against the same month last year, with an alert when one customer or one reason jumps. Nobody reads individual credit notes any more. Somebody still has to read the shape of them.

Notice what changes between the second rung and the third. It is not the agent, which does exactly the same thing. It is whether an approval queue exists and whether anyone is expected to open it. Out of the loop is not the same as out of sight: the check moved from the case to the pattern.

What has to exist before "on the loop" means anything

Supervision needs something to supervise. An operator with no queue, no exception list and no alert is out of the loop with a job title. Four things have to be in place.

A queue with an owner and a deadline. A list of open items, each one an agent waiting on a person, with a name against it and a rule for what happens when nobody answers by four o'clock. A Teams channel where approvals scroll past does not count.

Requests you can judge in under a minute. The IMDA framework asks for approval requests that are short and clear rather than long logs or raw data, with the risk of the action stated and a confidence score where you have one. It also separates the forms an answer can take: approve or reject for a simple action, edit the plan before the go-ahead for a complex one, and a written justification before approving anything high risk.

A stop that works and has been tested. Correcting one case is not the same as halting the agent. Know where the switch is, who else can press it, and how long it takes to bite, before the morning you need it.

A default of deny when the supervision itself fails. If the approval queue is down, the approvers are unreachable, or the agent proposes an action nobody wrote a policy for, the action does not go ahead. IMDA states it plainly: deny by default when approval infrastructure fails. An on-the-loop system that quietly degrades into an out-of-the-loop system while the reviewer is on holiday was never on the loop.

Automation bias, alert fatigue and the out-of-the-loop problem

Three failure modes are well documented, and each of them makes a supervisor look present while contributing nothing.

Automation bias is the tendency to accept what an automated system produces, especially when it has been right for months. The EU AI Act names it in the text, requiring that people assigned to oversee a high-risk system stay aware of the possible tendency of automatically relying or over-relying on the output. A reviewer who has approved 400 credit notes without finding a problem is not really reviewing the 401st.

Alert fatigue is the same failure arriving from the other direction. Set the exception threshold too wide and the queue fills with items that never needed a person, so people stop reading it. Narrow the trigger instead of asking for more discipline.

The out-of-the-loop performance problem is the oldest of the three and comes from human factors research on cockpits and control rooms. Mica Endsley and Esin Kiris showed in 1995 that operators watching an automated system lose the picture of what is going on, shift from active to passive processing, and are slower to take over when the automation fails. How much control the operator kept turned out to moderate that loss, which is an argument for the middle rung over the top one. Somebody who has not touched the process in six months cannot rescue it in ten minutes.

You can measure whether this is happening to you. IMDA suggests two indicators. The human override rate, the share of agent actions people reject or change, where a rate near zero points at rubber-stamping rather than at a flawless agent. And human response time during review, where a time that keeps shrinking points at automation bias or review fatigue. Your approval tool already has both numbers.

Human oversight under the AI Act

For high-risk AI systems the EU AI Act turns this from good practice into a duty, and Article 14 reads almost like a specification for an on-the-loop setup. The system has to be designed so natural persons can effectively oversee it while it is in use, and those persons have to be able to understand its capacities and limitations, monitor it closely enough to spot anomalies, correctly interpret its output, decide in any particular situation to disregard, override or reverse that output, and interrupt it through a stop button or a similar procedure that brings it to a halt in a safe state.

Article 26 puts the matching duty on the organisation deploying the system: human oversight has to be assigned to natural persons who have the necessary competence, training and authority, as well as the necessary support. Competence and authority are the operative words. A junior placed in front of an approval queue with no mandate to say no meets neither.

Most agents a small company runs are not high-risk under the Act, so none of this binds them. The list is still the best available checklist for anything that mails your customers or writes to your ledger, and working through it costs nothing.