Product24 June 2026 · 12 min read

Human-in-the-loop AI: a gate that asks too often stops being a gate

Human-in-the-loop sounds safe until the human is approving forty things a day without reading them. Which actions should ask, which should never run alone, and what a preview has to show for your yes to mean anything.

Konstantinos Gkolias

Konstantinos Gkolias

Founder, building Lumnya

Product

Autonomy you can trust.

Lumnyaai

First published 24 June 2026; rewritten and expanded 19 October 2026. There are two futures of AI being sold to store owners, and both are wrong. In the first, an AI runs your store while you sleep, and every mistake it makes is one you discover afterwards, in your revenue, with no explanation attached. In the second, the AI only ever suggests, and every output lands in your lap as one more thing to read, edit and do yourself. The usual answer to both is “keep a human in the loop”. This piece is about why that answer, on its own, is not enough, and what has to be true for a human's yes to actually protect anything.

  • A human in the loop protects you only if the human is actually checking. Decades of research on automation say people stop checking reliable-looking systems, and that training doesn't fix it.
  • So a gate has two failure modes: it can be missing, or it can be asked so often that the yes becomes a reflex.
  • The answer is to gate by consequence, not by default: how hard the action is to undo, and whether it reaches a customer or money.
  • Some actions should never run without a yes, at any setting. In Lumnya those are refunds, return decisions, emails to customers and live-chat messages.
  • A yes means something only if the preview is falsifiable: the exact change, checkable in two seconds. “Update pricing?” isn't a question you can answer. “520.00 → 572.00” is.

Autonomy without accountability is a liability. Autonomy with a receipt is a colleague.

The human in the loop stops looking

Human factors researchers have studied what happens when people supervise automated systems for decades, in cockpits, control rooms and clinics. A widely cited review by Raja Parasuraman and Dietrich Manzey, “Complacency and Bias in Human Use of Automation” (Human Factors, 2010), pulled the findings together. Two of them matter for anyone putting an AI in charge of part of a store.

  • Complacency. When a system is usually right, people monitor it less than they should, especially when they have other things competing for their attention. Expertise and practice don't remove the effect.
  • Automation bias. People over-rely on an automated recommendation, both missing problems the system didn't flag and following advice they would have rejected had they worked it out themselves. Training and instructions don't reliably remove it either.

Now picture a store owner with an AI that asks for approval forty times a day, and is right thirty-nine of them. By the end of the second week, the approve button is a reflex. The gate is still there, technically. It has stopped doing its job. A gate that asks about everything protects you from nothing, because the one request that deserved attention arrives looking exactly like the thirty-nine that didn't.

The opposite failure is better known. In July 2025, during a public trial, an AI coding agent on Replit deleted a live production database, despite an instruction not to make changes without approval, and then misreported what had happened. The instruction was in the conversation. Nothing outside the model enforced it. We wrote about that difference, between a rule the model is asked to follow and a rule the server enforces, in A prompt is not a contract.

This is no longer only an academic concern. The EU's AI Act writes it into law for high-risk systems: Article 14 requires that people overseeing such a system can “remain aware of the possible tendency of automatically relying or over-relying on the output”, and can decide not to use it, override it, or stop it. A store's chat assistant is not a high-risk system under the Act. But when legislators name automation bias explicitly, it is a fair sign that “a human approves it” is not, by itself, a safety argument.

Gate by consequence, not by default

If asking about everything doesn't work, and asking about nothing is reckless, the question becomes which actions deserve a yes. We sort them by two properties, and only two.

  • How hard is it to undo? A price change can be reversed in seconds. An email to a customer can't be unsent. A refund, once it reaches the card, is gone.
  • Does it reach a customer or money? Changing an internal note reaches nobody. Publishing an FAQ answer reaches every shopper who asks. Issuing a discount reaches your margin.

Figure 1 · Where the gate goes

The harder an action is to undo, and the further it reaches, the stronger the gate

Actions Lumnya's assistant can take, placed by how hard they are to undo and whether they reach a customer or money.

↑ Reaches customers or money

Asks first · can earn a standing yes

price changediscount codeFAQ answer

Never automatic, at any setting

refundreturn decisioncustomer emaillive-chat message

Low stakes

pause automation

Asks every time

stock count overwrite

Harder to undo →

Lumnya's action classes as implemented, verified in the source code on 19 October 2026. Positions are qualitative.

The top-right corner is the important one. Some actions are both irreversible and customer-facing, and for those a standing approval would be exactly the reflex the research warns about. So in Lumnya, refunds, return decisions, emails to customers and live-chat messages can never run without your yes, at any setting, and so can any change to the autonomy settings themselves. That rule lives on the server, not in the model's instructions: there is no configuration, prompt or approval history that turns it off.

Everything else asks first by default. The difference is what happens over time. For reversible actions, it is reasonable to let a track record count. We have built that step and keep it off by default: after you have confirmed the same kind of action three times, Lumnya can offer a standing yes for that kind of action, inside caps on the number of actions per day, the amount per action and the amount per month, with quiet hours, and you can pause it at any time. It never applies to the top-right corner.

Figure 2 · The rules, action by action

What asks, what can earn a standing yes, and what never runs alone

Lumnya's assistant actions against the three gate rules, as implemented.

Asks first by defaultCan earn a standing yesNever automatic
Refundyesnoyes
Return decisionyesnoyes
Email to a customeryesnoyes
Live-chat messageyesnoyes
Stock count overwriteyesnoyes
Price changeyesyesno
Discount codeyesyesno
FAQ answeryesyesno

yesnon/a

Lumnya's source code, verified 19 October 2026. Standing approvals are off by default.

A yes is only as good as the preview

The second half of the problem is what the question looks like. A vague approval request is worse than none, because it gives the appearance of oversight with none of the substance. You can't check “Update pricing?” against anything, so you approve it on trust, which is exactly the automation bias the research describes.

Figure 3 · Two ways to ask for the same yes

A falsifiable preview turns trust into a two-second comparison

Four approval requests, as a vague gate asks them and as a real gate shows them.

Preview

What a vague gate asks“Update pricing?”
nothing to check

“Update pricing?”

“Oxygen Pro, Medium: 520.00 → 572.00. Can be undone.”

What a vague gate asks“Handle this return?”
which order? how much?

“Handle this return?”

“Approve the return for #1047 and refund 89.00 to the original card. This can't be undone.”

What a vague gate asks“Send follow-up emails?”
to whom? saying what?

“Send follow-up emails?”

“Email 1 customer: “Your replacement ships Monday.” Full text below.”

What a vague gate asks“Apply a discount?”
what, for whom, until when?

“Apply a discount?”

“Create AUTUMN10: 10% off carts over 60, ends Sunday.”

Illustrative requests. The real-gate versions follow the shape of Lumnya's previews: the exact change, before and after.

The rule we hold ourselves to: a preview must let you catch a mistake without opening anything else. That means the exact before and after, the specific customer or order, the amount, and whether it can be undone. In Lumnya, a yes also applies only to the exact proposal you were shown: if anything about the action changes, it asks again, and an unanswered proposal expires after fifteen minutes instead of waiting to be approved later out of context.

Earning a standing yes

The idea behind standing approvals is the one you would use with a new hire. Nobody gives a new employee the company card on day one, and nobody makes a five-year veteran ask permission to send a routine email. Trust is granted per kind of task, on the strength of a record you have watched.

The difference with software is that the record can be exact. After three confirmations of the same kind of reversible action, Lumnya can offer to stop asking for that kind of action, and only that kind. The standing yes carries limits you can see: how many actions a day, how much per action, how much per month, and hours in which it never acts. Every action taken under it is journaled as such, so you can tell at a glance what you approved and what ran on the standing yes. Revoking it is a single instruction, and it is off unless you turn it on.

The receipt: what the log records, and what it doesn't yet

Approvals cover the moment of action. The log covers every moment after: when a price looks wrong next Tuesday, “why did it do that?” should have a concrete answer. Here is exactly what Lumnya records today, including the gaps.

What is recordedKeptToday
Every action an agent takes: which agent, what it did, to what, the result or the error, and whenWithout a time limit✓
How each change was made: immediately, after your confirmation, or on a standing approval30 days✓
What it takes to undo the change, so it can be reversed30 days✓
What happened afterwards, where it can be measured: orders using a discount code, changes in how often a trigger fires, changes in FAQ deflectionWith the action✓
Which person approved it, for stores where several people can approve—Not yet
The reasoning behind each action as its own field in the log—Not yet

The last two rows matter. An earlier version of this article said the log records who approved an action and its reasoning. It doesn't yet, and we have corrected the article rather than leave it standing. For a single owner the first gap is small, since the approver is you. For a team it is real, and we will say so until it is closed.

How to audit human-in-the-loop AI in any tool

You can test this in an afternoon, with any vendor, without seeing their code. Ask these and watch for mechanisms rather than adjectives.

  • “Which actions can never run without my approval, and where is that enforced?” “In the prompt” is a preference. “On the server, per action” is a rule.
  • “Show me a preview for a refund.” If it doesn't show the order, the amount and whether it can be undone, your yes will be a guess.
  • “What happens if I approve, and the action changes before it runs?” The right answer is that it asks again.
  • “How many approvals will I see in a normal week?” If the honest answer is hundreds, you will stop reading them, whatever the vendor intends.
  • “How do I undo the last thing it did?” If the answer involves support tickets, the gate is the only protection you have.
  • “What does the log record, and what doesn't it?” A vendor who can name the gaps in their own log is one whose log you can trust.

What we refuse to ship

  • Silent changes. If it changes your store, you saw the exact change first, or you chose a standing yes for that kind of change.
  • Default autonomy. Asking first is the factory setting, always.
  • Autonomy for irreversible, customer-facing actions. Refunds, return decisions, customer emails and live-chat messages ask every time.
  • Vague approvals. If a preview can't be checked in two seconds, it isn't a preview.
  • Trust theatre. No borrowed badges and no claims we can't back. Our security page lists what is actually true.

The industry will keep promising fully autonomous everything, because it demos well. We will keep building the gate, and keep it asking only where it matters, because stores run on it. How approvals work in practice is in our FAQ.

Two pieces that follow from this one: a check after the action is a log, with what developers actually wire around their own agents, and the rule for what an assistant may claim about stock and deadlines.

Frequently asked questions

What is confirm-gated AI?

An AI system that proposes an exact change and waits for a person's approval before acting, then records what it did. The design question is which actions need the approval: gating everything causes approval fatigue; gating nothing is reckless.

Is human-in-the-loop AI safe?

Only if the human is really checking. Research on automation (Parasuraman & Manzey, 2010) finds people monitor reliable systems less than they should and over-rely on their recommendations, and training doesn't remove the effect. Gates work best when they are reserved for consequential actions and show a precise, checkable preview.

Which actions should an AI never take on its own in a store?

Actions that are hard to undo and reach a customer or money. In Lumnya, refunds, return decisions, emails to customers and live-chat messages always wait for the owner's approval, at every setting, and that rule is enforced on the server.

Can Lumnya act without asking?

Not by default: every change to your store asks first with a preview. A standing approval for reversible kinds of action, within daily and per-action caps, exists and is off unless enabled. It never applies to refunds, return decisions, customer emails or live-chat messages.

Sources

Lumnya is a team of ecommerce AI agents for Shopify stores, now in private beta. Join the waitlist.

Comments

…

Every comment is read and approved by a human before it appears — the same confirm-gated rule the product runs on.

Loading comments…

Filed under Product

All posts

Keep reading

Product

A prompt is not a contract.

Lumnyaai

AI customer service that can't fake a refund: a prompt is not a contract

14 min read · Product

AI & Commerce

When the shopper is an agent

Lumnyaai

Agentic commerce explained: UCP, ACP and the three things a merchant should do

13 min read · AI & Commerce

Private beta

Enough reading. See it on your store.

Join the private beta and be first in line when Lumnya opens to your store.