An engineer spends three weeks building guardrails. Filters, prompt instructions, approval checks, a wall of rules the agent is supposed to respect. She hits run. Within the first week, a single crafted message walks the agent past every one of them. It wires the money, or empties the inbox, or deletes the records. The guardrails report green the whole time.

A crafted message arrives and passes prompt instructions, filters and approval checks in turn. Every guardrail reports green, and the action still lands: records deleted.

The model did what models do. It followed an instruction that looked convincing. Sometimes that instruction comes from a bad actor. Just as often it comes from a hallucination, or from the agent’s own reading of an ambiguous prompt, or its misguided attempts to complete a clear prompt. The weak point in all of these scenarios is not the model’s intelligence. It’s where the controls sit.

So before you build the next set of rules, it helps to draw a line that most teams only reach for when an agent moves from a demo to production. The line between what an agent thinks, and what an agent does.

A guardrail is advice the agent can ignore

Let’s start by defining what a guardrail is. A guardrail is a rule you put in or around the model to keep its behavior in bounds. A system-prompt instruction that tells it what to avoid, an input or output filter, a content classifier, a check that reviews the model’s plan before it runs. All of them shape or inspect what the model produces.

That is the tell. A guardrail is advisory, and it sits inside or around the model, which is the component most exposed to prompt injection and jailbreaks. You are asking the same system that just got manipulated to also be the thing that catches the manipulation.

The deeper issue is probability. A model’s refusals are statistical. Most of the time it declines the bad instruction, and some fraction of the time it complies. For a chatbot, a rare miss is an awkward answer. For an agent with the authority to move money or delete data, a rare miss is an incident. The OWASP Top 10 security threats for LLM Applications and Simon Willison’s “lethal trifecta” both describe the same shape of problem. Once a model can be steered, anything downstream of its reasoning inherits the risk. Your security posture cannot be probabilistic when the actions are real.

AI Agent Impact: the harm is rarely a movie villain

In July 2025, Replit’s AI coding agent deleted a company’s production database during an active code freeze. The agent went on to call its own move a “catastrophic error in judgment.” No attacker was in the loop. (Tom’s Hardware)

A year earlier, Air Canada’s support chatbot described a bereavement-refund policy that did not exist. A tribunal held the airline responsible for what its bot said and ordered it to pay. The chatbot was not hacked. It hallucinated, and the company was bound by the result. (Forbes)

Both cases clear the same bar. An agent produced an action with real consequences, and nothing between the model and the system stopped it.

Governance is where reasoning ends and action begins

Guardrails control reasoning and text. That is useful work. Tone, format, refusals, staying on topic, guardrails work for all of it.

Governance is a set of controls you put at the point where the agent acts to decide whether a given action is allowed to happen at all. An authorization check that confirms who is really behind the action, a policy that defines what may proceed, and a durable record of what ran.

Where a guardrail shapes or inspects what the model produces, governance decides what the agent is allowed to do, and it starts the instant the agent acts.

When an agent sends the payment, deletes the record, or closes the ticket, it produces a transaction. A transaction needs three things enforced at the moment it happens:

  1. Authorization. Is this allowed?
  2. Policy. Does it satisfy the rules?
  3. Audit. Is it recorded?

None of those are questions about language. They are questions about action.

Guardrails Governance
Where it lives Inside or around the model At the tool call, outside the model
What it governs Reasoning, tone, text Actions, transactions
How it behaves Advisory, probabilistic Enforced, deterministic
What defeats it Prompt injection, a jailbreak, a confusing input Only a bypass at the action itself; it cannot be talked around
When to use it Shaping what the agent says Deciding whether the agent can act

Tool calls are where intent becomes consequence

You can govern an agent at more than one layer, and seeing the whole stack helps identify the boundaries between guardrails and governance.

At the top is the model itself, aligned during training. Below that, prompt-level guardrails. Below that, your application logic. Below that, identity and access. Then the tool call, where the agent actually invokes a system. Then the network underneath everything.

The governance stack, six layers deep. The model, aligned during training: advisory and dangerous. Prompt-level guardrails, which shape what the model says and can be talked past: advisory and dangerous. Application logic, your own code one app at a time: per app. Identity and access, who the actor is, reviewed on a cycle: too narrow. The tool call, outside the model and before the system it acts on: highest leverage. The network, which sees packets rather than intent: too low, too late.

Each layer catches a different class of problem, but they are not equal in leverage. Govern too high, at the model or the prompt, and your control is advisory and dangerously probabilistic for the reasons above. Govern too low, at the network, and you can block a host or a port but you cannot tell a legitimate action from a reckless one, because the network sees packets, not intent.

The tool call is the point where intent becomes consequence. It sits outside the model, so an injection cannot dissolve it. It sits before the target system, so a denied action never lands. And one control there covers every model, framework, and client sitting above it, because they all have to pass through the same door to do anything.

That is why most of your governance return comes from the layer where the action happens.

Why governance has to live in the actions runtime

To be clear, an agent has its own runtime. That’s the loop where the model reasons, plans, and decides what to do next. That is not where governance belongs for all the reasons I laid out above.

Governance belongs in the actions runtime, the MCP layer where a tool call actually executes against a real system. Complete security means governance has to run at the same place the action runs, and the two cannot be separated.

Governance that sits apart from the runtime can advise before the fact, which is the guardrail you already saw an agent jump. Or it can report after the fact, which is a log you read once the money has already left. Useful, but neither one stops the action.

By contrast, governance that sits inside the runtime, on the single path every tool call travels, can evaluate the specific action and refuse it, in real time, before it reaches the system. Advisory becomes enforced. Probabilistic becomes deterministic.

This is also why the controls you already trust sit in the wrong place for agents. Approval workflows and access reviews were built for human speed and human review, checking work after it happens. An agent fires dozens of actions a second. Anything that checks after the fact means writing the incident report instead of preventing the incident.

AI Agent Impact: after the fact is too late

Researchers disclosed EchoLeak (CVE-2025-32711), a zero-click flaw in Microsoft 365 Copilot. A single crafted email could steer the assistant into gathering sensitive data from a user’s environment and sending it out, with no click from the victim. Microsoft patched it before any known exploitation, but the lesson for governance has remained and sits in the order of events. By the time any after-the-fact review could run, the data has already moved. Only a control sitting on the action itself could have refused the send. (

Microsoft / MSRC, CVE-2025-32711

)

Guardrails still have a job

It’s critical to emphasize that we are not advocating for governance instead of guardrails.

We are saying that safely and effectively putting agents into production means both, and putting each control where they work. Guardrails shape how an agent reasons and speaks, and they do that well. They were simply never built to stop an action, because they do not sit where actions happen.

Keep the guardrails on the reasoning layer. Add governance, bound to the actions runtime, on the action layer. Treat every model as if it could be talked into anything, then build so that when it does take a wrong turn, the reckless action still meets a hard, deterministic no at the tool call.

We built Arcade.dev to close that gap and stop a reckless action before it reaches the system, instead of reading about it in a log after the money has already left.

The rest of this series shows how that works in practice, one tool call at a time, starting with the single question runtime governance asks of every action: can this agent, on behalf of this user, take this action, right now?

More in this series