An AI agent is a model given the ability to act: to call tools, change records, send messages, move money. That shift, from producing text to taking action, is what makes agents useful and what makes them dangerous. A chatbot that is wrong produces a bad answer. An agent that is wrong produces a bad outcome.
The distance between an agent that demonstrates well and an agent that can run unattended is large, and it is consistently underestimated. A demo only has to succeed once, in front of an audience, on a path the builder chose. Production means succeeding on inputs no one anticipated, every time, with no one watching the moment it acts.
Reliability is the whole problem
A model that is correct ninety per cent of the time is impressive in a demo and a liability in production. When the agent only suggests, a human catches the other ten per cent. When the agent acts, that ten per cent becomes wrong actions taken at machine speed (refunds issued, records overwritten, messages sent) before anyone notices.
The work of putting an agent into production is mostly the work of containing that failure rate: deciding what the agent is allowed to do, what it must check before doing it, and how a mistake is caught and undone.
Bound what the agent can touch
The first control is scope. An agent should hold the narrowest set of capabilities its job requires, and no more. An agent that drafts replies has no business deleting accounts. Every tool and permission it carries is one more thing that can go wrong, so grant only what the task demands.
Where the agent does act, design the action so you can undo it: drafts rather than sends, soft deletes rather than hard ones, staged changes rather than live writes. And put a hard checkpoint in front of anything that cannot be taken back. A payment, a message to a customer, a destructive change — let these wait on a human confirmation rather than the agent’s own judgement.
Make the agent’s behaviour observable
You cannot trust what you cannot see. An agent making opaque decisions is impossible to debug, and impossible to improve.
So log what the agent does. Capture every meaningful action it takes, and enough of the reasoning and inputs behind each one to trace a wrong outcome rather than guess at it. Be deliberate about sensitive data, though. Redact or leave out customer details and secrets by default, and keep full input capture to a controlled debugging path rather than the standard log. This matters more for agents than for ordinary software. The same input will not always produce the same output, so the record of what actually happened is often the only way to understand a failure.
Test behaviour, not just code
You test conventional software by asserting that a given input produces an exact output. Agents resist this, because their outputs vary. The answer is an evaluation harness: a set of representative scenarios you run the agent against repeatedly, scoring whether its behaviour stays within acceptable bounds rather than whether it matches a fixed string.
Without that harness, there is no way to know whether a change to the prompt, the model, or the tools made the agent better or quietly worse. The team is left changing things and hoping — which is not a position from which anything should be trusted to act unattended.
Plan for the failure you cannot prevent
Even a well-built agent will encounter inputs it handles badly. Production readiness includes the fallback: when the agent is uncertain or out of its depth, it should escalate to a human rather than guess. An agent that knows the boundary of its competence and stops at it is far safer than one that is confident everywhere.
Most of this is unglamorous, and most of it is the actual work. The model is the part that arrives in an afternoon; the guardrails are the part that takes the project. Skipping them is the most common reason agent initiatives stall on the way to production, the broader pattern in why AI projects fail before they reach production.
Choosing which model sits inside the agent is its own decision, covered in choosing the right AI model for the job. Building agents that can be trusted to act is the work our AI transformation practice exists to do.
