The demo version of an AI agent is easy to recognise: it receives an instruction, calls a tool and returns an answer. The production version has a harder job. It must know when it is allowed to act, what evidence it can trust, how much it may spend and what to do when the world does not match its assumptions.

Start with authority, not autonomy

Before an agent receives tools, define its authority. Reading a document, drafting a message and sending that message are three different permissions. Combining them because they happen in one workflow turns a recoverable mistake into an external incident.

A production agent should have a clear boundary around the systems, records and people it may affect. Actions that create financial, legal or reputational consequences need an approval step that is visible to the operator.

  • Separate read, draft and execute permissions
  • Require approval for irreversible actions
  • Record who authorised each sensitive action

Make evidence inspectable

An answer is not reliable because it sounds complete. The system needs to preserve the source, timestamp and assumptions behind decisions. When retrieval returns conflicting material, the agent should expose the conflict instead of silently choosing the most convenient passage.

This evidence layer also changes how teams debug. They can inspect what the model saw, what tool returned a value and which policy allowed the next step instead of trying to reproduce a conversation from memory.

  • Store source references with outputs
  • Distinguish observed facts from inference
  • Keep tool results available for audit

Design failure as a normal state

Networks time out, APIs change and records disappear. A resilient agent treats these as expected operating states. Each workflow needs a retry limit, an idempotency strategy and a point where the agent stops and asks for help.

The practical goal is not an agent that never fails. It is a system that fails without duplicating work, losing context or hiding what happened.

  • Use idempotency for external mutations
  • Set cost, time and retry budgets
  • Provide a human-readable recovery path

Evaluate the workflow, not the conversation

Teams often judge an agent by reading a few responses. Production evaluation needs task-level measures: Did it choose the right tool? Did it respect permissions? Did it preserve the required facts? Did the final action produce the intended state?

A small set of real failure cases is more valuable than a large synthetic benchmark that does not resemble the work. Start with the situations that would cost the business trust, time or money, and keep them in the release gate.

  • Test real failure cases
  • Measure task completion and policy compliance
  • Review regressions before changing models or prompts