An AI agent is not a model with a better prompt, it is a loop in which the model observes state, decides what to do, acts through tools, reads the result, and repeats until it reaches a stopping condition. That loop is powerful because it can adapt when the path is unknown.
It is dangerous for exactly the same reason, so the practical question is not "can we make it autonomous?" but "which decisions should the model make, which decisions should code make, and where do humans approve risk?" Useful agents are controlled loops around tool calling, memory, plans, permissions, evals, and observability.
What makes something an agent
Many systems marketed as agents are really workflows, and a workflow calls a model inside a path your code defines: classify, retrieve, draft, score, send. An agent lets the model choose the next action based on observations. The distinction matters because agents cost more, fail in longer trajectories, and need stronger controls. A real agent has four parts: a model that chooses actions, tools that expose capabilities through tool calling. It has a loop that feeds observations back into the model. It has a stopping rule: success, failure, budget, timeout, or human handoff.
Use a workflow when the steps are known, and use an agent when the task path has to be discovered. Invoice extraction, support triage, and document classification are usually workflows. Debugging a failing integration, investigating a metrics anomaly, or reconciling a messy data room can be agent-shaped.
The loop: observe, reason, act, verify
The minimal production loop is observe, reason, act and verify. Observation gives the agent ground truth from files, traces, API responses, browser state and database rows. Reasoning then selects the next move, action changes state through a tool, and verification checks whether the action helped.
Verification is the part demos skip, and the part production cannot. A coding agent can run tests, a data agent can compare row counts, a browser agent can inspect the page, and a support agent can check whether the answer cites permitted evidence. Without verification an agent is just a model taking repeated guesses.
Tool calling is the contract
Tools are the boundary between language and consequences, which is why they carry the design weight. A tool schema says what the model may request, and the application decides whether the request is valid, authorized and safe to execute. That is why tool calling should be treated as an API contract rather than as prompt decoration, and why good tools are narrow, typed and boring.
refund_payment should require an account id, amount, reason code, and idempotency key. It should enforce policy outside the model, and it should return structured results the agent can use. Bad tools are broad escape hatches: run_sql, send_email, execute_command, or use_browser without scopes, dry-run modes, or approval gates.
MCP matters because it standardizes how tools are exposed to agents. Standardization helps integration, but it does not decide permissions for you. A dangerous tool remains dangerous when served through a standard protocol.
ReAct, CodeAct, and plan-and-execute
Agent patterns are ways to manage the loop.
| Pattern | Best fit | Main risk |
|---|---|---|
| Workflow | Known sequence of steps | Too rigid for open investigations |
| ReAct | Search, inspect, act, repeat | Can drift without good observations |
| Plan and execute | Multi-step tasks with clear milestones | Plans go stale after new evidence |
| CodeAct | Data, code, and automation tasks | Code can cause broad side effects |
| Computer use | Legacy software with no API | Slow, brittle, and hard to verify |
The ReAct pattern is the default because it keeps reasoning close to observation. Plan and execute helps when the task needs decomposition, but plans should be revised as evidence arrives. CodeAct is powerful when the agent can write small programs to inspect data or automate work. Computer use is the last resort for systems that only expose a graphical interface.
Memory and context engineering
Agent memory is not one thing, and the kinds do not mix well. There is short-term scratchpad state inside the current run, task memory such as files edited, tickets inspected and hypotheses rejected, and long-term user or organization memory. Each of the three has different privacy, freshness and retrieval requirements, and the common failure is to dump all of it into the context window. That raises cost, increases distraction and can bring back stale instructions, so better agents curate context instead, and they summarize observations, pin facts that must persist, retrieve only relevant memories, and distinguish data from instructions.
Memory should also be inspectable by the people who operate it. If an agent remembers a user preference, operators should know where it came from and how to delete it. If it stores a lesson from a failed run, the next run should see the evidence, not just a vague summary.
Permissions, prompt injection, and tool safety
Agents combine untrusted input with tools, which makes prompt injection and tool safety central. A malicious web page can tell a browser agent to reveal secrets, a support ticket can instruct an agent to ignore policy, and a document can contain hidden text that tries to change the task. Guardrails should be structural, which means the application enforces them whether or not the model cooperates.
- Give tools least privilege, and separate read tools from write tools.
- Require human approval for irreversible actions, and offer dry-run modes for the rest.
- Scope credentials to the task, and run risky tools in sandboxes.
- Never let retrieved or browsed text become authority over system policy.
The model can help classify risk, but the application must enforce risk. If a tool would delete data, move money, send external email, or change production configuration, the permission boundary should sit outside the model.
Multi-agent systems
Multi-agent orchestration helps when tasks truly benefit from separation: one agent researches, another writes code, another reviews, another tests. It can also add noise, because more agents mean more context handoffs, more duplicated reasoning, and more places for partial errors to become confident recommendations.
Use multiple agents when roles have different tools, permissions, or evaluation criteria. A security reviewer and a coding agent should not share the same authority. A researcher and a writer may benefit from separate context, but do not split a task just because it sounds sophisticated. A single well-instrumented loop is easier to debug.
Observability and evals
You cannot operate an agent you cannot replay, so observability means traces for prompts, model outputs, tool calls, tool results, retries, approvals, costs, latency, and verifier signals. It also means durable run state, so that long tasks can resume without losing context.
Agent evals should score trajectories rather than just final answers: did the agent choose safe tools, did it recover after a failed action? Did it ask for approval at the right time, and did it stop when evidence was insufficient? Those are all behavioral properties of the loop rather than of the answer.
Start with offline evals built from real tasks, capturing a successful human path and then letting the agent attempt the same goal in a sandbox. Score whether it reached the goal, how many steps it used, which tools it called, whether it violated policy, and whether its final output can be verified. The trace matters more than the final text because the trace tells you where the loop broke.
Then add online monitors, watching tool error rates, approval rates, timeout rates, cost per completed task, user intervention rate and rollback rate. An agent that completes 80 percent of tasks but needs expensive human cleanup on 20 percent may be worse than a workflow that completes 60 percent cleanly and escalates the rest.
Human handoffs and stopping rules
Good agents know when to stop, and stopping can mean success as easily as it means uncertainty, missing permission, missing evidence, tool failure, budget exhaustion, or risk escalation. A system that always tries one more step will eventually do something expensive or unsafe. Human handoff should preserve context, so the reviewer sees the goal, plan, actions taken, evidence found, tool outputs, current hypothesis, and proposed next action. A vague "agent needs help" handoff wastes the human's time and teaches the system nothing. A structured handoff turns the human into a verifier and gives the team data for the next eval.
Approval design should match the consequence of the action. Low-risk reversible changes can be auto-applied with logs, medium-risk changes can require review after a dry run, and high-risk actions require approval before execution. Some actions should never be exposed to the model at all. That is not a lack of ambition, it is how reliable automation survives contact with production systems.
Rollout strategy
The safest rollout path is shadow, assist, act. In shadow mode the agent observes real tasks and proposes actions without executing them, in assist mode users can accept or reject those proposals, and in act mode the agent executes within a narrow permission boundary and escalates anything outside it.
Each stage should have exit criteria of its own before the next one opens. Shadow mode should prove the agent can identify the right next action. Assist mode should prove users accept its proposals and corrections are declining. Act mode should start with reversible tasks, small accounts, or internal users. Expanding scope should require evidence, not optimism.
Versioning matters too, because prompts, tools, model versions, memory policies and guardrails all change behavior. Store every one of them with each run, because when a regression appears you need to know whether it came from a new model, a changed tool schema, a memory retrieval update, or a prompt edit. Do not roll out all tools at once.
Start with read-only tools, then reversible writes, then narrowly scoped irreversible actions after the agent has earned trust. Each new tool should come with a schema, examples, permission rules, dry-run behavior when possible, and eval cases for misuse. The tool surface is the agent's real product surface, and expanding it casually is how a helpful assistant becomes an unreliable operator.
Cost rollout needs the same discipline, because long agent runs can burn tokens through repeated observations, large tool outputs and verbose self-reflection. Set per-run budgets by task class, summarize large observations before placing them back into context, and keep raw artifacts available through tools instead of pasting everything into the prompt. An agent that is reliable but economically unbounded will still fail as a product.
Production takeaways
Build the workflow first, and promote to an AI agent only when the path cannot be scripted.
- Make tools narrow and typed, and put permission checks outside the model.
- Use guardrails that constrain actions, not just language.
- Treat agent memory as data with provenance.
- Trace every step through observability.
- Use multi-agent orchestration only when roles and permissions justify it.
An agent is a budget of autonomy, so spend it where adaptation is worth the cost and make every loop prove what it did.