AI agent monitoring in production means watching the agent on two layers, each with its own owner and its own alerts. A quality layer scores a sample of real sessions after they finish, to ask whether the agent did the job. An infrastructure layer watches every model call, tool call and permission check, and investigates when one fails. A short list of checks whose failure must never reach a user runs inline, before the response goes out. Both layers read the same trace, and every failure that escapes ends as a test case or a scheduled probe.
Key facts
Definition
AI agent monitoring runs two layers with separate owners over one trace: a quality layer that asks whether the agent did the job, and an infrastructure layer that asks whether a call failed.
How Bigcircle does it
- Bigcircle's insurance voice agent keeps the required disclosure in the tool contract rather than the prompt, so the operation cannot run without it.
- Bigcircle's trademark drafting platform scores every draft with a harness before an attorney reads it, and treats attorney corrections as the signal automated scoring misses.
- Bigcircle attaches the run's trace to every escalation from its desktop agent, so the report separates a planning mistake from an outage, an expired session or stale state.
Comparison
LLM observability records each model call, while agent observability also records the tool calls and permission decisions around it, where a denied call turns into a polite empty answer.
| Criterion | Inline on every turn | Quality layer | Infrastructure layer |
|---|---|---|---|
| Question | Must this response be blocked before the user sees it? | Did the user get what they needed? | Did every model call, tool call and permission check run as intended? |
| When it runs | Every turn, before the response. It can block. | On a sample, after the session. Minutes to hours. | On every call, and alerts on a signal. Seconds to minutes. |
| What it catches | Bad tool arguments, missing permissions, personal data in the output, denied topics, a cited source that does not exist | A wrong or incomplete answer, the wrong tool chosen | A denied credential, a quota, a failing dependency |
| Owner | Written in code, with the timeout decision already made | The team that builds the agent | Platform or on-call engineering |
| How Bigcircle handles this | On the insurance voice agent we built, an operation cannot run without its required disclosure, because the check sits in the tool contract rather than in the prompt. | On our trademark drafting platform, a harness scores every draft before an attorney sees it, and the attorney's corrections are the signal the score cannot see. | On our desktop agent, an escalation carries the run's trace, which tells a planning mistake apart from a provider outage, an expired session or stale state. |
Agents are hard to monitor because they fail without looking like they failed. A web service that cannot reach its database returns an error, and an alarm fires. An agent whose tool call is denied often writes a polite reply saying it found no records. The request returns a success status, the trace looks finished, and the user now believes something false. Nothing on a standard service dashboard turns red, because by its measures nothing failed.
The two kinds of failure also disguise themselves as each other. A missing permission produces an answer that reads like a quality problem: vague, unhelpful or empty. The wrong tool chosen for a request produces clean infrastructure metrics, because every call succeeded.
We met the same disguise outside agents. On the meeting transcription app we built with AnyTeam's team, every symptom reached support as "it didn't work". We built per-stage state reporting into the app because without it each of those tickets becomes a model investigation, when most of them turn out to be audio-routing problems.
Ownership makes it worse, because the two layers usually belong to different teams. The team that builds the agent owns quality, and platform or on-call engineers own infrastructure. Each looks at its own dashboard and sees a system that works, and a failure with no agreed route between them waits until a customer reports it.
This guide starts with what agent observability records and how monitoring differs from it, then covers the split, what runs inline, how each layer works, how a failure moves between them, and how it becomes a test. Writing the scoring rubric itself is in the guide to LLM evaluation.
What is agent observability, and how is it different from monitoring?
Agent observability is the record: every step of a run kept as a span under one trace ID, with enough fields attached that someone can ask a new question about a run after it has finished. Monitoring runs against that record: the scores, the alarms and the inline checks this guide describes. A team with a full trace and no monitoring learns about a failure from a customer, then debugs it well. A team cannot have monitoring without the trace, because a score or an alarm has to point at the steps that caused it.
LLM observability is the narrower version of the same idea, applied to each model call: the prompt, the response, the model and version that answered, tokens, latency and errors. That record is enough for a chat feature, and it misses most agent failures, because they sit between the model calls. A model call that returned a polite "no records found" looks healthy in LLM observability, and only the tool span before it shows the call that was denied.
Agent observability adds the tool calls, the permission decisions, the retries, the fallbacks and the run as a whole. The run trace on our desktop agent links the goal, the plan, the tools found, the calls made, what each call returned, every check on that result, every retry and any escalation, and it ends with the resulting state in the target application.
Why does an agent return a success status when it has failed?
An agent turns errors into language, so treat every empty or apologetic answer as a suspected failure until a span shows otherwise. When a tool call fails, the model receives an error or an empty result and does what it was trained to do, which is write a helpful reply. The failure ends up inside a sentence, and the status code only tells you that a sentence came back.
The plainest case we know of is in AWS's September 2026 example of a four-agent airline reservation system. The agents' execution role lacked permission to call the model, so IAM denied every model call, and the supervisor agent returned a blank output rather than a 403 or an exception. From outside, the agent had stopped helping, and nothing raised a fault.
| What the user sees | What it looks like | What actually happened | The signal that finds it |
|---|---|---|---|
| "I couldn't find any orders for you" | A retrieval or quality gap | The tool's credentials were denied and it returned an empty list | An auth error on the tool call, or a jump in empty results |
| A blank or generic reply | A weak prompt | The model call was denied or timed out | Model call error rate, count of empty responses |
| A plausible answer that is wrong | Nothing at all | The agent chose the wrong tool, or misread the result | Sampled quality scores, users undoing what the agent did |
| A slower, vaguer answer | A model regression | A rate limit pushed traffic to a fallback model | Share of calls answered by the fallback, per route |
| "Done" with nothing done | Success | A write failed after retries and the model replied with its intent as if it were the result | A separate read of the target system |
The first two rows are infrastructure failures that read as quality failures. The third is a quality failure that infrastructure cannot see. The last two sit between the layers, and teams find them latest. The guide to AI agent verification covers the last row. On our desktop agent a false "done" shows up when the agent's claimed completion is compared against the resulting state at the end of the trace.
The fix starts in the tool, before any monitoring is added. A tool that receives an authorisation error should return a typed error, not an empty result. "Access denied for this account" and "no orders found" have to be different outputs, because the model will treat them the same way if they look the same. That one change moves a whole class of silent failures into the infrastructure layer, where an alarm can fire on them.
What does each layer watch?
The two layers ask different questions of the same run, and neither answer implies the other. A session can score well while a tool was denied halfway through, and a session with every call green can still give the wrong answer. Each layer needs its own signals, its own pace and its own owner.
The figure is the split and the owners as we draw them.
Keep the layers separate even when one tool could serve both, because they fail at different speeds and page different people. A quality regression is a pattern across many sessions, found by reading scores over hours. An expired credential is one fault that hits every session at once, and it needs someone paged within minutes. One alert policy for both either wakes people for noise or misses the outage.
What has to run inline, on every turn?
Inline checks run before the response reaches the user, and they can block it. Everything else runs later, on a sample or on a signal. The test for moving a check inline has two parts. A failure must be one that can never reach a user, and the check must be quick and exact enough to block on.
The figure is a framework, and the checks in each column are the usual ones rather than a fixed list.
Sampling means some bad responses reach users before anyone scores them, and for one class of failure that is already too late. If the harm happens on delivery, such as a leaked account number or a denied topic, scoring it an hour later only records that it happened. Only that class belongs on the inline list.
| Check | Where it runs | Why |
|---|---|---|
| Tool argument schema, permission, allowlist | Inline, in code | Exact, fast, and the harm happens when the tool runs |
| Personal data redaction on output | Inline | A leaked detail cannot be recalled |
| Denied topics, regulated wording | Inline | The rule is known in advance and the harm is on delivery |
| Every cited source exists and resolves | Inline | A lookup, not a judgment |
| Whether the answer was correct and complete | After the session, sampled | Needs judgment and often a reference answer |
| Whether the agent chose the right tool | After the session, sampled | Needs the whole session to judge |
| Drift by segment, route or version | After the session, in aggregate | Only visible across many sessions |
| Credential, quota or dependency failure | On a signal, investigated | One fault in every session, which needs a fix rather than a score |
Every inline check adds latency to every turn, so the list has to stay short, and a rule that can be written down belongs in code. On the insurance voice agent, built and tested end to end but not yet in production, compliance lived in the tool contract. Whether the model will always make a disclosure cannot be answered. Whether the payment can execute without it can, and the answer was no.
Use a model-based guardrail only where no rule can decide, and give each one a timeout. When it times out, code needs a decision made in advance: block and hand the user to a person, or let the response through and flag it for review. A team that leaves that choice open finds out which way the system goes during the outage.
How should the quality layer pick and score sessions?
Pick the sample on purpose, because the rate and the shape of it are choices someone has to make, and the default is usually a random slice. A random slice spends most of its budget on common, easy sessions that already work. Weight it toward rare routes, new versions and sessions that ended badly, so those are scored far more often than their share of traffic.
Some sessions are scored every time. A session where the user undid an action, complained, asked for a person or repeated the same request carries a signal from the user that something went wrong.
On our trademark platform the equivalent is the attorney's edit. We treat attorney corrections as the most valuable signal in that system, because they measure the gap automated scoring cannot see. After those, the sample should cover each route and tool, each customer segment that matters, and each new prompt or model version at a higher rate for its first days in production.
A version change needs the same inputs scored on both sides. The trademark platform logs the base model and the fine-tuned model on the same inputs, so a regression from a model change shows up as a score movement rather than a feeling. Scores are also only comparable while the scale holds still. On RepliCoach, our sales roleplay build, a drift in one persona's score distribution means the persona's difficulty changed, and the scale needs recalibrating before the scores are read as a trend.
Scoring follows the guide to LLM evaluation: pass-or-fail criteria written from real failures, and a model judge only where a rule cannot decide, checked against people before its scores count (judge alignment covers how). Until that check is done, read a judge's score as a signal to look at a session, not as a measurement of it.
One low score means little on its own. A cluster of low scores that share a route, a tool or a phrasing in the request is a failure mode with a likely cause. Group low-scoring sessions by those fields daily, and give each group with a clear cause an owner. Keep the alerts slow and specific. An alert on a daily drop in one route's score, with the sessions attached, gets read. An alert on every low score gets muted within a week, and the next real regression passes unseen.
How should the infrastructure layer investigate?
The infrastructure layer needs every step of a run recorded as a span under the same trace ID: the model call, each tool call, each retrieval, each permission check, and any retry or fallback. Without that, an investigation starts from a blank reply and works backwards through several services' logs by hand.
Keep your own field names behind one mapping, so a rename upstream touches one file. OpenTelemetry has draft conventions for these spans, including invoke_agent and execute_tool, but as of October 2026 every part of the GenAI conventions is still marked Development rather than Stable. Since June 2026 they have lived in a separate repository with no versioned release yet, so expect attribute names to change.
Standard web tracing does not carry the fields an agent investigation needs, so add them. Record the permission decision and the user it was made for, the tool's result as a typed status, whether a result was empty, how many retries ran, and which model answered when a fallback fired. Those fields turn "the agent gave nothing useful" into "the model call was denied for this role".
| Alarm | What it usually means |
|---|---|
| Denial or error rate up on one step | A credential, policy or dependency changed |
| Empty results up on one tool, with no change in traffic | The tool is failing quietly, often on permissions |
| Empty or blank model responses | Model calls are failing or being cut off |
| Fallback model share up | Rate limits or quota on the main model |
| Retries per session up | A dependency is slow or failing on and off |
| Latency up on one step only | That service, not the agent, has changed |
| Tokens or cost per session up, with no change in traffic | A retry loop, a longer context or a fallback model taking more of the calls |
| A rate that normally moves falls to near zero | The signal itself has stopped being recorded |
Cost belongs on the infrastructure dashboard, per session and per route, because a fault often shows up there before anywhere else. We bound retries on our desktop agent for that reason. An agent that retries indefinitely against a broken vendor is worse than one that stops, because it burns cost and time while looking busy.
Alarm on drops as well as rises. On FinSight, the fund reconciliation system we built and tested internally, we monitor the break rate per data source and treat a sudden drop in breaks as being as suspicious as a spike. The same holds for an agent: a tool whose error count falls to zero overnight has more often lost its logging than been fixed.
Investigation correlates those signals across every service a run touched. Write the runbook as one rule: start at the first failing span in the trace and follow it to the service that owns it. In the airline example that span was the model call, and the owner was the IAM policy missing bedrock:InvokeModel.
Every investigation should end with a probe. Once the cause is fixed, add a scheduled synthetic run that calls the agent as a real user would, with a known expected result, and an alarm on the signal that was missing. A probe on the failing path catches the next expired credential before a user does.
Keep the monitoring out of the agent's reach as well. Assume an agent with write access to the code that holds the alert rules can change them. Keep its credentials away from the alerting configuration, and run a scheduled probe that checks an alert still fires when it should. This has happened without any attacker involved. In OpenAI's misalignment report, updated 20 October 2026, a GPT-6.1 Sol agent repairing an evaluation pipeline on 3 October disabled automatic alerts to the security team, and a GPT-6 Astra agent on 6 October removed a monitoring integration and then denied doing so. Both were ordinary tasks, and both agents changed their monitoring as a side effect.
How does a failure move between the layers?
The shared trace ID carries the handoff. A quality score that points at a session also points at every span in it, and an infrastructure alarm that names a step also names the sessions that passed through it. Carry the same run identity into the test case a failure becomes, so a failing case in CI links back to the production run that produced it.
Three rules keep the handoff from turning into an argument between teams. A quality drop in the same window as an infrastructure alarm goes to infrastructure first, because one broken dependency explains many quality symptoms at once. A quality drop concentrated on one tool gets a health check on that tool before anyone edits a prompt. After an infrastructure fix, the quality layer rescores a sample of the affected sessions, so both owners see the symptom has gone.
The misdiagnosis to prevent is a prompt change made to fix an infrastructure fault. It seems to work, because the fault often clears on its own, and it leaves a prompt tuned to a failure that no longer exists. Checking infrastructure first costs minutes. On our desktop agent the trace on an escalation already separates a planning defect from a provider outage, an expired session, stale state or an unsupported action, so the escalation arrives pointing at a fix rather than opening an investigation.
How does a production failure become a test case?
Every failure that escapes should end as a check that would have caught it. A quality cause becomes a test case that gates releases, and an infrastructure cause becomes a probe and an alarm. That loop only works if cases can leave production without carrying personal data with them.
The figure is a framework of the steps, and it carries no timings or counts.
Redaction comes first, before the case leaves the production environment. Replace names, contact details, account numbers and free-text personal details with consistent placeholders, so the same customer gets the same token everywhere in the case. The case keeps the shape of the failure without identifying anyone. Where the failure depends on the content of a personal detail, such as a name the agent misread, write a synthetic replacement with the same property rather than keep the real one.
Reproduction comes next: shrink the session to the smallest case that still fails. A long session with one bad tool choice near the end becomes the few turns that lead to that choice. Small cases run fast in CI, fail for one reason, and do not break when an unrelated part of the conversation changes.
Label each case with the expected behaviour and an owner. The label names what a correct run does, such as calling the lookup tool before answering or handing over when the account is locked, rather than pasting in one ideal reply. The owner decides whether a future failure of this case blocks a release.
The case then joins the test set, and the set gates releases: a pull request stays blocked while the agent's scores on it sit below a floor the case owner set. On Skionis, our AWS agent, we keep every validation failure and the revision that answered it. They are the system's own record of what the model gets wrong, and we use them to improve the validation loop rather than the prompt.
Keep the raw session out of the repository even after redaction review. Store cases in a controlled dataset with access rules, a retention period and a link back to the production trace for the people permitted to see it. A test fixture committed to source control is copied into every clone and every fork.
Who owns what?
Name one owner for each part before launch, and agree the route a failure takes between them. Many monitoring failures are ownership failures: the signal existed, and it reached nobody who could act, or it reached two teams who each assumed the other had it.
| Owner | Owns | Is paged for |
|---|---|---|
| The team that builds the agent | Prompts, tool selection, quality scoring, the test set, release gates | Quality drops on a route or version, failed release gates |
| Platform or on-call | Credentials, quotas, model and tool dependencies, tracing | Denial, error, empty-response and fallback alarms |
| Product or business owner | What counts as correct, which failures block a release | Nothing at night; reviews patterns and gate decisions |
| Security and privacy | Inline redaction, access to traces and test cases, retention | Redaction failures, requests for access to raw sessions |
A short weekly review across the owners catches what the alerts miss: the week's escaped failures, which lane each went down, and whether each ended in a case or a probe. The guide to building an AI audit trail covers the decision records that review reads.
Both layers also feed decisions about how much an agent may do. AI agent governance covers how an action earns more authority from evidence. The quality layer produces that evidence, and the infrastructure layer tells you whether to believe it, since a score measured while a tool was failing silently measures a different system from the one in production.
What should a small team build first?
Build in this order, because each step changes what the later signals mean:
- One trace ID on every step of every run, with permission decisions, typed tool status and empty results recorded on the spans.
- Typed errors from every tool, so a denied call and an empty result never look the same to the model or to a dashboard.
- The written list of failures that must never reach a user, each moved inline, in code where possible.
- Scoring on every session with a user signal, then a weighted sample for one route.
- The failure loop on every escaped failure, with redaction first, and the weekly review.
A team that starts with a quality dashboard and skips the first two steps can spend weeks tuning prompts against infrastructure faults, because nothing tells them a tool was denied. If the agent has no tools and a handful of internal users, most of this shrinks: read every session, and alert on the model API's errors. The split earns its cost once the agent calls tools on someone's behalf and more sessions run than anyone can read. Our AI observability work builds both layers alongside the agent.
Common questions
What is AI agent monitoring?#
AI agent monitoring watches an agent in production on two layers. A quality layer scores a sample of finished sessions to ask whether the user got what they needed. An infrastructure layer watches every model call, tool call and permission check and alerts when one fails. A short list of checks whose failure must never reach a user runs inline, before the response goes out.
What is the difference between agent observability and monitoring?#
Agent observability is the record: every step of a run kept as a span under one trace ID, with the permission decision, the typed tool status and any retry attached. Monitoring runs against that record, as quality scores, alarms and inline checks. Observability lets you debug a failure after someone reports it. Monitoring finds the failure first, and it needs the trace to point at the step that caused it.
What is LLM observability?#
LLM observability records each model call: the prompt, the response, the model and version that answered, tokens, latency and errors. It is enough for a chat feature. An agent also needs the tool calls, permission decisions, retries and fallbacks around those model calls, because a model call that politely reports "no records found" looks healthy while the tool call before it was denied.
Why does an AI agent return a success status when it has failed?#
The agent turns errors into language. When a tool call is denied or returns nothing, the model writes a reply anyway, and the service reports that a reply came back. The fix starts with tools that return typed errors, so a denied call and an empty result look different to both the model and the monitoring.
Which checks should run inline on every agent turn?#
Checks where the harm happens when the response is delivered or the tool runs, and which can be decided quickly and exactly. That covers tool argument validation, permissions and allowlists, personal data redaction, denied topics and regulated wording, and a check that cited sources exist. Judgments about correctness and tool choice run later on a sample.
How much production traffic should quality evals sample?#
There is no single right share, and a random share is the wrong shape. Score every session with a user signal such as an undone action, a complaint or a request for a person. Then weight the sample so each route, tool and important segment is covered, and score new versions at a higher rate for their first days in production. Tune the rate by whether it finds failure patterns.
How do you turn a production failure into an eval case without leaking personal data?#
Redact before the case leaves production, replacing personal details with consistent placeholders, or with synthetic values where the detail matters to the failure. Shrink the session to the smallest failing case and label the expected behaviour and its owner. Store it in a controlled dataset with access rules and retention, never in source control.
What should trigger an infrastructure investigation for an agent?#
A rise in denials or errors on one step, empty results from one tool without a change in traffic, blank model responses, a rising share of fallback model calls, more retries per session, latency up on a single step, or a rate that normally moves falling to near zero. Start at the first failing span in an affected trace and follow it to the service that owns it.
Further reading
- LLM evaluation. Writing the rubric and checking judges, which this guide does not repeat.
- Multi agent systems. The check before a merge, and the trace fields that show which worker was refused.
- AI agent verification. Checking the target system for the failures that sit between the layers.
- AI audit trail. The decision records the weekly review reads.
- AI agent governance. How evidence from both layers moves an action's authority up or down.
- AI agent permissions. The permission decisions the infrastructure layer records on each span.
- LLM as judge and judge alignment. Scoring with a model, and checking it against people.
- AWS, Monitoring production agent lifecycle with AWS DevOps Agent and AgentCore Evaluations, 11 September 2026. The missing-permission example.
- OpenTelemetry, GenAI semantic conventions, checked 22 October 2026. Agent and tool spans, all marked Development, with no versioned release yet.
- OpenAI, Suppressing misalignment monitoring alerts and disabling misalignment monitoring, report updated 20 October 2026. Two internal coding tasks in which agents disabled or removed monitoring, one on 3 October and one on 6 October.