AI Agent Observability: How to Debug an Agent in Production

AI agent observability is the practice of recording every agent run as a single, replayable trace: the input, each model call and tool call in order, the arguments and results, tokens, cost, and how the run ended. Agents fail silently far more often than they crash, so a 200 response tells you nothing. This guide covers what to instrument, which four numbers to watch, and how to wire it up.

TL;DR

• The unit of observability for an agent is the run, not the request. One trace id ties every model call and tool call in that run together.
• Log tool arguments and step index. They are the two fields that reveal an agent looping on the same failing call, and they are the two most often skipped.
• Agents mostly fail without throwing. Watch task completion rate, step count p95, per tool error rate, and cost per run p95.
• Use OpenTelemetry spans so agent traces sit next to your HTTP and database spans instead of in a separate tool nobody opens.
• Pipe flagged production runs straight into your eval set. Real failures make better test cases than anything you invent.

Why agents break differently than normal software

A normal service fails loudly. A request throws, the status code changes, the error rate graph spikes, someone gets paged. You already have tooling for that shape of problem.

An agent fails quietly. The model picks the wrong tool and the tool returns a valid empty result. A search call gets called four times with slightly different queries because the first result was unhelpful. A retrieval step pulls the wrong document and the model writes a confident, well-formatted, wrong answer. In every one of those cases your service returned 200, your latency looked normal, and your error rate did not move.

That gap is the whole reason agent observability exists as a separate discipline. You are not watching for exceptions. You are watching for runs that technically succeeded and were still wrong.

The unit is the run, not the request

One user message can produce a dozen model calls and tool calls. If you log those as separate, unrelated lines you have made the hardest part of debugging impossible: you cannot see the sequence. The fix is to treat the run as a tree. One root span for the whole run, one child span per step, all carrying the same trace id.

Read left to right and the failure explains itself. The write step errored, the agent retried, then spent four seconds writing a final answer that was probably shaped by the failed call. Without the tree, you have six unrelated log lines and a user telling you the output was wrong.

What to log at every step

Two levels, and both matter.

At the run level, one record per user request: trace id, user or tenant id, the input, the final output, total input and output tokens, total cost, wall clock duration, step count, and a terminal status. Make that status an explicit enum, not a boolean. completed, failed, refused, hit_step_limit, and timed_out are five different problems that a success: false field flattens into one.

At the step level, one record per model call and per tool call: trace id, step index, step type, model id and version, tool name, the full arguments the model passed, the raw result or error, tokens and duration for that call, and a retry flag.

Two fields carry most of the debugging value and get skipped constantly. Tool arguments, because an agent calling the right tool with the wrong arguments is the single most common failure I see, and you cannot spot it from the tool name alone. Step index, because an agent calling search_docs five times in a row is obvious when the steps are numbered and invisible when they are not. If you are still designing those tool interfaces, how to design tools for an AI agent covers the input side of the same problem.

Redact at write time. Run prompts and tool results through a scrubber before they hit storage, not before they hit a dashboard. Storage is where the leak happens.

The four numbers that tell you the agent is healthy

Error rate is close to useless here because the interesting failures never error. Watch these instead.

Task completion rate is the primary number. Define a valid end state for your agent, then measure the fraction of runs that reach it. Everything else is diagnostic.

Step count at p95 is your early warning. When the median run takes four steps and p95 jumps from nine to twenty-two, something upstream changed: a tool got slower, a document got moved, a prompt edit made the instructions ambiguous.

Error rate per tool has to stay per tool. Averaged across eight tools, one endpoint returning 500 on every call looks like a two percent blip. Split out, it is obviously the thing to fix.

Cost per run at p95 doubles as a confusion detector. A confused agent reads more, calls more, and retries more, so spend rises before quality visibly drops. If your bill moved and your traffic did not, the agent is working harder for the same result. How to cut your AI API bill goes deeper on the spend side.

Wire it up in an afternoon

You do not need a platform migration. Four steps.

1. Create a trace id at the entry point and thread it through every function in the run. If your language has context propagation, use it. If not, pass the id explicitly. This is the whole foundation, and it takes twenty minutes.
2. Wrap your model client and your tool dispatcher. Every model call and every tool call goes through one function each in most codebases. Add the span there once instead of adding logging at forty call sites.
3. Emit OpenTelemetry spans. Root span agent.run, child spans llm.call and tool.call, with attributes for model, tokens, cost, tool name, and retry. Store large payloads (prompts, tool results) separately and put a pointer on the span, because prompt bodies will blow past attribute size limits fast.
4. Add a feedback hook. One thumbs down button, or one support tag, writing the trace id to a table. That table is the most valuable thing you will build this quarter.

Point it at whatever backend you already run. Langfuse, Braintrust, Arize Phoenix, and LangSmith are purpose-built for this shape and will take OTel directly. Datadog, Honeycomb, and Grafana work fine too if that is where your team already looks. The tool matters much less than the trace id discipline. For a lightweight way to catch regressions right after a deploy, the canary method is the pattern I run, and my AI daily driver stack covers the rest of the toolchain around it.

Turn traces into your next eval set

This is the payoff, and most teams stop one step short of it.

Every flagged run is a free test case written by reality. Take the runs that hit the step limit, the runs users thumbed down, the runs where a tool errored twice, and the runs in the top decile of cost. Strip them to input plus expected outcome and add them to your eval suite. Do that weekly and your suite gets harder and more representative on its own, instead of testing the same eight cases you thought of on day one. How to test an AI agent covers how to score them once they are in there.

The loop is: observe production, harvest failures, add evals, change the prompt or the tools, verify against the suite, ship, observe again. An agent without that loop does not improve. It just accumulates workarounds. I break down builds like this one every week in the newsletter.

The bottom line

Agents fail while returning 200, so exception monitoring will not save you. Give every run one trace id, log every model call and tool call as a span with its arguments and step index, and watch completion rate, step count p95, per tool error rate, and cost per run p95. Then feed the runs that went badly back into your eval set. The whole thing is an afternoon of instrumentation and it is the difference between debugging an agent and guessing about one.