How TruLens Tracing Helps You Find Where an AI Agent Fails

TruLens traces every step of an AI agent—each span records latency, inputs, outputs, tokens, and cost—so a bad final answer can be traced back to the specific step that caused it. It is built for teams debugging agents that fail in ways normal software doesn't, and it is open source and OpenTelemetry-native. Use it when you need per-step visibility rather than only an end-output check; the tracing value depends on your agent being instrumented so its actions are captured as spans.

Why agent failures need step-level traces

TruLens frames the core problem directly: agents fail differently than normal software. A wrong answer at the end may originate in retrieval, a tool call, a prompt, or a model response several steps earlier. Checking only the final output tells you that something went wrong, not where. Tracing closes that gap by recording what happened at each step, so a bad answer has a traceable cause.

What each trace captures

Per the site, every span is recorded with:

• Latency — how long each step took
• Inputs and outputs — what went into and came out of each step
• Tokens and cost — consumption recorded per step

Because these are attached per step, you can compare a failing run against a healthy one and see which span diverged—and whether the fix is a quality problem or a cost problem.

Reading a trace: tree and timeline views

TruLens presents traces in two complementary views:

The trace detail view also exposes raw attributes, so you can inspect the underlying recorded data for a span rather than relying only on the summary.

Connecting scores to steps

Tracing alone shows what happened; scores show how good it was. TruLens scores every step, and its judges explain the score—so a low score points to exactly what went wrong rather than just flagging a bad output. The site reports that its judges are benchmarked against human annotations out of the box, including:

• 95% of agent errors caught with Agent GPA on TRAIL/GAIA (267 of 281 human-annotated errors, versus 55% for the baseline trace judge), per arXiv:2510.08847
• 0.81 Groundedness F1 on an LLM-AggreFact holdout
• 0.93 Context relevance NDCG@5 in an AIMultiple benchmark (23 March 2026)

Treat these as the vendor's cited benchmarks, not guarantees for your workload—your agent's domain and data determine how well the judges transfer.

Tuning judges to your domain

Out-of-the-box judges are the starting point. TruLens lets you align them to your domain with your private data: add a rubric, provide examples, and adjust the score range, with the plumbing handled for you. The site's example begins with importing Metric and Selector from trulens.core to define a custom metric. This matters because a generic judge may not recognize what "correct" means in your specific agent—tuning is how you make the failure signal trustworthy.

A practical way to use it

1. Instrument the agent so its actions are emitted as OpenTelemetry spans.
2. Reproduce a failure and open its trace.
3. Use the tree view to locate the branch where behavior diverged, and the timeline to check for slow or out-of-order steps.
4. Read the per-step scores and judge explanations to identify the specific step and reason for the failure.
5. Tune the judge with your own rubric and examples if the score doesn't match your notion of quality.
6. Re-run and compare traces to confirm the fix and check whether cost can be cut without losing quality.

Where it fits and what to check

TruLens is aimed at teams that need both tracing and evals for agents and want an open-source, OpenTelemetry-native option. Before adopting, confirm that your stack can emit OpenTelemetry spans and that the benchmarked judges perform well on samples from your own domain—the published numbers are evidence of capability, not a substitute for validating on your data. Pricing and access details beyond "open source" are not specified on the page, so verify those directly if they affect your decision.