Agent Observability

Agents are already becoming part of everyday software. Sometimes users work with them directly. Sometimes they sit behind a workflow and quietly make decisions on their own. In both cases, a single request can quickly turn into multiple model calls, tool calls, retries, and intermediate steps before the final answer shows up.

That is usually where debugging gets hard. When the final answer is wrong, slow, or incomplete, the answer itself is not enough. You need to see the full run: what prompt started it, which model was used, what tools were called, what each tool received, what came back, how many tokens were spent, and where the run started to drift. That is the job of Agent Observability.

Agent observability with Parseable

Parseable Agent Observability gives you that view inside Prism. You can begin with a high-level view of agent activity, look at model and tool behavior separately, and then move into a single agent run when you need the full execution path.

It is built on the OpenTelemetry GenAI semantic conventions, so the telemetry follows a standard shape for model calls, token usage, inputs, outputs, tool calls, and agent spans. For message content, Parseable follows the native GenAI model: use gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, and the gen_ai.client.inference.operation.details event when you capture full request and response details.

Once your agent is instrumented, the Agents page becomes the place where you understand how that agent behaves in production. You can see how often it runs, which models it uses, how many tokens it consumes, where it spends time, which tools it calls, and where a run needs review.

Explore agent activity

The Agents page is organized around four tabs: Overview, Models, Tools, and Agent Runs. The dataset selector at the top lets you choose the traces dataset you want to inspect, while the time range picker controls the window used by every chart and table on the page.

Overview

Start with the Overview tab when you want to understand the shape of agent activity before going deeper. It gives you the main health and usage signals in one place.

The summary cards show total cost, agent runs, errors, total tokens, LLM calls, and tool calls for the selected period. These are useful first checks. You can quickly see whether usage increased, whether errors appeared, and whether the growth is coming from more agent runs, more model calls, more tool calls, or heavier token usage.

Below the summary, the charts break the run down by model, tool, user, and time. Total cost by model shows which model is driving spend. Tool usage shows which tools are being called most often. Token usage by user and model helps you see who or what is consuming tokens. Cost over time, latency by tool, and latency by model help you spot spikes that are easier to miss in a table.

Use the time range picker in the top-right corner when you want to move from a broad view to a smaller investigation window. The whole page follows that selected range.

Models

The Models tab is for understanding model behavior. Use it when you want to answer questions like: which model is used most, which model is the most expensive, where are tokens being spent, and whether a model is getting slower over time.

The top cards show total cost, most used model, highest spend model, input tokens, output tokens, and cached tokens. This gives you a quick model-level summary before you inspect the charts.

The charts let you compare total cost by model, cost over time by model, tokens by user and model, cost per agent, cost per team, cost by agent run, token usage over time, and model usage over time. This helps you see whether a cost spike came from a model switch, a specific user, a team, or a small number of expensive agent runs.

Tools

The Tools tab helps you understand how the agent interacts with the outside world. This matters because many agent failures are not model failures. They happen when a tool is slow, returns unexpected output, fails repeatedly, or gets called too often.

The top cards show the most used tool, slowest tool, total tool calls, total agent runs, and errors. The charts below show tool calls over time, tool latency over time, tool usage, and tool failures.

The tool failures table gives you a path back to the run that failed. It includes the start time, tool name, duration, and trace ID, so you can move from a failing tool trend to the exact trace that explains it.

Agent Runs

The Agent Runs tab is where you move from charts to individual runs. Use it when you need to inspect a specific prompt, a failed run, an expensive run, or a run that called too many tools.

The timeline at the top shows when runs happened. The table below lists each run with date, prompt, model, tokens, cost, duration, and operations. Operations show how many model interactions and tool calls happened inside the run, including failed tool calls when they exist.

Use Add filter to narrow the table when you already know what kind of run you are looking for. You can filter by fields such as model, tool, token counts, duration, errors, and other captured attributes. Use Find in data when you want to search within the currently visible results.

When you open a run, Parseable shows the trace behind it. User-facing input appears as user messages in the UI. Model or agent output appears as agent messages. Under the hood, keep this telemetry native to OpenTelemetry by recording message content with gen_ai.input.messages, gen_ai.output.messages, and gen_ai.client.inference.operation.details rather than relying on custom message event names.

Manual instrumentation using the OpenTelemetry SDK

If you want to instrument your own agent, you can do that directly with the OpenTelemetry SDK. This works well when you have a custom agent, a provider-specific client, or a framework setup where you want full control over what gets captured.

At minimum, instrument these parts of the agent workflow:

1. Agent-level spans: invoke_agent spans that wrap the full agent loop
2. Chat spans: one chat {model} span for every LLM request
3. Tool execution spans: one execute_tool {tool.name} span for every tool or command execution
4. GenAI events and message attributes: gen_ai.client.inference.operation.details, gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, tool calls, tool outputs, reasoning blocks, and evaluation results attached to the active span

The manual instrumentation guide shows how to structure those spans and what to capture so the data shows up cleanly in Parseable.

Language support matrix

Any language that can emit OpenTelemetry traces over OTLP can send agent observability data to Parseable. The manual guide currently uses Python examples, but the schema is language-agnostic.

Collector configuration

After instrumentation is in place, the next step is deciding how you want to send the trace data to Parseable. For most teams, an OpenTelemetry Collector is the better default because it gives you batching and a more reliable path. If your setup is small, you can also export directly from the application.

OpenTelemetry Collector (Recommended)

Put an OpenTelemetry Collector between your application and Parseable when you want a cleaner and more production-friendly path for export. Save the following as parseable-genai-collector.yaml:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s
    send_batch_size: 256

exporters:
  otlphttp/parseable:
    endpoint: ${PARSEABLE_URL}
    encoding: json
    headers:
      Authorization: "Basic ${PARSEABLE_AUTH}"
      X-P-Stream: "${STREAM_NAME}"
      X-P-Log-Source: "otel-traces"
      X-P-Dataset-Tag: "agent-observability"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/parseable]

Run the collector:

export PARSEABLE_URL=${PARSEABLE_URL}          # e.g. https://ingest.parseable.com
export PARSEABLE_AUTH=${PARSEABLE_AUTH}          # base64(username:password)
export STREAM_NAME=${PARSEABLE_DATASET_NAME}

otelcol-contrib --config parseable-genai-collector.yaml

Direct to Parseable

For smaller setups, you can send traces directly from the application to Parseable with OTLP environment variables. This avoids running a separate collector process.

export OTEL_EXPORTER_OTLP_ENDPOINT=${PARSEABLE_URL}
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic ${PARSEABLE_AUTH},X-P-Stream=${PARSEABLE_DATASET_NAME},X-P-Log-Source=otel-traces,X-P-Dataset-Tag=agent-observability"
export OTEL_SERVICE_NAME=my-agent
export OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

Next steps