OpenLIT Agent Observability

OpenLIT is an OpenTelemetry-native observability SDK for LLM applications and AI agents. It can instrument supported agent frameworks, model providers, and tools so one trace shows the complete path from an agent invocation to its workflow steps, tool executions, and model calls.

This integration is framework-neutral. Use the same Parseable and OpenTelemetry Collector configuration with CrewAI, OpenAI Agents SDK, LangChain or LangGraph, Agno, LlamaIndex, and other frameworks supported by OpenLIT.

Agent application
  |
  | OpenLIT instrumentation
  v
Agent invocation -> workflow -> tool calls -> model calls
  |
  | OTLP/HTTP
  v
OpenTelemetry Collector
  |
  +--> openlit-agent-traces   Agent Observability and trace dataset
  +--> openlit-agent-metrics  Metrics dataset

Prerequisites

• A running Parseable instance
• A Parseable API key with dataset creation and ingest access
• Python 3.9 or newer
• An OpenTelemetry Collector Contrib binary or container
• A supported agent framework and its model-provider API key

Set up OpenLIT Agent Observability

export PARSEABLE_URL="https://<parseable-host>:8000"
export PARSEABLE_API_KEY="<parseable-api-key>"
export OPENLIT_TRACE_STREAM="openlit-agent-traces"
export OPENLIT_METRIC_STREAM="openlit-agent-metrics"

curl -X PUT "$PARSEABLE_URL/api/v1/logstream/$OPENLIT_TRACE_STREAM" \
  -H "X-API-Key: ${PARSEABLE_API_KEY}" \
  -H "X-P-Log-Source: otel-traces" \
  -H "X-P-Telemetry-Type: traces" \
  -H "X-P-Dataset-Tag: agent-observability"

curl -X PUT "$PARSEABLE_URL/api/v1/logstream/$OPENLIT_METRIC_STREAM" \
  -H "X-API-Key: ${PARSEABLE_API_KEY}" \
  -H "X-P-Log-Source: otel-metrics" \
  -H "X-P-Telemetry-Type: metrics"

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s

exporters:
  otlphttp/parseable_traces:
    endpoint: "${env:PARSEABLE_URL}"
    encoding: json
    headers:
      X-API-Key: "${env:PARSEABLE_API_KEY}"
      X-P-Stream: "${env:OPENLIT_TRACE_STREAM}"
      X-P-Log-Source: otel-traces

  otlphttp/parseable_metrics:
    endpoint: "${env:PARSEABLE_URL}"
    encoding: json
    headers:
      X-API-Key: "${env:PARSEABLE_API_KEY}"
      X-P-Stream: "${env:OPENLIT_METRIC_STREAM}"
      X-P-Log-Source: otel-metrics

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/parseable_traces]

    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/parseable_metrics]

otelcol-contrib --config otel-collector-config.yaml

pip install openlit

import openlit

openlit.init(
    otlp_endpoint="http://localhost:4318",
    service_name="support-agent",
    environment="production",
    capture_message_content=False,
)

# Import and run your supported agent framework only after initialization.

export OTEL_METRIC_EXPORT_INTERVAL=1000
python agent.py

What you get in Parseable

Exact span names and attributes depend on the framework, provider, and OpenLIT version. A complete agent trace can contain:

Common fields include:

OpenLIT also emits metrics such as gen_ai.client.operation.duration, gen_ai.client.token.usage, gen_ai.usage.cost, gen_ai.server.time_to_first_token, and gen_ai.server.time_per_output_token.

Query agent telemetry

Agent and workflow operations:

SELECT
  "gen_ai.operation.name" AS operation,
  COUNT(*) AS spans,
  COUNT(DISTINCT "span_trace_id") AS runs
FROM "openlit-agent-traces"
WHERE p_timestamp > NOW() - INTERVAL '24 hours'
GROUP BY operation
ORDER BY spans DESC;

Model usage caused by agent runs:

SELECT
  "gen_ai.request.model" AS model,
  COUNT(*) AS calls,
  SUM(CAST("gen_ai.usage.input_tokens" AS BIGINT)) AS input_tokens,
  SUM(CAST("gen_ai.usage.output_tokens" AS BIGINT)) AS output_tokens
FROM "openlit-agent-traces"
WHERE "gen_ai.operation.name" = 'chat'
  AND p_timestamp > NOW() - INTERVAL '24 hours'
GROUP BY model
ORDER BY calls DESC;

Troubleshoot

• Dataset appears in Traces but not Agents: The dataset was probably auto-created without the Agent Observability tag. Recreate an empty dataset with X-P-Log-Source: otel-traces, X-P-Telemetry-Type: traces, and X-P-Dataset-Tag: agent-observability before ingesting data.
• Traces contain only chat operations: OpenLIT is observing direct model calls, not an agent-framework execution. Confirm that the installed framework is supported, initialize OpenLIT before importing it, and execute the framework's agent runner rather than only its underlying model client.
• Tool calls appear without tool execution spans: A model can request a tool without the application executing it. Confirm that the agent framework dispatches the tool and that OpenLIT supports instrumenting that framework's tool runtime.
• No traces appear: Check Collector logs, confirm the application exports OTLP/HTTP to port 4318, and verify the Parseable exporter includes X-P-Stream and X-P-Log-Source: otel-traces.
• Traces appear but metrics are empty: Metrics use a periodic exporter. Keep the application alive until the first export or temporarily set OTEL_METRIC_EXPORT_INTERVAL=1000.
• Prompts or responses appear in traces: Set capture_message_content=False. This affects only newly generated telemetry; review or remove older records separately.

Related documentation

• OpenLIT SDK
• Parseable Agent Observability
• Parseable OpenTelemetry traces
• Parseable OpenTelemetry metrics
• OpenTelemetry Collector