Pydantic AI

Pydantic AI is a Python framework from the Pydantic team for building typed agents. It is useful when you want agent code to feel like normal Python, with tools, structured outputs, retries, validation, and model calls kept close to your application logic.

When an agent runs in production, the final answer is usually the least interesting part of the story. You also need to know which model was called, which tools ran, how long each step took, how many tokens were used, and where a retry or failure happened. Pydantic AI has built-in OpenTelemetry instrumentation for that path, and Parseable can receive the emitted telemetry directly.

This guide shows how to send Pydantic AI traces and metrics to Parseable. The traces power Parseable Agent Observability, while the metrics help you track usage and latency over time.

How it works

Pydantic AI emits OpenTelemetry spans for agent runs, model requests, and tool execution after instrumentation is enabled. A single agent run usually becomes one trace with the agent span at the top, model-call spans below it, and tool spans where the agent calls Python functions.

Pydantic AI application
  |
  | OpenTelemetry traces and metrics
  v
Parseable
  |
  +--> pydantic-ai-traces   traces dataset and Agent Observability
  +--> pydantic-ai-metrics  metrics dataset

You can send telemetry through an OpenTelemetry Collector if that is already part of your deployment. For a small application or first test, sending directly to Parseable is also fine.

Prerequisites

Before you start, keep these ready:

Python 3.10 or later
A running Parseable instance
A Parseable API key with ingest access
An OpenAI API key, or another model-provider key supported by Pydantic AI

Set up Pydantic AI with Parseable

Create Parseable datasets

Create a trace dataset and tag it for Agent Observability:

curl -X PUT "$PARSEABLE_URL/api/v1/logstream/pydantic-ai-traces" \
  -H "X-API-Key: ${PARSEABLE_API_KEY}" \
  -H "X-P-Log-Source: otel-traces" \
  -H "X-P-Telemetry-Type: traces" \
  -H "X-P-Dataset-Tag: agent-observability"

Create a metrics dataset when you also want token usage, latency, and streaming metrics:

curl -X PUT "$PARSEABLE_URL/api/v1/logstream/pydantic-ai-metrics" \
  -H "X-API-Key: ${PARSEABLE_API_KEY}" \
  -H "X-P-Log-Source: otel-metrics" \
  -H "X-P-Telemetry-Type: metrics"

X-P-Dataset-Tag tags the dataset itself. Do not use X-P-Tag-* for this step, because those headers add fields to ingested records instead.

Install dependencies

Install Pydantic AI with OpenAI support and the OpenTelemetry OTLP HTTP exporter:

pip install "pydantic-ai-slim[openai]" \
  opentelemetry-sdk \
  opentelemetry-exporter-otlp-proto-http

Configure OpenTelemetry

Create agent.py:

import os

from opentelemetry import metrics, trace
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from pydantic_ai import Agent, InstrumentationSettings, RunContext
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

parseable_url = os.environ["PARSEABLE_URL"].rstrip("/")
parseable_api_key = os.environ["PARSEABLE_API_KEY"]

resource = Resource.create(
    {
        "service.name": "pydantic-ai-agent",
        "deployment.environment.name": "production",
    }
)

tracer_provider = TracerProvider(resource=resource)
tracer_provider.add_span_processor(
    BatchSpanProcessor(
        OTLPSpanExporter(
            endpoint=f"{parseable_url}/v1/traces",
            headers={
                "X-API-Key": parseable_api_key,
                "X-P-Stream": "pydantic-ai-traces",
                "X-P-Log-Source": "otel-traces",
            },
        )
    )
)
trace.set_tracer_provider(tracer_provider)

metric_reader = PeriodicExportingMetricReader(
    OTLPMetricExporter(
        endpoint=f"{parseable_url}/v1/metrics",
        headers={
            "X-API-Key": parseable_api_key,
            "X-P-Stream": "pydantic-ai-metrics",
            "X-P-Log-Source": "otel-metrics",
        },
    ),
    export_interval_millis=5000,
)
meter_provider = MeterProvider(resource=resource, metric_readers=[metric_reader])
metrics.set_meter_provider(meter_provider)

Agent.instrument_all(
    InstrumentationSettings(
        version=5,
        include_content=True,
        tracer_provider=tracer_provider,
        meter_provider=meter_provider,
    )
)

model = OpenAIChatModel(
    "gpt-4o-mini",
    provider=OpenAIProvider(api_key=os.environ["OPENAI_API_KEY"]),
)
agent = Agent(model, name="weather_agent")


@agent.tool
def get_weather(ctx: RunContext[None], city: str) -> str:
    return f"The weather in {city} is sunny and 24 C."


try:
    result = agent.run_sync("What is the weather in Bengaluru?")
    print(result.output)
finally:
    tracer_provider.force_flush()
    tracer_provider.shutdown()
    meter_provider.force_flush()
    meter_provider.shutdown()

Run the application:

export PARSEABLE_URL="https://<parseable-host>:8000"
export PARSEABLE_API_KEY="<parseable-api-key>"
export OPENAI_API_KEY="<openai-api-key>"

python agent.py

What the instrumentation captures

With InstrumentationSettings(version=5), Pydantic AI emits the current instrumentation format. In Parseable, a normal tool-using run should look like this:

invoke_agent weather_agent
├── chat gpt-4o-mini
├── execute_tool get_weather
└── chat gpt-4o-mini

The agent span represents the full run. Model spans show the provider, requested model, response model, token usage, latency, and errors. Tool spans show the tool name, tool call ID, arguments, result, and duration. This is the structure that makes it possible to move from a high-level agent run into the exact model or tool step that changed the outcome.

For Agent Observability, the most important tool fields are:

Field	Meaning
`gen_ai.tool.name`	Tool function name
`gen_ai.tool.call.id`	Tool call identifier
`gen_ai.tool.call.arguments`	Arguments passed to the tool
`gen_ai.tool.call.result`	Tool result returned to the agent

If the Agent Observability setup page reports missing tool arguments or results, confirm that you are using instrumentation version=5, include_content=True, and that the run actually invoked a tool.

Metrics

Pydantic AI records OpenTelemetry histograms for model usage and streaming behavior:

Metric	Unit	When emitted
`gen_ai.client.token.usage`	tokens	Model or embedding requests, split by `gen_ai.token.type`
`operation.cost`	USD	When the model price is known
`gen_ai.client.operation.time_to_first_chunk`	seconds	Streaming requests only

Each metric point carries model and provider attributes such as gen_ai.provider.name, gen_ai.operation.name, gen_ai.request.model, and gen_ai.response.model. Streaming latency appears only when your code consumes the stream far enough for Pydantic AI to observe the first chunk.

Since these are histogram metrics, query data_point_sum and data_point_count rather than data_point_value.

Example token usage query:

SELECT
  attributes->>'gen_ai.token.type' AS token_type,
  SUM(data_point_sum) AS tokens
FROM "pydantic-ai-metrics"
WHERE metric_name = 'gen_ai.client.token.usage'
GROUP BY token_type;

Example average time to first chunk:

SELECT
  SUM(data_point_sum) / NULLIF(SUM(data_point_count), 0) AS avg_ttf_chunk_seconds
FROM "pydantic-ai-metrics"
WHERE metric_name = 'gen_ai.client.operation.time_to_first_chunk';

Content and privacy

include_content=True captures prompts, completions, tool arguments, and tool results. This is useful for understanding agent behavior, and it gives Agent Observability enough detail to show tool input and output. It can also capture sensitive data, so review this setting before using it in production.

If your production policy is to avoid content capture, use:

Agent.instrument_all(
    InstrumentationSettings(
        version=5,
        include_content=False,
        include_binary_content=False,
        include_model_request_parameters=False,
        tracer_provider=tracer_provider,
        meter_provider=meter_provider,
    )
)

With include_content=False, you still get span structure, model metadata, usage, latency, and errors. The tradeoff is that prompt content, tool arguments, and tool results will not be available in Parseable.

Instrument all agents or one agent

Agent.instrument_all(...) applies to every agent that does not provide its own instrumentation capability. This is the simplest setup for most applications.

If different agents need different privacy settings or telemetry destinations, configure instrumentation per agent:

from pydantic_ai import Agent, InstrumentationSettings
from pydantic_ai.capabilities import Instrumentation

settings = InstrumentationSettings(
    version=5,
    include_content=False,
    tracer_provider=tracer_provider,
    meter_provider=meter_provider,
)

agent = Agent(model, capabilities=[Instrumentation(settings=settings)])

Token aggregation

Model request spans use standard token attributes such as gen_ai.usage.input_tokens and gen_ai.usage.output_tokens. Agent run spans use gen_ai.aggregated_usage.* by default, because they contain totals from their child model calls. This avoids double-counting when dashboards aggregate token usage across parent and child spans.

If your own queries already separate agent spans from model spans, you can disable the aggregated names:

InstrumentationSettings(
    version=5,
    use_aggregated_usage_attribute_names=False,
)

Change this only if you know how your dashboards count usage, since it can affect totals.

Verify in Parseable

Open the pydantic-ai-traces dataset from the Traces page. You should see agent, model, and tool spans in one trace. If the dataset is tagged with agent-observability, it will also appear in the Agent Observability flow.

Open the pydantic-ai-metrics dataset from the Metrics page. You should see metrics such as gen_ai.client.token.usage, operation.cost, and gen_ai.client.operation.time_to_first_chunk after the application has emitted enough model calls.

Troubleshooting

No traces appear

Confirm Agent.instrument_all(...) runs before the agent call, and check that the trace exporter points to ${PARSEABLE_URL}/v1/traces with X-P-Stream: pydantic-ai-traces and X-P-Log-Source: otel-traces.
Metrics are missing

Confirm the MeterProvider is passed to InstrumentationSettings, and check that the metric exporter points to ${PARSEABLE_URL}/v1/metrics with X-P-Stream: pydantic-ai-metrics and X-P-Log-Source: otel-metrics.
Tool arguments or results are missing

Confirm include_content=True and run an agent path that actually calls a tool. If include_content=False, Pydantic AI intentionally removes prompts, completions, tool arguments, and tool results from telemetry.
Streaming latency is missing

Use a streaming run and consume or close the stream. Pydantic AI records time-to-first-chunk only after the streaming path produces the required timing data.
Cost is missing

Cost is recorded only when Pydantic AI can resolve pricing for the model. Token usage and latency should still be available.