Skip to main content

OpenTelemetry Logging

OpenTelemetry is an open-source observability framework that provides a standard way to collect, process, and export telemetry data. It simplifies the instrumentation of applications and services for monitoring and troubleshooting.

Parseable supports OpenTelemetry logging, enabling you to ingest log data in the OpenTelemetry format. This integration allows you to leverage the rich telemetry data generated by your applications to gain insights into their performance and behavior.

Benefits of using Parseable to ingest OpenTelemetry logs:

  • Easy Integration: Easily ingests OpenTelemetry logs for efficient monitoring.
  • Scalability: Handles large volumes of log data effectively.
  • Real-time Monitoring: Provides real-time insights into application performance.
  • Vendor Agnostic: Works with various OpenTelemetry Collectors.

Getting Started with OpenTelemetry Logging in Parseable

To get started with OpenTelemetry logging in Parseable, follow these steps:

Instrument Your Applications

Use the OpenTelemetry SDK to instrument your applications and services to generate telemetry data.

Configure the OpenTelemetry Collector

Set up the OpenTelemetry Collector to receive telemetry data from your applications and services and forward it to Parseable. You can use any Opentelemetry Collector that supports OTLPHTTPEXPORTER.

note

As of our current version, we only support HTTP exporter.( Ingestion over HTTP is supported) Protohuf ingestion is not yet supported. We'll update this document once it is supported.

info

OTEL FLAG: Set the X-P-Log-Source flag to otel in the log configuration to indicate that the log data is in the OpenTelemetry format.

You need to use the OTELHTTPEXPORTER to enable parseable to recieve the Opentelemetry logs. Here is an example configuration for the OpenTelemetry Collector to send telemetry data to Parseable:

exporters:
otlp/parseable:
endpoint: "http://<your-parseable-instance-url>/v1/logs"
headers:
X-P-Log-Source: "otel"
X-P-stream-name: "<your-stream-name>"
compression: "gzip"
timeout: 5s
headers:
Authorization: "Bearer <your-token>"

Change the <your-parseable-instance-url>, <your-stream-name>, and <your-api-key> with your Parseable instance URL, stream name, and Bearer Token, respectively.

Data flattening

The OpenTelemetry logs that are in the form of a nested JSON. When the flag X-P-Log-Source is set to otel, Parseable will flatten the nested JSON to a flat JSON object. This is done to make the data more queryable and filterable. Here's a quick comparison of the nested JSON and the flattened JSON:

Nested JSON:


"name": "foo",
"context": {
"trace_id": "0x01be986dc847202bedc26bb9eaeb938d",
"span_id": "0x039c36ca53465947",
"trace_state": "[]"
},
"kind": "SpanKind.INTERNAL",
"parent_id": null,
"start_time": "2024-06-11T08:47:46.379395Z",
"end_time": "2024-06-11T08:47:46.380386Z",
"status": {
"status_code": "UNSET"
},
"attributes": {},
"events": [],
"links": [],
"resource": {
"attributes": {
"telemetry.sdk.language": "python",
"telemetry.sdk.name": "opentelemetry",
"telemetry.sdk.version": "1.24.0",
"service.name": "unknown_service"
},
"schema_url": ""
}

Flattened JSON:

{
"name": "foo",
"context.trace_id": "0x01be986dc847202bedc26bb9eaeb938d",
"context.span_id": "0x039c36ca53465947",
"context.trace_state": "[]",
"kind": "SpanKind.INTERNAL",
"parent_id": null,
"start_time": "2024-06-11T08:47:46.379395Z",
"end_time": "2024-06-11T08:47:46.380386Z",
"status.status_code": "UNSET",
"resource.attributes.telemetry.sdk.language": "python",
"resource.attributes.telemetry.sdk.name": "opentelemetry",
"resource.attributes.telemetry.sdk.version": "1.24.0",
"resource.attributes.service.name": "unknown_service",
"resource.schema_url": ""
}

Parseable flatens the JSON based on the Key and the value. The key is formed by concatenating the parent key and the child key with a .. The value is the value of the child key.

Analyze Telemetry Data

Use Parseable's log analysis tools to query, filter, and visualize the telemetry data ingested from your applications and services.

Get Updates from Parseable

Subscribe to keep up with latest news, updates and new features on Parseable