Scrub PII at the Source with OpenTelemetry Collector

Why does this matter now.

Because observability pipelines carry more application context than most teams expect.

A log line can hold an email address. A span attribute can hold a full URL with query parameters. A database statement can carry customer identifiers. A request header can carry a token. By the time that data has already landed in an observability backend, fixing it becomes cleanup work. The better option is to make sure it never leaves your environment in the first place.

That is why source-side scrubbing matters. If the OpenTelemetry Collector removes, masks, or hashes sensitive fields before export, every downstream system only ever sees the cleaned version.

This is also not only a logs problem. Traces carry plenty of sensitive context too. Span attributes, request headers, route parameters, SQL statements, prompt payloads, and custom business fields can all show up in trace data. If the rule only covers logs, the same value can still leak through another signal.

Why the Collector is the right control point

The OpenTelemetry Collector already sits in the middle of the telemetry path. It receives data, processes it, and then exports it. That makes it the most natural place to apply privacy rules once and keep them consistent everywhere else.

In the simplest form, the flow looks like this:

application or SDK
  -> OpenTelemetry Collector receiver
  -> processor
  -> exporter
  -> observability backend

That placement matters more than it first appears to.

If the processor runs before the exporter, the scrubbed version is the only version that leaves your network. You do not need to rely on a backend feature, a retention policy, or a later cleanup step to keep the data safe.

In practice, the risky fields are usually not hard to guess. They tend to be the same ones teams keep around because they are useful during debugging: Authorization headers, user identifiers, raw URLs, query parameters, SQL statements, session payloads, or prompt content. The problem is not that engineers do not know these fields are sensitive. The problem is that they often leave them in the pipeline because removing them later feels easier. Most of the time, it is not.

Three processor patterns that matter

The Collector gives you a few ways to handle this, and they are useful for different kinds of data.

Attributes processor

This is the cleanest place to start when you already know which fields are sensitive. If your services consistently emit keys like user.email, session.id, or http.request.header.authorization, then the rule can be direct and boring, which is a good thing.

processors:
  attributes/scrub_known_fields:
    actions:
      - key: http.request.header.authorization
        action: delete
      - key: user.email
        action: hash
      - key: session.id
        action: delete

For stable schemas, this is usually enough. You know the keys, you know the action, and the rule stays easy to reason about.

Redaction processor

Things get messier when the schema is not stable. One service writes api_token, another writes access_token, and a third one hides an email address inside a free-form value. That is where the redaction processor becomes more useful.

You can use it as a strict allowlist:

processors:
  redaction/allowlist:
    allow_all_keys: false
    allowed_keys:
      - http.method
      - http.route
      - http.status_code
      - service.name

Or you can keep most fields and block suspicious keys or values:

processors:
  redaction/mask_patterns:
    allow_all_keys: true
    redact_all_types: true
    blocked_key_patterns:
      - ".*token.*"
      - ".*password.*"
      - ".*secret.*"
    blocked_values:
      - "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}"

This is the processor to reach for when the data is unpredictable and you need a stronger safety net than a few hard-coded keys.

Transform processor

The transform processor is the flexible option. It is the one to use when the rule depends on context, signal type, or a pattern that needs a little more logic.

This is also where traces become part of the story in a very direct way. The Collector is not limited to cleaning log records. It can act on spans too.

processors:
  transform/scrub_sensitive:
    error_mode: ignore
    trace_statements:
      - replace_pattern(span.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
      - delete_key(span.attributes, "http.request.header.authorization")
    log_statements:
      - delete_key(log.attributes, "http.request.header.authorization")

That is the part many teams miss at first. Sensitive data is not only something you strip from log bodies. It can live inside span attributes just as easily, and the Collector gives you a way to clean that before export as well.

Sending scrubbed telemetry to Parseable

Once the processor rules are in place, the rest of the flow is straightforward. The Collector receives logs and traces, scrubs them, and only then exports them to Parseable.

receivers:
  otlp:
    protocols:
      grpc:
      http:
 
processors:
  attributes/scrub_known_fields:
    actions:
      - key: http.request.header.authorization
        action: delete
      - key: user.email
        action: hash
  batch:
 
exporters:
  otlphttp/parseablelogs:
    logs_endpoint: "https://<YOUR_PARSEABLE_URL>/v1/logs"
    headers:
      X-P-Stream: "sanitized-logs"
      X-P-Log-Source: "otel-logs"
 
  otlphttp/parseabletraces:
    traces_endpoint: "https://<YOUR_PARSEABLE_URL>/v1/traces"
    headers:
      X-P-Stream: "sanitized-traces"
      X-P-Log-Source: "otel-traces"
 
service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [attributes/scrub_known_fields, batch]
      exporters: [otlphttp/parseablelogs]
    traces:
      receivers: [otlp]
      processors: [attributes/scrub_known_fields, batch]
      exporters: [otlphttp/parseabletraces]

The most important detail here is not the endpoint name. It is the order of the pipeline. The processor sits before the exporter, so the cleaned version is the one that gets forwarded.

If you are starting this work from scratch, it is tempting to focus only on logs because they feel more obviously risky. But traces deserve the same treatment. A span can easily include a full request URL, route parameters, SQL fragments, or custom attributes that were added for convenience during debugging. If logs are clean and traces are not, you have not really solved the problem. You have only moved it.

Validate before you trust it

Privacy rules are only useful if you know they are actually doing what you think they are doing.

One simple way to validate a new rule is to add the debug exporter for a short period and inspect what the Collector forwards after processing:

exporters:
  debug:
    verbosity: detailed

Then include it temporarily in the same pipeline:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [transform/scrub_sensitive, batch]
      exporters: [debug, otlphttp/parseabletraces]

That gives you a better answer than assumption ever will. You can see the final payload after the scrubbing logic has run, which is the only version that really matters.

Most teams also do better when they start with the smallest rule that solves the problem. If the keys are known, use the attributes processor. If the schema is less predictable, use redaction. If the rule depends on the signal or the context, use transform. The goal is not to build the most elaborate privacy pipeline on day one. The goal is to stop the obvious leaks first and then tighten the rules as you learn where sensitive data really shows up.

Parseable gives you a fast place to work with logs and traces once the data reaches your observability pipeline. But the privacy decision should happen before that data gets shipped anywhere. The Collector gives you that control point, and in most environments it is the cleanest place to use it.

Conclusion

Source-side PII scrubbing is not only a compliance step. It is one of the most practical ways to make an observability pipeline safer without losing the fields that still help during debugging.

If your team already uses OpenTelemetry, you do not need to wait for a backend-side feature to get started. The Collector already gives you the processors to delete, mask, hash, or transform sensitive fields before export. That applies to logs, and it applies to traces too.

That is really the whole point.

The best time to decide what should leave your network is before it leaves your network.