The OpenTelemetry Collector is often treated like plumbing. Put it between your applications and your observability backend, add a few processors, point the exporters at the right place, and move on.

That works until the Collector becomes the busiest process in the path.

A few things can happen at once. Traffic spikes. A backend slows down. Exporter queues start filling. Tail sampling holds more traces than usual. Batch sizes grow. A receiver keeps accepting data faster than the rest of the pipeline can move it. The Collector is still doing exactly what you asked it to do, but memory keeps climbing.

At that point, you do not want the Collector to be polite. You want it to protect itself.

That is the job of the OpenTelemetry memory_limiter processor. It is a guardrail that lets the Collector apply backpressure before memory pressure turns into an out-of-memory kill.

What it does

The memory_limiter processor periodically checks the Collector process memory. When usage crosses a configured threshold, it starts refusing new telemetry. When usage crosses a higher threshold, it also asks the Go runtime to run garbage collection.

It is not a magic memory reducer. It does not shrink exporter queues, rewrite your sampling strategy, or make an undersized Collector suddenly production-ready. The official OpenTelemetry Collector docs are very clear about this: the processor helps mitigate out-of-memory situations, but it does not replace proper sizing and pipeline design.

Think of it as a circuit breaker in the telemetry path.

apps / agents
     |
     v
  receivers <------ retryable error
     |                    ^
     v                    |
memory_limiter -----------+
     |
     v
enrich / filter
     |
     v
   batch
     |
     v
 exporters
     |
     v
 Parseable
 
memory pressure is checked here,
before the expensive work continues.

That sketch is the important mental model.

The limiter sits early in the pipeline. If memory is healthy, telemetry flows through normally. If memory is too high, the limiter refuses new data by returning a non-permanent error to the previous component. Well-behaved receivers retry and slow down. That gives the Collector time to recover instead of falling over.

Why it matters

Collector memory usage is workload-specific. Two teams can run the same binary with completely different memory profiles.

One team may run a lightweight agent on every Kubernetes node. Another may run a central gateway that receives all traces for a region. A third may use tail sampling, queued retry, transform processors, and multiple exporters. The Collector is flexible enough for all of these shapes, but memory behavior changes as soon as you change the pipeline.

Here are the common pressure points:

Exporter queues grow when the destination is slow, unavailable, rate-limited, or temporarily unreachable.
Tail sampling holds traces in memory while waiting to decide whether to keep or drop them.
Large batches improve throughput but increase the amount of telemetry held at once.
Expensive processors such as transform, redaction, span metrics, or service graph processing can add temporary allocations.
Traffic spikes can move memory faster than the Collector checks it.
Non-OTLP receivers may need to decode or buffer incoming data before the memory limiter can reject it.

The memory limiter is useful because it gives the Collector an explicit behavior under pressure. Without it, memory pressure usually ends with the operating system or container runtime making the decision for you.

Soft limit, hard limit, and spike limit

The processor works with two limits:

Hard limit: the maximum heap memory target for the Collector.
Soft limit: the point where the Collector starts refusing data.

The soft limit is calculated from the hard limit minus the spike limit.

soft limit = hard limit - spike limit

For example:

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 4000
    spike_limit_mib: 800

This gives you:

hard limit = 4000 MiB
soft limit = 4000 MiB - 800 MiB = 3200 MiB

Below 3200 MiB, the Collector behaves normally. Above 3200 MiB, the memory limiter starts refusing data. Above 4000 MiB, it continues refusing data and also forces garbage collection.

Once memory drops below the soft limit again, normal operation resumes.

That middle area between soft and hard limit is the buffer. It exists because memory can rise between checks. If your traffic is spiky, make the check interval shorter or make the spike limit larger. A good starting point is a spike limit around 20 percent of the hard limit.

Where to place it

This is the rule I would make boring and consistent:

processors: [memory_limiter, batch]

The memory limiter should be the first processor in each pipeline.

If you put it after batch, transform, tail_sampling, or other processors, then the Collector has already spent memory doing work before the limiter gets a chance to say no. You lose the main benefit, which is early backpressure.

Here is a minimal Collector configuration:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
 
processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 4000
    spike_limit_mib: 800
 
  batch:
    send_batch_size: 1024
    timeout: 5s
 
exporters:
  otlp:
    endpoint: tempo.example.com:4317
 
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp]

That config is not fancy, but the order is right.

Parseable example

If you are sending telemetry to Parseable through OTLP HTTP, the same rule applies. Put memory_limiter first, then do the normal pipeline work, then export.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
 
processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 1800
    spike_limit_mib: 400
 
  resource:
    attributes:
      - key: deployment.environment
        value: production
        action: upsert
      - key: service.namespace
        value: checkout
        action: upsert
 
  batch:
    send_batch_size: 1024
    timeout: 5s
 
exporters:
  otlphttp/parseablelogs:
    logs_endpoint: "https://<PARSEABLE_URL>/v1/logs"
    headers:
      Authorization: "Bearer <PARSEABLE_API_KEY>"
      X-P-Stream: "otel-logs"
      X-P-Log-Source: "otel-logs"
 
  otlphttp/parseabletraces:
    traces_endpoint: "https://<PARSEABLE_URL>/v1/traces"
    headers:
      Authorization: "Bearer <PARSEABLE_API_KEY>"
      X-P-Stream: "otel-traces"
      X-P-Log-Source: "otel-traces"
 
service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [memory_limiter, resource, batch]
      exporters: [otlphttp/parseablelogs]
 
    traces:
      receivers: [otlp]
      processors: [memory_limiter, resource, batch]
      exporters: [otlphttp/parseabletraces]

The memory limiter is not there because Parseable requires it. It is there because the Collector is now part of your production data path. If the destination is temporarily slow, or if traffic spikes, the Collector should degrade in a controlled way instead of disappearing.

Kubernetes settings

In Kubernetes, the Collector usually runs with a container memory limit. In that case, percentage-based settings are easier to keep aligned with the deployment.

processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20

If the Collector container has a 2 GiB memory limit, limit_percentage: 80 targets about 1.6 GiB as the hard limit. With a 20 percent spike limit, the soft limit lands around 1.2 GiB.

Here is a deployment sketch:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: otel-collector
spec:
  replicas: 2
  selector:
    matchLabels:
      app: otel-collector
  template:
    metadata:
      labels:
        app: otel-collector
    spec:
      containers:
        - name: otel-collector
          image: otel/opentelemetry-collector-contrib:latest
          args:
            - "--config=/etc/otelcol/config.yaml"
          env:
            - name: GOMEMLIMIT
              value: "1300MiB"
          resources:
            requests:
              cpu: 500m
              memory: 1Gi
            limits:
              cpu: "2"
              memory: 2Gi

Notice the GOMEMLIMIT setting. The OpenTelemetry docs recommend using GOMEMLIMIT along with the memory limiter. The Go runtime limit helps garbage collection react earlier. The memory limiter is the pipeline-level circuit breaker if memory still climbs.

A sensible starting point is:

Set the container memory limit first.
Set GOMEMLIMIT below the memory limiter hard limit.
Set the memory limiter hard limit below the container limit.
Leave enough headroom for non-heap memory and normal process overhead.

For a 2 GiB Collector container, a reasonable first pass might look like:

container memory limit: 2048 MiB
GOMEMLIMIT:            1300 MiB
memory_limiter hard:   ~1600 MiB
memory_limiter soft:   ~1200 MiB

These are starting values, not universal values. Watch the Collector under real traffic and tune from there.

Fixed limits

Use limit_mib when the Collector runs on a VM, bare metal host, or another environment where you know the memory budget and do not want the Collector to scale its heap target with the host's total memory.

For example, a central gateway with 8 GiB available might start here:

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 6400
    spike_limit_mib: 1280

That sets the hard limit at 6.4 GiB and the soft limit at 5.12 GiB.

If the same Collector also uses tail sampling, you may need more memory or a smaller sampling window. Tail sampling is useful, but it works by holding traces while it waits for a decision. That memory has to come from somewhere.

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 6400
    spike_limit_mib: 1280
 
  tail_sampling:
    decision_wait: 10s
    num_traces: 50000
    expected_new_traces_per_sec: 2500
 
  batch:
    timeout: 5s
 
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, tail_sampling, batch]
      exporters: [otlphttp/parseabletraces]

The processor order is still the same. The limiter comes first.

How to monitor it

Do not wait for a crash loop to learn whether the limiter is configured correctly. Watch the Collector itself.

Enable Collector telemetry and scrape it:

service:
  telemetry:
    metrics:
      level: detailed
      readers:
        - pull:
            exporter:
              prometheus:
                host: 0.0.0.0
                port: 8888

Then look for memory and refusal signals. Metric names can vary a little by Collector version and distribution, so inspect /metrics in your environment, but these are the kinds of signals to watch:

otelcol_process_runtime_heap_alloc_bytes
otelcol_process_runtime_total_alloc_bytes
otelcol_receiver_accepted_spans
otelcol_receiver_refused_spans
otelcol_receiver_accepted_log_records
otelcol_receiver_refused_log_records
otelcol_exporter_queue_size
otelcol_exporter_send_failed_spans

If refused telemetry appears during a short traffic spike and then returns to zero, the limiter is probably doing its job. If refusals continue for minutes, the Collector is not keeping up. At that point the fix is not simply "raise the limit." You need to find where memory is being held.

Start with these checks:

Is an exporter destination slow or unavailable?
Are exporter queues growing?
Is tail sampling holding too many traces?
Are batch sizes too large for the traffic pattern?
Did a new processor add significant memory overhead?
Is one receiver accepting a much larger payload than usual?

The memory limiter is an early warning system as much as a safety mechanism. When it triggers frequently, it is telling you the Collector needs attention.

What happens to data

This is the detail that matters operationally.

When the memory limiter refuses data, it returns a retryable error to the previous component in the pipeline. Receivers that understand this should retry and apply backpressure. That protects data better than accepting it into a process that is about to be killed.

But this only works if the previous component can actually retry.

If you use standard OpenTelemetry receivers, you are usually in good shape. If you use custom receivers, unusual connectors, or components that do not retry correctly, refused data can be lost. This is why the official docs recommend placing the limiter as the first processor, with a receiver before it.

The memory limiter is not a storage queue. It is a pressure valve.

Common mistakes

One common mistake is putting memory_limiter after batch. That lets the batch processor allocate memory before the limiter checks the pipeline, which weakens the main reason to have it in the first place. The limiter should sit before batch, not after it.
Another mistake is setting the hard limit too close to the container limit. The Collector process uses memory outside the Go heap too, so if the container limit is 2 GiB, setting limit_mib: 2048 is asking for trouble. Leave some headroom for normal process overhead and non-heap memory.
Teams also underestimate exporter queues. If an exporter is backed up, forced garbage collection may not free much memory because the queued data is still referenced. The limiter can slow intake, but it cannot make a slow or unhealthy downstream destination recover on its own.
Using the same limits everywhere is another easy trap. A node agent, a sidecar Collector, and a regional gateway do not behave the same way, so they should not all carry the same memory settings. The limits need to match the shape of the workload.
It is also worth treating refusals as a real signal instead of normal background noise. Short bursts can be acceptable, but continuous refusals usually mean the Collector is overloaded, undersized, or blocked downstream.

Recommended defaults

If you are rolling this out across a production fleet, use a simple policy first:

Add memory_limiter as the first processor in every logs, metrics, and traces pipeline.
Use limit_percentage in containers and limit_mib on VMs.
Start with check_interval: 1s.
Use a spike limit around 20 percent of the hard limit.
Set GOMEMLIMIT below the memory limiter hard limit.
Monitor heap usage, refused telemetry, exporter queue size, and exporter failures.
Tune based on real traffic, not only on the memory limit written in YAML.

That policy will not solve every memory issue, but it gives the Collector a predictable failure mode. Predictable is good. Predictable means you can reason about the system at 2 a.m.

Closing note

The OpenTelemetry Collector sits in a sensitive place. It is close enough to your applications to receive raw telemetry, and close enough to your backend to feel every downstream delay. That makes it powerful, but it also means it needs its own protection.

The memory_limiter processor is one of the simplest protections you can add. It does not make sizing unimportant. It does not remove the need to monitor the Collector. It simply gives the Collector a chance to push back before memory pressure becomes a crash.

That is why it belongs in production Collector configs.

OpenTelemetry Memory Limiter Processor