Skip to content

OTel Collector exporting all specify metric and logs excepting one log file data #13886

@FarzanaJade

Description

@FarzanaJade

Description
I am seeing unusual behavior in the OpenTelemetry Collector. Metrics and most logs are being exported correctly, but one of the log file is not exported at the specific scheduled interval. The issue only affects this particular log file(ping_results.log)even though the batch processor is not set/configured the logs are going as batch at a time for particular log file— other logs and metrics continue to flow normally.

Collector Version
version 0.125.0

Platform: Linux server

Config File Snippet

receivers:
   filelog:
    include: [ /var/log/remote_ip_logs/*.log ]
    
exporters:
  debug:
      verbosity: normal
    endpoint: "<openobserve-endpoint>"
    headers:
       Authorization: "<key>"
       stream-name: default

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [otlp,debug]
    logs:
      receivers: [ filelog ]
      processors: [filter/security_events ]
      exporters: [ debug, otlphttp/openobserve ]

Expected Behavior
All configured log files should be exported at the specified intervals.

Actual Behavior
Metrics and most logs are exported successfully, but one log file intermittently fails to export at the correct interval and send all logs at a time after the time passed but again it's sending all logs in batch even though i didn't set batch processor.

Steps to Reproduce

-Configure collector with the filelog receiver as shown above.
-Generate logs in /var/log/remote_ip_logs/.
-Observe that one log file is skipped at certain intervals.

Additional Info

Restarting the collector does not consistently resolve the issue.
Other logs from the same receiver path are exported without issues.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions