​
Sending Logs with Vector

This guide demonstrates how to send logs to Metoro using Vector and the OpenTelemetry Collector. This setup provides a robust and scalable way to collect and forward logs to Metoro. For a complete working example, see our Vector Logs Example repository.
You must add a service.name attribute to your logs. This attribute is required by Metoro to properly organize and display your logs. You can set this using Vector’s transforms (shown below) or directly in your application logging configuration.

​
Architecture

The setup consists of two main components:
  1. Vector: Acts as the log collector and forwarder. In this example, it generates sample syslog messages and forwards them to the OpenTelemetry Collector.
  2. OpenTelemetry Collector: Receives logs from Vector, processes them, and forwards them to Metoro in the correct OpenTelemetry format.

​
Vector Configuration

Vector needs to be configured to forward logs to the OpenTelemetry Collector. Here’s the sink configuration: 
transforms:
  add_service_name:
    type: remap
    inputs: ["your_source_name"]
    source: |
      # Add a service name to help organize logs in Metoro
      .service.name = "your-service-name"

sinks:
  syslog_sink:
    type: socket
    inputs: ["add_service_name"]  # Use the transform output
    address: "otel-collector:1514"  # Address of your OpenTelemetry Collector
    mode: "tcp"
    encoding:
      codec: "text"

​
Vector Source Options

Vector supports many different sources for collecting logs:
  • kubernetes_logs: Collect logs from Kubernetes containers
  • file: Read logs from files
  • syslog: Accept syslog messages
  • journald: Read from systemd journal
  • And many more

​
OpenTelemetry Collector Configuration

The OpenTelemetry Collector needs to be configured to receive logs from Vector and forward them to Metoro. Here’s the configuration: 
receivers:
  # Configure the syslog receiver
  syslog:
    tcp:
      listen_address: 0.0.0.0:1514
    protocol: rfc5424

processors:
  # Batch logs for efficient processing
  batch:
    timeout: 1s
    send_batch_size: 4096

exporters:
  # Configure the Metoro exporter
    otlphttp:
      endpoint: "http://metoro-exporter.metoro.svc.cluster.local/api/v1/send/logs/otel"
      logs_endpoint: "http://metoro-exporter.metoro.svc.cluster.local/api/v1/send/logs/otel"
      insecure: true

service:
  # Set up the processing pipeline
  pipelines:
    logs:
      receivers: [syslog]
      processors: [batch]
      exporters: [otlphttp]

​
OpenTelemetry Collector Components

  1. Receivers: Configure how the collector receives data
    • In this case, we use the syslog receiver on port 1514
    • Supports RFC5424 format syslog messages
  2. Processors: Configure data processing
    • Batch processor groups logs for efficient sending
    • Adjustable timeout and batch size settings
  3. Exporters: Configure where to send the data
    • Uses OTLP HTTP protocol to send to Metoro
    • Endpoint points to your Metoro instance

​
Customization

​
Log Sources

To collect logs from your applications, configure an appropriate Vector source for your use case. See the Vector documentation for available source types and their configurations.

​
Batch Settings

Adjust the batch processor settings in the OpenTelemetry Collector for your needs: 
processors:
  batch:
    timeout: 1s  # Increase for higher latency tolerance
    send_batch_size: 4096  # Adjust based on log volume

​
Troubleshooting

Common issues and solutions:
  1. Vector can’t connect to the OpenTelemetry Collector:
    • Verify the collector address is correct
    • Check that the collector is listening on the specified port
    • Ensure network connectivity between Vector and the collector
  2. Logs not appearing in Metoro:
    • Check the OpenTelemetry Collector logs for errors
    • Verify the Metoro endpoint is correct
    • Check that logs are being received by the collector

​
Next Steps

  • Configure Vector to collect your application logs
  • Add filters and transforms in Vector to process logs
  • Set up monitoring for Vector and the OpenTelemetry Collector
  • Configure log retention and archival policies