Traces Overview

​

How Tracing Works in Metoro

Metoro uses eBPF technology at its core to provide comprehensive tracing capabilities. The Metoro node agent runs on every host in your cluster and monitors all network calls, deserializing them to inspect the underlying protocols and create traces. This means that you don’t need to instrument your code to get tracing data. You can just start using Metoro and get traces for all your services, including third party services.

​

Supported Protocols

Metoro currently supports tracing for the following protocols:

• HTTP(s)
• gRPC
• Postgres
• MySQL
• Redis
• Memcache
• MongoDB
• Kafka
• Cassandra

Every request made using these protocols from any container in your cluster is automatically recorded and sent to Metoro’s observability backend for inspection.

​

HTTPs Support

Metoro supports HTTPS tracing through eBPF instrumentation of SSL libraries (like OpenSSL and BoringSSL) within the binary itself. When calls to read or write TLS are made, Metoro inspects the unencrypted content at the kernel level, without modifying any application code.

​

Trace Attributes

Each trace is enriched with various attributes depending on the protocol. Common attributes across all protocols include:

• Client namespace
• Server namespace
• Client container ID
• Server container ID
• Client availability zone
• Server availability zone
• Server service name
• Client service name

Protocol-specific attributes are also captured. For example, HTTP traces include:

• HTTP path
• HTTP status code
• HTTP URL
• HTTP host
• HTTP flavor
• Request duration

For database protocols, Metoro decodes and records the actual queries, providing visibility into every database operation.

​

Trace Redaction

Metoro provides comprehensive trace redaction capabilities to protect sensitive information in your traces. For detailed information about configuring and managing trace redaction, see the Trace Redaction documentation.

​

The Trace View

The trace view provides a powerful interface to explore and analyze your traces across all clusters. You can filter traces using:

• Any trace attribute
• Regex search patterns
• Time-based filtering

​

Service Map Generation

Metoro automatically generates service maps based on the traced communications between services. This provides a visual representation of how your services interact with each other, making it easier to understand your application’s architecture and dependencies.

​

Bring Your Own Tracing (OpenTelemetry)

eBPF tracing is a powerful tool, but it does have some limitations, it does not support all protocols and does not currently support distributed tracing (ebpf creates individual spans but is unable to track chains of request calls currently). If you are already using OpenTelemetry, you can send your traces to Metoro by configuring the OpenTelemetry Collector to send traces to Metoro. The Metoro exporter is a fully compliant opentelemetry collector, so you can send your traces to Metoro to be queried and visualized. Follow the guide at OpenTelemetry Tracing Integration

​

RED Metrics Generation

The trace data is used to generate various RED Metrics (Rate, Errors, Duration), including:

• Request metrics
• Error metrics
• Duration metrics

These metrics are available in the Service View or in Custom Dashboards and provide valuable insights into your application’s performance and behavior.