Custom Metrics OpenTelemetry Integration

This guide explains how to send custom metrics to Metoro using OpenTelemetry. Metoro supports the OpenTelemetry Protocol (OTLP) for metric ingestion, allowing you to send metrics from any application or service that can export OpenTelemetry metrics.

Prerequisites

A Metoro account
An application configured with OpenTelemetry

High Level Overview

The Metoro exporter running in each cluster accepts metrics over OTLP/HTTP using protobuf or JSON payloads. The standard environment-variable configuration below uses the http/protobuf protocol.

Endpoint Configuration

Use the following OTLP base URL when your SDK or collector automatically appends the signal-specific path:

http://metoro-exporter.metoro.svc.cluster.local/api/v1/custom/otel

Alternatively, use the full metrics endpoint when configuring a signal-specific URL:

http://metoro-exporter.metoro.svc.cluster.local/api/v1/custom/otel/v1/metrics

These endpoints use OTLP over HTTP and are available within the Kubernetes cluster where the Metoro exporter is installed. The older /api/v1/custom/otel/metrics endpoint remains available for compatibility, but new configurations should use the OTLP-compatible endpoints above. The Metoro exporter does not expose an OTLP/gRPC endpoint.

Authentication

No additional authentication is required when sending metrics from within the cluster to the Metoro exporter.

Reserved Names

Custom metric names must not collide with metrics generated by Metoro. Metoro drops custom OTLP metrics that use Metoro node-agent metric names, such as container_resources_cpu_usage_seconds_total or node_resources_memory_total_bytes. Names beginning with metoro_ are also reserved for Metoro-generated metrics.

Language-Specific Examples

Python

Here’s an example Python script that publishes deployment metrics to Metoro:

import sys

from kubernetes import client, config
from opentelemetry import metrics
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import (
    PeriodicExportingMetricReader,
)

# Main
if __name__ == "__main__":
    metric_reader = PeriodicExportingMetricReader(
        OTLPMetricExporter(endpoint="http://metoro-exporter.metoro.svc.cluster.local/api/v1/custom/otel/v1/metrics")
    )
    provider = MeterProvider(metric_readers=[metric_reader])

    # Sets the global default meter provider
    metrics.set_meter_provider(provider)

    # Creates a meter from the global meter provider
    meter = metrics.get_meter(__name__)

    config.load_kube_config()

    namespace = sys.argv[1]
    app = client.AppsV1Api()

    deployment_data = app.list_namespaced_deployment(namespace)
    desired_replicas = meter.create_gauge("custom_metrics.desired_replicas")
    available_replicas = meter.create_gauge("custom_metrics.available_replicas")

    for item in deployment_data.items:
        desired_replicas.set(item.status.replicas, {"deployment": item.metadata.name})
        available_replicas.set(item.status.available_replicas, {"deployment": item.metadata.name})

    # Make sure that all the metrics are exported before the application exits
    provider.force_flush()

This script:

Creates two gauge metrics:
- custom_metrics.desired_replicas - The desired number of replicas for a deployment
- custom_metrics.available_replicas - The number of available replicas for a deployment
Includes a deployment attribute with the name of the deployment
Uses the OpenTelemetry SDK to export metrics to Metoro

Go

package main

import (
    "context"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp"
    "go.opentelemetry.io/otel/sdk/metric"
    "go.opentelemetry.io/otel/sdk/resource"
    semconv "go.opentelemetry.io/otel/semconv/v1.24.0"
)

func initMetrics() (*metric.MeterProvider, error) {
    ctx := context.Background()

    exporter, err := otlpmetrichttp.New(ctx,
        otlpmetrichttp.WithEndpoint("metoro-exporter.metoro.svc.cluster.local"),
        otlpmetrichttp.WithURLPath("/api/v1/custom/otel/v1/metrics"),
        otlpmetrichttp.WithInsecure(), // Since we're in-cluster
    )
    if err != nil {
        return nil, err
    }

    resource := resource.NewWithAttributes(
        semconv.SchemaURL,
        semconv.ServiceName("your-service-name"),
    )

    meterProvider := metric.NewMeterProvider(
        metric.WithReader(metric.NewPeriodicReader(exporter)),
        metric.WithResource(resource),
    )

    return meterProvider, nil
}

Node.js

const {
  MeterProvider,
  PeriodicExportingMetricReader,
} = require('@opentelemetry/sdk-metrics');
const { OTLPMetricExporter } = require('@opentelemetry/exporter-metrics-otlp-http');
const { Resource } = require('@opentelemetry/resources');
const { SemanticResourceAttributes } = require('@opentelemetry/semantic-conventions');

const resource = new Resource({
  [SemanticResourceAttributes.SERVICE_NAME]: 'your-service-name',
});

const metricExporter = new OTLPMetricExporter({
  url: 'http://metoro-exporter.metoro.svc.cluster.local/api/v1/custom/otel/v1/metrics',
  headers: {},
});

const meterProvider = new MeterProvider({
  resource: resource,
});

meterProvider.addMetricReader(new PeriodicExportingMetricReader({
  exporter: metricExporter,
  exportIntervalMillis: 1000,
}));

Via environment variables

Most OpenTelemetry SDKs support the generic OTLP environment variables. The SDK appends /v1/metrics to the base URL:

export OTEL_EXPORTER_OTLP_ENDPOINT="http://metoro-exporter.metoro.svc.cluster.local/api/v1/custom/otel"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"

Metric Types

Metoro ingests and queries every OpenTelemetry metric type:

Counter - A value that can only increase or be reset to zero. Queried as per-bucket increases, with counter resets handled automatically. Both cumulative and delta aggregation temporality are supported.
UpDownCounter - A total that can go up and down (for example active requests or queue depth). Queried like a gauge: the values you see are the reported totals, not differences.
Gauge - A value that can arbitrarily go up and down.
Histogram - Tracks the distribution of values over time. Query with count, sum, or percentile(q) aggregations (for example percentile(0.99)).
Exponential Histogram - The high-resolution histogram encoding used by many modern SDK configurations. Query with count, sum, or percentile(q); percentiles are computed from the exponential buckets with log-scale interpolation. Values at or below zero resolve to 0 in percentile results.
Summary - Legacy pre-computed quantiles (commonly produced by Prometheus client libraries via the collector). Query with count, sum, or percentile(q); the nearest quantile reported by the producer is used, since summary quantiles cannot be interpolated or merged exactly.

Attributes and Context

When sending metrics via OpenTelemetry, you can include additional attributes that will be indexed and searchable in Metoro:

Use resource attributes to define static information about the service
Use metric attributes to include dynamic information with each metric value
Link metrics with traces using trace context propagation

Troubleshooting

If you encounter issues with OpenTelemetry metric ingestion:

Verify your endpoint URL is correct
Check your network connectivity to the Metoro OTLP endpoint
Enable debug logging in your OpenTelemetry SDK
Verify your metrics appear in the Metoro metrics view
Contact support if issues persist

Get Started

AI SRE

Concepts

Ingestion Settings

Traces

Logs

Metrics

Profiling

Kubernetes Resources

Dashboards

Infrastructure

Advisor

Alerts & Monitoring

Integrations

Uptime Monitoring

User Management

On-Premises

Administration

Custom Metrics OpenTelemetry Integration

Prerequisites

High Level Overview

Endpoint Configuration

Authentication

Reserved Names

Language-Specific Examples

Python

Go

Node.js

Via environment variables

Metric Types

Attributes and Context

Troubleshooting

Additional Resources

​Prerequisites

​High Level Overview

​Endpoint Configuration

​Authentication

​Reserved Names

​Language-Specific Examples

​Python

​Go

​Node.js

​Via environment variables

​Metric Types

​Attributes and Context

​Troubleshooting

​Additional Resources

Prerequisites

High Level Overview

Endpoint Configuration

Authentication

Reserved Names

Language-Specific Examples

Python

Go

Node.js

Via environment variables

Metric Types

Attributes and Context

Troubleshooting

Additional Resources