​
Why is the trace destination showing as “Unknown External Service”?

When Metoro monitors calls from services inside your cluster to external services, it uses a combination of client-side tracing and DNS resolution to identify the destination services. Here’s how it works:

​
The Process

  1. Client-Side Tracing: Metoro monitors all calls made from inside your cluster to external services. For example, when Service A (inside your cluster) calls Service B (outside your cluster), through client-side traces, Metoro can see the IP address of the destination.

  2. Protocol-Specific Resolution:

    HTTP/HTTPS Protocols: For HTTP-based protocols, the destination hostname is present in the request headers. Metoro directly uses this hostname without needing additional DNS resolution. This provides accurate service identification for HTTP/HTTPS calls.

    Other Protocols (Redis, MongoDB, etc.): For protocols where hostname isn’t part of the protocol data, Metoro captures DNS resolution results when the connection is first established. This information is saved to a key-value store (Redis node running in your cluster). The stored mapping helps identify services in subsequent traces.

  3. IP Resolution Process: When exporters encounter a trace with an unknown IP address (external cluster IP), they attempt to resolve it by looking up the IP in the key-value store. If the IP address cannot be found in the key-value store, the service gets labeled as “External Unknown Service”.

The DNS resolution results are stored with different retention periods based on your tier:

  • Hobby tier: DNS results are stored for 1 hour
  • Scale tier: DNS results are stored for 1 week

This becomes important for long-lived connections, especially with protocols like Redis. For example, if you’re using Redis in the Hobby tier and your connection stays alive for more than an hour, subsequent traces might show “Unknown External Service”. This happens because the original DNS resolution has expired, but the connection is still active and no new DNS resolution has occurred.

If a container makes a request to an IP address without making a DNS query that resolves to that IP, it will be shown as “Unknown External Service”. Here’s an example:

import requests

# Example 1: HTTP request directly to IP
# This will show as "Unknown External Service" because no DNS query is made
response = requests.get("http://1.2.3.4:8080/api/data")

# Example 2: Using hostname
# This will show the proper service name because DNS resolution occurs
response = requests.get("http://api.example.com/api/data")

​
What are the different types of traces in Metoro?

Metoro captures two main types of traces to provide comprehensive visibility into your service communications.

​
Client-Side Traces

Client-side traces are generated by inspecting the outgoing system calls from services within your cluster. When Service A makes requests, Metoro monitors these calls at the system level to understand the destination IP address, the protocol being used, the timing of the request, and other relevant metadata.

​
Server-Side Traces

Server-side traces are generated when your services receive incoming requests. These traces provide information about the incoming request details, how your service processed the request, the response sent back, and performance metrics for the request handling. Server-side traces are particularly useful for understanding how your services handle incoming traffic.

By default, all traces you see are client-side traces when the client IP is within your cluster. However, for incoming calls from external sources (where the client IP is outside your cluster), Metoro automatically switches to using server-side traces.

​
How do I know if a trace is a client-side or server-side trace?

You can identify the type of trace by looking at the trace attributes. If the attribute metoro.is_server_span is set to true, then it’s a server-side trace. If this attribute is not present or set to false, then it’s a client-side trace.

Was this page helpful?

Service MapLogs Overview