Trace Redaction

​

How Trace Redaction Works

Metoro provides the ability to redact sensitive information from your traces before they are stored. This is particularly useful for protecting sensitive data like:

• User IDs in URLs
• Account numbers in paths
• Authentication tokens
• Personal information in query parameters

The redaction process follows these steps:

1. Pattern Matching: Define regex patterns in re2 format to match sensitive information in URLs and paths
2. Service-Specific Rules: Apply redaction rules to specific services or globally across all services
3. Replacement: Replace matched patterns with custom text (e.g., replace user IDs with “[REDACTED]”)

Redaction is applied to both:

• http.url attributes (full URLs)
• http.path attributes (path components)

​

Configuration

Redaction rules can be configured in the Settings page. To add a new redaction rule:

1. Navigate to Settings
2. Select the Data Ingestion Settings tab
3. Click on Trace Redactions in the left sidebar
4. Click the Add Rule button in the top right corner

The same view will show you all your current redaction rules, allowing you to manage and remove existing rules as needed.

Each rule consists of:

• Environments (default is all environments)
• Service names (default is all services)
• Pattern (regex to match text to redact)
• Replacement text (text to replace matched text with)

For example, to redact user IDs from a URL path:

• Pattern: /users/(\d+)
• Replacement: /users/[REDACTED_USER_ID]

You can test your regex rules in the modal above by clicking “Test Regex” button, or use an online regex tester.

​

Reliability Features

The redaction system includes several reliability features:

• Invalid Pattern Handling: If a regex pattern is invalid, the original URL is preserved
• Synchronization: Background process keeps redaction rules up-to-date
• Environment Isolation: Rules can be scoped to specific environments

​

Order of Operations

1. Traces are collected from your services by the metoro-node-agent component running in your cluster
2. The collected traces are sent to the metoro-exporter component, also running in your cluster
3. In the metoro-exporter:
   • Redaction rules are validated and compiled
   • Rules are applied based on service name and environment matches
   • Both client and server service names are checked against the rules
   • Matching traces are redacted according to the rules
4. Only after redaction are the traces sent to Metoro’s observability backend for storage and display

The metoro-exporter automatically resyncs the redaction rules every minute to ensure it has the latest configuration. This means that any changes you make to redaction rules will be picked up automatically within a minute, without requiring a manual restart.

This architecture ensures that sensitive information never leaves your cluster and is redacted before it reaches any external system, including Metoro’s observability backend. The redaction happens as close to the source as possible, providing an additional layer of security for your sensitive data.

​

Important Note About Rule Order

The order in which redaction rules are applied is non-deterministic. Therefore, you should design your redaction rules to be independent of each other. For example:

❌ Incorrect approach (rules depending on order):

Rule 1: Pattern: `/users/(\d+)/details` → `/users/[ID]/details`
Rule 2: Pattern: `/users/[ID]` → `/[REDACTED]`

✅ Correct approach (independent rules):

Rule 1: Pattern: `/users/(\d+)/details` → `/users/[REDACTED_ID]/details`
Rule 2: Pattern: `/users/(\d+)` → `/users/[REDACTED_ID]`

Each rule should be self-contained and not rely on the output of other rules. This ensures consistent redaction regardless of the order of application.

​

Troubleshooting

If your trace redaction rules are not working as expected, here are some common issues and solutions:

1. Invalid Regex Patterns

   • Make sure your regex patterns are valid re2 expressions
   • Invalid regex patterns will not compile and therefore will not be applied to trace endpoints
   • Use the “Test Regex” feature in the UI or an online regex tester to validate your patterns

2. Attribute Limitations

   • Currently, redactions are only applied to http.path and http.url attributes
   • If you need to redact data from different attributes, please contact the Metoro team for support.