> ## Documentation Index
> Fetch the complete documentation index at: https://metoro.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Log Filters

Metoro provides granular control over which logs are sent from your cluster for ingestion. You can configure inclusion and exclusion patterns to manage your log data effectively.

<img src="https://mintcdn.com/metoro/WUkDA8ZzUF2HFKec/images/log-settings.png?fit=max&auto=format&n=WUkDA8ZzUF2HFKec&q=85&s=7095956be5f741037284ea2cade04ed7" alt="Log ingestion settings configuration page" width="2730" height="1276" data-path="images/log-settings.png" />

## Overview

Log ingestion settings allow you to:

* Include only specific logs that match certain patterns
* Exclude sensitive or unnecessary logs from being sent out of your cluster
* Apply filters based on services, namespaces, or any log attribute
* Control log ingestion across different environments
* Pause filters temporarily without deleting their configuration

## Configuring Log Filters

You can configure log filters from the [settings page](https://us-east.metoro.io/settings) under the **Data Ingestion Settings -> Log Filters** tab.

Drop rules can also be declared as Kubernetes custom resources and managed through GitOps; see [CRD Managed Ingestion Rules](/docs/ingestion-settings/crd-managed-ingestion-rules). Filters managed that way appear here with a **Managed** badge and are read-only in the UI.

Each log filter consists of:

1. **Include/Exclude Pattern**: Choose whether to include or exclude logs that match your filter
2. **Environment Selection**: Apply the filter to specific environments or all environments
3. **Filter Type**: Filter by services, by namespaces, or build a custom filter on any log attribute
4. **Matching Criteria**: The services, namespaces, or attribute values to match, plus optional message patterns in [re2](https://github.com/google/re2/wiki/Syntax)

### Filter Types

* **Service**: Match logs from specific services (or all services), optionally combined with a message pattern.
* **Namespace**: Match logs from specific namespaces, optionally combined with a message pattern.
* **Custom**: Match logs on any attribute, the same way you search them in the logs view. A custom filter combines attribute values to match, attribute values to exclude, and message regexes to match or exclude. Any attribute you can search logs by can be used, including the detected `log_level`, Kubernetes metadata, and fields of your structured logs.

### Pausing Filters

You can pause a log filter from the filter table when you want to temporarily stop applying it without deleting the rule. Paused filters remain visible in settings and can be resumed later. While paused, the filter is not sent to exporters and does not affect which logs leave your cluster.

### Include Filters

Include filters allow you to specify which logs should be sent to Metoro. Only logs matching these patterns will be **sent out of your cluster**. This is useful when you want to:

* Only collect logs from specific services
* Only ingest logs containing certain patterns
* Limit log ingestion to specific namespaces
* Only ingest logs with specific attribute values, for example a minimum log level

### Exclude Filters

Exclude filters prevent matching logs from being sent to Metoro. This is useful for:

* Protecting sensitive data
* Reducing noise from verbose logging
* Managing ingestion costs
* Filtering out unnecessary debug logs

## Best Practices

1. **Start with Broad Patterns**: Begin with broader patterns and refine them based on your needs
2. **Test Your Patterns**: Use the regex pattern carefully to ensure you're not accidentally excluding important logs
3. **Regular Review**: Periodically review your filters to ensure they still align with your needs
4. **Adjust log alerts**: If you have log alerts set up, make sure they are up to date with your log filters

## Examples

Here are some common use cases for log filters:

1. **Exclude Sensitive Data**

```
Type: Exclude
Environment: All Environments
Filter Type: Service
Services: /k8s/authorization/auth-service
Pattern: password=.*
```

This filter will exclude logs containing the pattern `password=.*` from the `auth-service` service in `authorization` namespace in all environments.

2. **Include Only Error Logs**

```
Type: Include
Environment: All Environments
Pattern: .*ERROR.*|.*FATAL.*
Filter Type: Namespace
Namespaces: namespaceX
```

This filter will include logs containing the patterns `ERROR` or `FATAL` from all services in the `namespaceX` namespace in all environments.

3. **Exclude Debug Logs**

```
Type: Exclude
Environment: dev
Pattern: .*DEBUG.*
Filter Type: Service
Services: All Services
```

This filter will exclude logs containing the pattern `DEBUG` from all services in the `dev` environment.

4. **Exclude by Attribute (Custom Filter)**

```
Type: Exclude
Environment: All Environments
Filter Type: Custom
Attribute Filters: log_level=debug
```

This filter will exclude all logs whose detected log level is `debug`, regardless of how the message is formatted. Custom filters can match on any log attribute and can combine attribute matches, attribute exclusions, and message regexes in one rule.
