Log Transformations

Documentation Index

Fetch the complete documentation index at: https://metoro.io/docs/llms.txt

Use this file to discover all available pages before exploring further.

​

Creating Log Transformations

1. Navigate to the Logs page in the Metoro UI.
2. In the search bar, click the Transformations button (shown below) on the most left side.

3. Log Transformations input pills will appear, under the search bar (see below).

4. Fill in the details:
   • inputAttribute: Specify the log attribute you want to transform. This can be a standard log attribute (like message, kubernetes.container.name, etc.) or a custom attribute you have created from previous transformations.
   • regex: Enter a regular expression in re2 form to match the log messages or attributes you want to transform. Use capturing groups to extract specific parts of the log.
   • outputAttribute: Enter the name of the new attribute you want to create from the transformation. This custom attribute will appear in custom columns and can be used the same way as any other log attribute for filtering and searching.

5. To see the outputAttribute as a column in the logs table, click the Columns button (shown below) on the most right side of the search bar.

6. Ta da! Your custom attribute is now available for use in log searches and saved searches.

​

More Examples and Features

1. You can use multiple transformations in a single search. Each transformation will create a new custom attribute that can be used independently.
2. You can chain transformations by using the output of one transformation as the input for another. This allows you to build complex parsing logic.
3. You can use transformations to create alerts.
4. You can use transformations to plot graphs.
5. You can use MetoroQL syntax if you prefer to write transformations in a more structured query format. For example:
   count(transformAttribute(transformAttribute(logs{callerFunc=~".+"}, "callerFunc", extract("caller", ".*/(.*):.*")), "status", extract("message", "request(.*)")))
   

​

MetoroQL Syntax for Log Transformations

count(transformAttribute(logs, <outputAttributeName>, extract(<inputAttributeName>, <regex>)))

• You can use a nested transformAttribute, like this:

count(transformAttribute(transformAttribute(logs, <outputAttributeName1>, extract(<inputAttributeName1>, <regex1>)), <outputAttributeName2>, extract(<inputAttributeName2>, <regex2>)))

• You can group by the custom attribute you created using transformAttribute in your MetoroQL queries. For example:

count(transformAttribute(logs, "callerFunc", extract("message", "caller=(.*)"))) by (callerFunc)

• You can use the output attribute you created to filter logs in your MetoroQL queries. For example:

count(transformAttribute(logs{callerFunc=~".+"}, "callerFunc", extract("message", "caller=(.*)")))

Easily swap between MetoroQL and UI by using the < > icon on the right side of the search bar.