​
Overview

Metoro provides comprehensive tracking of Kubernetes resources, capturing every update to give you a complete history of your cluster’s state. This “time-travel” capability allows you to view your cluster’s state at any point in time, not just its current state.

​
Resource Tracking

The cluster agent watches and exports the following Kubernetes resources:

  • Pods
  • Deployments
  • StatefulSets
  • ReplicaSets
  • DaemonSets
  • Nodes
  • ConfigMaps
  • Services
  • Jobs
  • CronJobs
  • Endpoints
  • Events
  • Horizontal Pod Autoscalers
  • Ingresses
  • Namespaces

Metoro explicitly does not export secrets. If you have resources you want to exclude from exporting, you can remove them from the service account permissions given to the cluster agent. See the permissions here.

​
Resource Metrics

Kubernetes resources can be analyzed using the kubernetes_resource metric type. This allows you to:

​
Count Resources

Track the number of resources by various attributes:

# Count pods by service
kubernetes_resource{kind="Pod"}
| group_by(service_name)

# Count resources by namespace
kubernetes_resource{kind="Deployment"}
| group_by(namespace)

​
Resource Attributes

Common attributes available for filtering and grouping:

  • Environment
  • Namespace
  • Kind
  • Resource name
  • Service name
  • Additional attributes specific to resource types

​
Track Resource State

Monitor specific resource attributes over time:

# Track HPA expected replicas
kubernetes_resource{kind="HorizontalPodAutoscaler"}
| select(max(status.expectedReplicas))

# Compare current vs desired replicas
kubernetes_resource{kind="HorizontalPodAutoscaler"}
| select(
    max(status.currentReplicas),
    max(status.desiredReplicas)
)

​
JSON Path Queries

When querying Kubernetes resources metrics, you can use JSON Path expressions to extract specific values from the resource manifests. This is particularly useful when you want to aggregate or analyze specific fields within your Kubernetes resources.

​
Using JSON Path in Metrics

The JSON Path functionality allows you to:

  • Extract specific fields from Kubernetes resource manifests
  • Aggregate values from nested structures
  • Query array elements and complex objects

For example, you can:

  • Get the number of replicas from a Deployment: spec.replicas
  • Extract resource limits: spec.template.spec.containers[*].resources.limits.cpu
  • Count available ports: spec.ports[*].port

​
Syntax

The JSON Path syntax follows the standard format:

  • . is used for child operator
  • [*] is used for array iteration
  • [?()] supports filters

​
Examples

Here are some common use cases:

  1. Deployment Resource Requests and Limits
spec.template.spec.containers[*].resources.requests.memory
spec.template.spec.containers[*].resources.limits.cpu
  1. Pod Image
spec.template.spec.containers[*].image
  1. Deployment Configuration
spec.replicas
spec.strategy.type
  1. Service Configuration
$.spec.ports[*].port
$.spec.type
  1. Deployment Pod Template
$.spec.template.spec.containers[*].image
$.spec.template.spec.containers[*].name

​
Usage in Metoro

An example of a JSON Path query in Metoro:

Note: If no JSON Path is specified or the path doesn’t match any values, the metric will return null values.

​
Viewing Resources

You can view resources associated with each service in the Metoro UI:

  1. Navigate to a service page
  2. Select the Kubernetes tab
  3. View current and historical resource states

​
Use Cases

  1. Change Tracking:
  • Track configuration changes
  • Monitor scaling events
  • Debug resource modifications
  1. Resource Analysis:
  • Count resources by type
  • Monitor replica counts
  • Track resource distribution
  1. Historical Debugging:
  • View past cluster states
  • Analyze resource evolution
  • Investigate incidents

​
Best Practices

  1. Resource Monitoring:
  • Track critical resource counts
  • Monitor scaling patterns
  • Set up alerts for unexpected changes
  1. Historical Analysis:
  • Use time-travel for debugging
  • Compare states across time periods
  • Track configuration evolution
  1. Metric Usage:
  • Group resources meaningfully
  • Track relevant attributes
  • Combine with other metric types