Skip to content

Creating metrics from log events using filters

A log metric filter is a rule that you define for a CloudWatch Logs log group. When you create a log metric filter, you specify a filter pattern and an AWS CloudWatch metric. The filter pattern is used to extract metric data from the log events in the log group. The metric data is stored in the specified CloudWatch metric.

A CloudWatch metric can subsequently be used to create alarms and dashboards using either Grafana or CloudWatch.

For more information about log metric filters, see Working with Metric Filters in the AWS CloudWatch Logs User Guide.

This example shows how to create a log metric filter using Terraform and terraform/templates/

Step 1: Download and configure the log metric filter template

  1. Locate the stack directory in your IaC repository for the environment you want to work with. For example, if you want to work with the dev environment and you have followed the Golden Path step-by-step guide, the stack directory is dev/infra.

  2. Download the log_metric_filter template from golden-path-iac repository

    ok get-template log_metric_filter

Step 2: Edit the template

  1. Open the file in your preferred editor.

  2. Edit the variables inside the module block. The table blow shows the variables that you can edit.

    Variable Description Example value
    name The name of the metric filter. my-application-logs-error-metric
    log_group_name The name of the log group to create the metric filter for. my-application-logs
    pattern The filter pattern to use. For more information, see Filter and Pattern Syntax. ERROR
    metric_transformation_name The name of the CloudWatch metric. MyApplication
    metric_transformation_namespace The namespace of the CloudWatch metric. ErrorCount

Step 3: Deploy the template

  1. Run terraform init to initialize the Terraform configuration.

  2. Run terraform plan to see what changes will be made.

  3. Run terraform apply to apply the changes.

Step 4: Verify deployment

Verify that the new Metric filter is aggregating and filtering logs by creating a log in your log group matching the filter patter. For example, if you have created a filter for ERROR logs, you can make your application log an error message and check if the metric filter is aggregating the log.

  "level": "ERROR",
  "message": "Something went wrong"

The metric filter should aggregate the log and create a new metric in CloudWatch. You can verify this by going to the CloudWatch console and checking the metrics for the log group has increased by 1.