# Forward logs from CloudWatch and S3 to Datadog

This guide shows you how to forward logs from AWS CloudWatch and S3 (such as ALB logs) to Datadog.

## Before you begin

Make sure that you have:

- Completed the [getting started guide](../initial-setup-of-your-aws-environment/set-up-datadog.md) to set up basic Datadog integration.
- A CloudWatch log group or S3 bucket with logs that you want to forward to Datadog.

## Things to consider

<!-- prettier-ignore-start -->
!!! tip "Recommended practice"
    Set up log forwarding in each stack where a log source exists rather than creating a centralized forwarding setup. This approach reduces coupling between stacks and makes it easier to manage permissions and dependencies.

!!! tip "All bucket or log group tags are forwarded to Datadog"
    As an example, if a log group or S3 bucket logically belongs to a specific application, you might want to add a `service` tag to it in AWS. The same tag then makes its way to Datadog and makes it easier to query across all relevant logs for a given application.

!!! note "ALB logs and `load-balancing-alb` template"
    If you're using the `load-balancing-alb` template, ALB log forwarding is automatically configured in newer versions. Check if you need to update your template version and apply the update (both in the ALB stack and associated `-data` stack).
<!-- prettier-ignore-end -->

## Step 1: Forward CloudWatch logs

Use the `datadog-log-subscription` module to forward CloudWatch logs. Add it to your Terraform configuration:

```hcl title="dev/stacks/example-cloudwatch-data/datadog.tf"
module "forward_to_datadog" {
  source              = "git@github.com:oslokommune/golden-path-iac//terraform/modules/datadog-log-subscription?ref=datadog-log-subscription-v0.1.2"
  environment         = local.environment
  cloudwatch_sources  = {
    example = { # (1)!
      log_group_name  = aws_cloudwatch_log_group.application.name
    }
  }
}
```

1. Any unique name works. Pick something meaningful for future readers — for example, the app name or load balancer name.

## Step 2: Forward S3 logs

For S3, add S3 sources to the same module:

```hcl title="dev/stacks/example-s3-data/datadog.tf"
module "forward_to_datadog" {
  source              = "git@github.com:oslokommune/golden-path-iac//terraform/modules/datadog-log-subscription?ref=datadog-log-subscription-v0.1.2"
  environment         = local.environment
  s3_sources = {
    example = { # (1)!
      bucket_name = aws_s3_bucket.logs.name
    }
  }
}
```

1. Any unique name works. Pick something meaningful for future readers — for example, the app name or load balancer name.

## Step 3: Apply the configuration

Run Terraform to create the log forwarding resources:

```sh
terraform plan
terraform apply
```

## Step 4: Verify log forwarding

Check the [Log Explorer in Datadog](https://app.datadoghq.eu/logs) for incoming logs. Use the `env` and `source` tags to filter results:

```text
env:pirates-dev source:(elb OR cloudwatch)
```

<!-- prettier-ignore-start -->
!!! note "Existing log entries aren't forwarded"
    Only log entries created in CloudWatch or S3 _after_ enabling forwarding are sent to Datadog.

!!! warning "Timestamp format requirements"
    If your logs contain a `timestamp` attribute, it must use ISO8601, UNIX (milliseconds EPOCH), or RFC3164 format. Unsupported formats cause logs to be filtered by Datadog. See [Datadog preprocessing documentation](https://docs.datadoghq.com/logs/log_configuration/pipelines/?tab=date#preprocessing) for details.
<!-- prettier-ignore-end -->