Instrument a Java container for Datadog

This guide will help you instrument Java applications running as containers in Amazon ECS, enabling generation and collection of logs, metrics, and traces.

Things to consider

Recommended practice

Enable auto-instrumentation to automatically capture useful telemetry from HTTP requests, database queries, external service calls, and application logs without having to make any application code changes.

Before you begin

Make sure that you have:

Completed the getting started guide to set up basic Datadog integration.
A Java application deployed using the app template

Step 1: Enable telemetry collection and auto-instrumentation

Auto-instrumentation provides logs, metrics, and traces for your Java application without requiring any code changes.

Update to the newest version of the app template for the application you want to instrument.

Edit your package-config.yml file to enable telemetry collection:

TelemetryCollection:
  Enable: true
  AutoInstrumentation:
    Enable: true # You can disable this if you are already bundling the Java Agent with your application
    Runtime: java # This is the only supported runtime for auto-instrumentation

Apply the configuration:
```
ok pkg install
```
Apply the Terraform configuration:
```
terraform plan
terraform apply
```

Step 2: Customize OpenTelemetry configuration (optional)

You can customize the OpenTelemetry collector configuration by using the otel-config-generator module. This allows you to add custom processors, modify sampling policies, or filter specific metrics.

Example: Filter out specific metrics

stacks/dev/app-example/_gp_telemetry_collection_override.tf

module "otel_config" {
  # Add a custom processor to drop specific metrics
  additional_processors = {
    "filter/drop_metrics" = {
      metrics = {
        exclude = {
          match_type = "strict"
          metric_names = [
            "http.client.request.body.size"
          ]
        }
      }
    }
  }

  # Use the custom processor in the metrics pipeline
  pipeline_processors = {
    "metrics" = {
      processors = ["filter/drop_metrics"]
    }
  }
}

Example: Only keep traces from a specific endpoint

stacks/dev/app-example/_gp_telemetry_collection_override.tf

module "otel_config" {
  # Configure tail sampling to only keep traces from a specific endpoint
  tail_sampling_config = {
    additional_policies = [
      {
        name = "drop-everything-but-one-route"
        type = "drop"
        drop = {
          drop_sub_policy = [
            {
              name = "inverted-match"
              type = "string_attribute"
              string_attribute = {
                key                    = "http.route"
                values                 = ["^/my-route$"]
                enabled_regex_matching = true
                invert_match           = true
              }
            }
          ]
        }
      }
    ]
  }
}

Example: Bring your own OTel Collector configuration

Warning

When you bring your own OTel configuration, you will lose useful behavior and telemetry tags that the Golden Path provides out-of-the-box. Consider using the otel-config-generator when possible to extend the default configuration rather than replacing it completely, or ask Kjøremiljø to support your use case.

stacks/dev/app-example/config_override.tf

locals {
  otel_config = yamlencode(file("my-config.yml"))
}

For advanced configuration examples, check out the otel-config-generator examples.

Step 3: Add manual instrumentation (optional)

Auto-instrumentation provides basic telemetry, but custom business metrics require manual instrumentation with the OpenTelemetry SDK. Add the API dependency and use @WithSpan annotations or the API directly.

For detailed instructions on manual instrumentation, see the OpenTelemetry Java documentation.

Step 4: Verify instrumentation

Check Datadog for incoming traces, metrics, and logs. Use the env (like pirates-dev) and service (like too-tikki) tags in Datadog to filter the results.