<p>I have a kubernetes cluster that's running datadog and some microservices. Each microservice makes healthchecks every 5 seconds to make sure the service is up and running. I want to exclude these healthcheck logs from being ingested into Datadog.</p>
<p>I think I need to use <code>log_processing_rules</code> and I've tried that but the healthcheck logs are still making it into the logs section of Datadog. My current Deployment looks like this:</p>
<pre><code>apiVersion: apps/v1
kind: Deployment
[ ... SNIP ... ]
spec:
  replicas: 2
  selector:
    matchLabels:
      app: my-service
  template:
    metadata:
      labels:
        app: my-service
        version: &quot;fac8fb13&quot;
      annotations:
        rollme: &quot;IO2ad&quot;
        tags.datadoghq.com/env: development
        tags.datadoghq.com/version: &quot;fac8fb13&quot;
        tags.datadoghq.com/service: my-service
        tags.datadoghq.com/my-service.logs: |
          [{
            &quot;source&quot;: my-service,
            &quot;service&quot;: my-service,
            &quot;log_processing_rules&quot;: [
              {
                &quot;type&quot;: &quot;exclude_at_match&quot;,
                &quot;name&quot;: &quot;exclude_healthcheck_logs&quot;,
                &quot;pattern&quot;: &quot;\&quot;RequestPath\&quot;: \&quot;\/health\&quot;&quot;
              }
            ]
          }]
</code></pre>
<p>and the logs coming out of the kubernetes pod:</p>
<pre><code>$ kubectl logs my-service-pod

{
  &quot;@t&quot;: &quot;2022-01-07T19:13:05.3134483Z&quot;,
  &quot;@m&quot;: &quot;Request finished HTTP/1.1 GET http://10.64.0.80:5000/health - - - 200 - text/plain 7.5992ms&quot;,
  &quot;@i&quot;: &quot;REDACTED&quot;,
  &quot;ElapsedMilliseconds&quot;: 7.5992,
  &quot;StatusCode&quot;: 200,
  &quot;ContentType&quot;: &quot;text/plain&quot;,
  &quot;ContentLength&quot;: null,
  &quot;Protocol&quot;: &quot;HTTP/1.1&quot;,
  &quot;Method&quot;: &quot;GET&quot;,
  &quot;Scheme&quot;: &quot;http&quot;,
  &quot;Host&quot;: &quot;10.64.0.80:5000&quot;,
  &quot;PathBase&quot;: &quot;&quot;,
  &quot;Path&quot;: &quot;/health&quot;,
  &quot;QueryString&quot;: &quot;&quot;,
  &quot;HostingRequestFinishedLog&quot;: &quot;Request finished HTTP/1.1 GET http://10.64.0.80:5000/health - - - 200 - text/plain 7.5992ms&quot;,
  &quot;EventId&quot;: {
    &quot;Id&quot;: 2,
    &quot;Name&quot;: &quot;RequestFinished&quot;
  },
  &quot;SourceContext&quot;: &quot;Microsoft.AspNetCore.Hosting.Diagnostics&quot;,
  &quot;RequestId&quot;: &quot;REDACTED&quot;,
  &quot;RequestPath&quot;: &quot;/health&quot;,
  &quot;ConnectionId&quot;: &quot;REDACTED&quot;,
  &quot;dd_service&quot;: &quot;my-service&quot;,
  &quot;dd_version&quot;: &quot;54aae2b5&quot;,
  &quot;dd_env&quot;: &quot;development&quot;,
  &quot;dd_trace_id&quot;: &quot;REDACTED&quot;,
  &quot;dd_span_id&quot;: &quot;REDACTED&quot;
}
</code></pre>
<p>EDIT: Removed 2nd element of the <code>log_processing_rules</code> array above as I've tried with 1 and 2 elements in the rules array.</p>
<p>EDIT2: I've also tried changing <code>log_processing_rules</code> type to INCLUDE at match in an attempt to figure this out:</p>
<pre><code>&quot;log_processing_rules&quot;: [
  {
    &quot;type&quot;: &quot;include_at_match&quot;,
    &quot;name&quot;: &quot;testing_include_at_match&quot;,
    &quot;pattern&quot;: &quot;somepath&quot;
  }
]
</code></pre>
<p>and I'm still getting the health logs in Datadog (in theory I should not as <code>/health</code> is not part of the matching pattern)</p>

<p>All of these answers are correct in their own ways, but my specific issue was that the Datadog annotations for the <code>source</code> and <code>service</code> were not properly quoted:</p>
<pre><code>        ad.datadoghq.com/my-service.logs: |
          [{
            &quot;source&quot;: &quot;my-service&quot;,    # Needs Quotes
            &quot;service&quot;: &quot;my-service&quot;,   # Needs Quotes
            &quot;log_processing_rules&quot;: [
              {
                &quot;type&quot;: &quot;exclude_at_match&quot;,
                &quot;name&quot;: &quot;exclude_healthcheck_logs&quot;,
                &quot;pattern&quot;: &quot;\&quot;RequestPath\&quot;: \&quot;\/health\&quot;&quot;
              }
            ]
          }]
</code></pre>
