<p>We have a Google Coud Platform project where we use a cluster to deploy our apps, we have noticed on our billing this month that Stackdriver Logging increased too much so we started to create exclusions.</p>

<p>We excluded logs from <code>resource.type="container"</code> without any problem,
<strong>but we are not able to do it from</strong> <code>resource.type="k8s_cluster"</code></p>

<p><em>filter to create our desired exclusion:</em></p>

<pre><code>resource.type="k8s_cluster"
protoPayload.serviceName="k8s.io"
protoPayload.resourceName="app.k8s.io/v1beta1/namespaces/default/applications/prometheus-1"
</code></pre>

<p>A curious thing is that Stackdriver Monitoring doesn't detect any data <em>(0B ingested)</em> from there, neither from our gcp-project (<a href="https://console.cloud.google.com/logs/usage?project=xxxx" rel="nofollow noreferrer">https://console.cloud.google.com/logs/usage?project=xxxx</a>) unless we filter it using the logs viewer, then we can see the logs.</p>

<p>Grafana is able to detect the data with Stackdriver Datasource, you can see it <a href="https://i.stack.imgur.com/ielfr.png" rel="nofollow noreferrer">here</a></p>

Kitsuness
<p>The filter for <code>resource.type="k8s_cluster"</code> contains Audit Logs for <a href="https://cloud.google.com/logging/docs/audit/#system-event" rel="nofollow noreferrer">System Events</a> that are always enabled, hence, they won't be excluded from Stackdriver Logging unlike <code>resource.type="container"</code> that only has container stdout and stderr logging.</p>

<p>Furthermore, you can determine which logs belong to the Audit Logs stream by adding the '@type' condition to your Stackdriver filter:</p>

<pre><code>resource.type="k8s_cluster"
protoPayload.serviceName="k8s.io"
protoPayload.resourceName="app.k8s.io/v1beta1/namespaces/default/applications/prometheus-1"
protoPayload."@type"="type.googleapis.com/google.cloud.audit.AuditLog"
</code></pre>

<p>Finally, Grafana is able to display these log-based metrics since they're exported into it via Datasource instead of going into Stackdriver.</p>
