<p>I'm migrating my cluster to GKE using autpilot mode, and I'm trying to apply fluentbit for logging (to be sent to Elasticsearch and then Kibana to be alerted on a slack channel).</p>
<p>But it seems that GKE Autopilot doesn't want me to do anything on the <code>hostPath</code> other than reading into files inside <code>/var/log</code> according to this <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview#host_options_restrictions" rel="nofollow noreferrer">documentation</a>. However Fluentbit needs to access <code>/var/lib/docker/containers</code> which is different from <code>/var/log</code> and also access to write inside <code>/var/log</code></p>
<p>Is there a way to get around this or how do you usually log in GKE Autopilot with alerts?
Experience sharing is also welcome</p>

<p>Citing the official documentation:</p>
<blockquote>
<h3>External monitoring tools</h3>
<p>Most external monitoring tools require access that is restricted. Solutions from several Google Cloud partners are available for use on Autopilot, however not all are supported, and <strong>custom monitoring tools cannot be installed on Autopilot clusters.</strong></p>
<p>-- <em><a href="https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview#external_monitoring_tools" rel="nofollow noreferrer">Cloud.google.com: Kubernetes Engine: Docs: Concepts: Autopilot overview: External monitoring tools </a></em></p>
<hr />
<h3>Host options restrictions</h3>
<p>HostPort and hostNetwork are not permitted because node management is handled by GKE. Using <a href="https://kubernetes.io/docs/concepts/storage/volumes/#hostpath" rel="nofollow noreferrer">hostPath</a> volumes in write mode is prohibited, <strong>while using hostPath volumes in read mode is allowed only for <code>/var/log/</code> path prefixes</strong>. Using <a href="https://kubernetes.io/docs/concepts/policy/pod-security-policy/#host-namespaces" rel="nofollow noreferrer">host namespaces</a> in workloads is prohibited.</p>
<p>-- <em><a href="https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview#host_options_restrictions" rel="nofollow noreferrer">Cloud.google.com: Kubernetes Engine: Docs: Concepts: Autopilot overview: Host options restrictions</a></em></p>
</blockquote>
<p>As you've already found the access to the <code>/var/lib/docker/containers</code> directory is not possible with the <code>GKE</code> in <code>Autopilot</code> mode.</p>
<p>As a <strong>workaround</strong> you could try to <strong>either</strong>:</p>
<ul>
<li>Use <code>GKE</code> cluster in <code>standard</code> mode.</li>
<li>Use <code>Cloud Operations</code> with its Slack notification channel. You can read more about this topic by following:
<ul>
<li><em><a href="https://cloud.google.com/monitoring/alerts" rel="nofollow noreferrer">Cloud.google.com: Monitoring: Alerts</a></em></li>
<li><em><a href="https://cloud.google.com/monitoring/support/notification-options#slack" rel="nofollow noreferrer">Cloud.google.com: Monitoring: Support: Notification options: Slack</a></em></li>
</ul>
</li>
</ul>
<p>I'd reckon you could also consider checking the guide for exporting logs to <code>Elasticsearch</code> from <code>Cloud Logging</code>:</p>
<ul>
<li><em><a href="https://cloud.google.com/architecture/exporting-stackdriver-logging-elasticsearch" rel="nofollow noreferrer">Cloud.google.com: Architecture: Scenarios for exporting Cloud Logging: Elasticsearch</a></em></li>
</ul>
<hr />
<p>Additional resources:</p>
<ul>
<li><em><a href="https://stackoverflow.com/a/67599710/12257134">Stackoverflow.com: Answer: Prometheus on GKE Autopilot?
</a></em></li>
</ul>
