<p>A while back a <code>GKE</code> cluster got created which came with a <code>daemonset</code> of:</p>

<pre><code>kubectl get daemonsets --all-namespaces
...
kube-system   prometheus-to-sd           6         6         6       3            6           beta.kubernetes.io/os=linux                                                355d
</code></pre>

<p>Can I delete this <code>daemonset</code> without issue? 
What is it being used for?
What functionality would I be losing without it?</p>

<h2><strong>TL;DR</strong></h2>

<p><strong>Even if you delete it, it will be back.</strong></p>

<hr>

<h2>A little bit more explanation</h2>

<p>Citing explanation by user @Yasen what <code>prometheus-to-sd</code> is: </p>

<blockquote>
  <p>prometheus-to-sd is a simple component that can scrape metrics stored in <a href="https://prometheus.io/docs/instrumenting/exposition_formats/" rel="nofollow noreferrer">prometheus text format</a> from one or multiple components and push them to the Stackdriver. Main requirement: k8s cluster should run on GCE or GKE.</p>
  
  <p><a href="https://github.com/GoogleCloudPlatform/k8s-stackdriver/tree/master/prometheus-to-sd" rel="nofollow noreferrer">Github.com: Prometheus-to-sd</a></p>
</blockquote>

<p>Assuming that the command deleting this daemonset will be:</p>

<p><code>$ kubectl delete daemonset prometheus-to-sd --namespace=kube-system</code> </p>

<p><strong>Executing this command will indeed delete the daemonset but it will be back after a while.</strong></p>

<p><code>prometheus-to-sd</code> daemonset is managed by <strong>Addon-Manager</strong> which will recreate deleted daemonset back to original state. </p>

<p>Below is the part of the <code>prometheus-to-sd</code> daemonset <code>YAML</code> definition which states that this daemonset is managed by <code>addonmanager</code>: </p>

<pre><code>  labels:
    addonmanager.kubernetes.io/mode: Reconcile
</code></pre>

<p>You can read more about it by following: <a href="https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/addon-manager" rel="nofollow noreferrer">Github.com: Kubernetes: addon-manager</a></p>

<hr>

<p>Deleting this daemonset is strictly connected to the monitoring/logging solution you are using with your <code>GKE</code> cluster. There are 2 options: </p>

<ul>
<li>Stackdriver logging/monitoring</li>
<li>Legacy logging/monitoring </li>
</ul>

<h3>Stackdriver logging/monitoring</h3>

<p>You need to completely disable logging and monitoring of your <code>GKE</code> cluster to delete this daemonset. </p>

<p>You can do it by following a path: </p>

<p><code>GCP -&gt; Kubernetes Engine -&gt; Cluster -&gt; Edit -&gt; Kubernetes Engine Monitoring -&gt; Set to disabled</code>. </p>

<p><a href="https://i.stack.imgur.com/q28q2.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/q28q2.png" alt="Disabling Stackdriver"></a></p>

<h3>Legacy logging/monitoring</h3>

<p>If you are using a legacy solution which is available to <code>GKE</code> version <code>1.14</code>, you need to disable the option of <code>Legacy Stackdriver Monitoring</code> by following the same path as above. </p>

<p><a href="https://i.stack.imgur.com/GHKed.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/GHKed.png" alt="Disabling Legacy"></a></p>

<p>Let me know if you have any questions in that. </p>
