<p>We have a Kubernetes 1.1 cluster on AWS provisioned using <code>kube-up.sh</code>.</p>

<p>Part of the base installation includes <code>fluentd-elastisearch</code>. We want to uninstall it. Specifically, we have been unsuccessful in removing the static pods running one-per-node.</p>

<p>We do not use the Kubernetes-hosted <code>fluentd-elastisearch</code>, but instead use an externally hosted instance. As far as I can tell, <code>fluentd-elastisearch</code> is not required to run Kubernetes, and so I have been <em>trying</em> to remove it from our cluster.</p>

<p>There seem to be two parts to the elastisearch setup.
The first is the <code>addon</code> defined on the master in <code>/etc/kubernetes/addons/fluentd-elasticsearch</code>. We moved this file out of the addons directory and manually deleted the associated Replication Controllers.</p>

<p>This leaves the static pods:</p>

<pre><code>kube-ac --namespace=kube-system get pods
NAME                                                             READY     STATUS    RESTARTS   AGE
fluentd-elasticsearch-ip-10-0-5-105.us-west-2.compute.internal   1/1       Running   1          6d
fluentd-elasticsearch-ip-10-0-5-124.us-west-2.compute.internal   1/1       Running   0          6d
fluentd-elasticsearch-ip-10-0-5-180.us-west-2.compute.internal   1/1       Running   0          6d
fluentd-elasticsearch-ip-10-0-5-231.us-west-2.compute.internal   1/1       Running   0          6d
</code></pre>

<p>We believe the static pods are launched on each node due to the presence on each node of <code>/etc/kubernetes/manifests/fluentd-es.yaml</code>.</p>

<p>This file appears to be placed by <code>salt</code> configuration <code>/srv/pillar/cluster-params.sls</code> which contains <code>enable_node_logging: 'true'</code>.</p>

<p>We flipped the flag to <code>'false'</code>, killed the existing nodes, allowing new ones be provisioned via the Auto Scaling Group. Unfortunately the newly spawned hosts still have the static <code>fluentd-elasticsearch</code> pods.</p>

<p>There are a couple of other possible files we think may be involved, on the master host:</p>

<pre><code>/var/cache/kubernetes-install/kubernetes/saltbase/salt/fluentd-es/fluentd-es.yaml
/var/cache/salt/minion/files/base/fluentd-es/fluentd-es.yaml
</code></pre>

<p>We are hitting a wall with our lack of <code>salt</code> experience. Pointers most welcome.</p>

<p>I believe we have working steps to remove <strong>fluentd</strong> from a cluster which <em>already</em> has it installed.</p>

<ol>
<li>Delete <strong>fluentd-elastisearch</strong> addon (on <strong>master</strong>)

<ul>
<li><code>rm</code> (or <code>mv</code>) <code>/etc/kubernetes/addons/fluentd-elasticsearch/</code></li>
<li>Delete remnant ReplicationControllers: 

<blockquote>
  <p>kubectl  --namespace=kube-system delete rc elasticsearch-logging-v1 kibana-logging-v1</p>
</blockquote></li>
</ul></li>
<li>Disable logging in salt-stack (on <strong>master</strong>). This will prevent nodes spawned in the future from having the <strong>fluentd static pod</strong>.

<ul>
<li>In <code>/srv/pillar/cluster-params.sls</code> change existing settings to 

<blockquote>
  <p>enable_node_logging: 'false'</p>
  
  <p>logging_destination: 'none'</p>
</blockquote></li>
<li><code>salt '*' saltutil.clear_cache</code></li>
<li><code>salt '*' saltutil.sync_all</code></li>
</ul></li>
<li>On existing nodes, manually remove the <strong>fluentd static pod</strong>

<blockquote>
  <p>rm /etc/kubernetes/manifests/fluentd-es.yaml</p>
</blockquote></li>
</ol>
