<p>I want to setup elastic stack (elastic search, logstash, beats and kibana) for monitoring my kubernetes cluster which is running on on-prem bare metals. I need some recommendations on the following 2 approaches, like which one would be more robust,fault-tolerant and of production grade. Let's say I have a K8 cluster named as K8-abc.</p>

<p>Approach 1- Will be it be good to setup the elastic stack outside the kubernetes cluster? </p>

<p>In this approach, all the logs from pods running in kube-system namespace and user-defined namespaces would be fetched by beats(running on K8-abc) and put into into the ES Cluster which is configured on Linux Bare Metals via Logstash (which is also running on VMs). And for fetching the kubernetes node logs, the beats running on respective VMs (which are participating in forming the K8-abc) would fetch the logs and put it into the ES Cluster which is configured on VMs. The thing to note here is the VMs used for forming the ES Cluster are not the part of the K8-abc. </p>

<p>Approach 2- Will be it be good to setup the elastic stack on the kubernetes cluster k8-abc itself?</p>

<p>In this approach, all the logs from pods running in kube-system namespace and user-defined namespaces would be send to Elastic search cluster configured on the K8-abc via logstash and beats (both running on K8-abc). For fetching the K8-abc node logs, the beats running on VMs (which are participating in forming the K8-abc) would put the logs into ES running on K8-abc via logstash which is running on k8-abc.</p>

<p>Can some one help me in evaluating the pros and cons of the before mentioned two approaches? It will be helpful even if the relevant links to blogs and case studies is provided.</p>

<p>I would be more inclined to the <strong>second solution</strong>. It has many advantages over the first one however it may seem more complex as it comes to the initial setup. You can actually ask similar question when it comes to migrate any other type of workload to <strong>Kubernetes</strong>. It has many advantages over VM. To name just a few:</p>

<ol>
<li><strong><a href="https://www.stratoscale.com/blog/kubernetes/auto-healing-containers-kubernetes/" rel="nofollow noreferrer">self-healing cluster</a></strong>, </li>
<li><strong><a href="https://kubernetes.io/docs/concepts/services-networking/service/#discovering-services" rel="nofollow noreferrer">service discovery</a></strong> and integrated <a href="https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/" rel="nofollow noreferrer"><strong>load balancing</strong></a>,</li>
<li>Such solution is much easier to scale (<a href="https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/" rel="nofollow noreferrer"><strong>HPA</strong></a>) in comparison with VMs,</li>
<li>Storage orchestration. <strong>Kubernetes</strong> allows you to automatically mount a storage system of your choice, such as local storage, public cloud providers, and <a href="https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes" rel="nofollow noreferrer">many more</a> including <strong><a href="https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/" rel="nofollow noreferrer">Dynamic Volume Provisioning</a></strong> mechanism.</li>
</ol>

<p>All the above points could be easily applied to any other workload and may bee seen as <strong>Kubernetes</strong> advantages in general so let's look why to use it for implementing <strong>Elastic Stack</strong>:</p>

<ol start="5">
<li>It looks like <strong>Elastic</strong> is actively promoting use of <strong>Kubernetes</strong> on their <a href="https://www.elastic.co/elasticsearch-kubernetes" rel="nofollow noreferrer">website</a>. See also <a href="https://www.elastic.co/blog/alpha-helm-charts-for-elasticsearch-kibana-and-cncf-membership" rel="nofollow noreferrer">this</a> article.</li>
<li>They also provide an official <strong><a href="https://github.com/elastic/helm-charts/tree/master/elasticsearch" rel="nofollow noreferrer">elasticsearch helm chart</a></strong> so it is already quite well supported by <strong>Elastic</strong>.</li>
</ol>

<p>Probably there are many other reasons in favour of <strong>Kubernetes</strong> solution I didn't mention here. <a href="https://medium.com/faun/https-medium-com-thakur-vaibhav23-ha-es-k8s-7e655c1b7b61" rel="nofollow noreferrer">Here</a> you can find a hands-on article about setting up <strong>Highly Available and Scalable Elasticsearch on Kubernetes</strong>.</p>
