<p>I have a Elastic search setup.</p>

<p>I need to deploy the same in kubernetes</p>

<p>Purpose : Automatically scale up the ES server with all its data to 3 instances if the CPU/RAM reaches 90% and scale it down to one instance with all its data</p>

<p>Using <code>ReplicationController</code> scaling  can be possible</p>

<p>I need , How to setup ES with all it's current data in kubernetes</p>

<p>Can anyone help me on this out?</p>

<p>You will want <a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/" rel="nofollow noreferrer"><code>StatefulSet</code></a>s (<a href="https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/" rel="nofollow noreferrer">tutorial</a>) for part of that problem (they honor <code>kubectl scale sts --replicas=$foo</code>, just like an <code>rc</code>)</p>

<p>Having members with predictable hostnames (as a <code>StatefulSet</code> will do) makes the <code>discovery.zen.ping.hostnames</code> a ton easier to configure -- even if there is only one ES node actually alive at the time (that is: you can specify <code>my-es-0.my-es.my-ns.svc.cluster.local,my-es-1.my-es.my-ns.svc.cluster.local,etc</code> in the <code>zen</code> list, and the first ES node will whine, but the 2nd and upward will benefit from the actual <em>discovery</em> part)</p>

<blockquote>
  <p>scale it down to one instance with all its data</p>
</blockquote>

<p>Good luck with that; you'd have to re-balance the shards of every index with each resizing operation, since ES wants to distribute the load out across all members of its cluster -- the exact opposite of your consolidation objective.</p>

<p>I'd bet you could make a custom Horizontal Pod Autoscaler implementation that waited until the rebalancing operations had completed before destroying the Pods, but I'm not aware right now of any out-of-the-box mechanism for doing that.</p>
