<p>Is it possible to deploy a self-healing and scaling Redis-like key-value store that I can run in Kubernetes (or Swarm or any other automated cloud env)?</p>

<p>The challenges I found with Redis:</p>

<ol>
<li>You need to create a cluster manually with <code>redis-trib</code></li>
<li>New nodes need to be added to the cluster explicitly</li>
<li>Nodes need to be removed explicitly</li>
<li>Nodes do not replicate data in their shards peer to peer, but rather use a master-slave model</li>
</ol>

<p>The above means that the following scenarios all will fail. I have a simple 3-master and 3-slave cluster. "Master A" fails, leading Kubernetes to start a new "Master A" in &lt;1 second:</p>

<ul>
<li>new "Master A" has no knowledge of the cluster and will not join</li>
<li>"Slave A", which had copies of the data, now syncs from the new "Master A", and loses all data, defeating the purpose of slave replica</li>
<li>In completely new startup, "Master A" might start before "Master B", is initialized (assuming I even can automate the cluster init), before "Master B" is ready, and thus "Master B" never really joins the cluster</li>
</ul>

<p>Questions:</p>

<ol>
<li>Is there a way to automate Redis cluster init and sync in a non-predictable, non-persistent storage without <em>any</em> human interaction? </li>
<li>If not, is there an alternative that is a clean, in-memory key-value store (persistence is less important to me), that is self-healing and works peer-to-peer?</li>
</ol>

<p>Consul/etcd/zookeeper all work 100% peer-to-peer and self-heal (which is great), but their performance (supposedly) is far below Redis in-memory KV. They aren't built, e.g. for looking up a session with each Web API request. This is partially due to non-sharding (100% copies), partially due to disk writes.</p>

<p>Kafka's model (although a message queue, not a KV store) works well too (but depends on zk underneath): partitions and replicas, but essentially self-healing. I talk to one broker, it tells me where my topic ("shard") lives, I get it. </p>

<p>Is there any way to get that full autonomy using Redis so I can deploy in kube/swarm/cloud, or an alternative that provides similar performance with the autonomous model?</p>

<p>You can use the Kubernetes Redis Example which work with Redis Sentinel. In case the master fails, sentinel promotes a slave to the new master. The Replication Controller boots a new Slave Pod. Your application connects to sentinel and from that service you will get the ip of the new master. </p>

<p>Redis Sentinel <a href="https://redis.io/topics/sentinel" rel="nofollow noreferrer">https://redis.io/topics/sentinel</a></p>

<p>Kubernetes Example <a href="https://github.com/kubernetes/examples/tree/master/staging/storage/redis" rel="nofollow noreferrer">https://github.com/kubernetes/examples/tree/master/staging/storage/redis</a></p>
