title: Elasticsearch node keeps restarting
body: <p>I have a simple 3 node cluster here at home used for testing. 3 eligible master nodes and two data nodes. However, one of the two data nodes can never seem to initialize the data shards properly. </p>
  
  <p>The cluster health and shards overview illustrate this.</p>
  
  <pre><code>{"cluster_name":"test","status":"yellow","timed_out":false,"number_of_nodes":3,"number_of_data_nodes":2,"active_primary_shards":28,"active_shards":50,"relocating_shards":0,"initializing_shards":2,"unassigned_shards":4,"number_of_pending_tasks":0}
  </code></pre>
  
  <p>The shards overview indicate that the various shards are being initialized </p>
  
  <pre><code>.marvel-2015.05.28 0 r INITIALIZING              127.0.0.1 ES2 
  .marvel-2015.05.28 0 p STARTED      7656  21.3mb 127.0.0.1 ES1 
  .kibana            0 r STARTED                   127.0.0.1 ES2 
  .kibana            0 p STARTED        16  13.8kb 127.0.0.1 ES1 
  kibana-int         4 p STARTED         0    115b 127.0.0.1 ES1 
  kibana-int         2 r INITIALIZING              127.0.0.1 ES2   
  </code></pre>
  
  <p>However, at some point during this process, the node just dies and restarts.</p>
  
  <pre><code>{"cluster_name":"olympus","status":"yellow","timed_out":false,"number_of_nodes":2,"number_of_data_nodes":1,"active_primary_shards":28,"active_shards":28,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":28,"number_of_pending_tasks":0}
  </code></pre>
  
  <p>The node eventually comes back up and the aforementioned process starts up again, but then it just dies eventually yet again.</p>
  
  <p>The logs on the problematic node doesnt seem to say much either.</p>
  
  <pre><code>[2015-05-28 11:31:38,984][INFO ][cluster.service          ] [ES2] detected_master [ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]], added {[ES3][onaC5Y2bSd-nVTDRXsxjBg][es3pi][inet[/192.168.88.21:9300]]{data=false, master=true},[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]],}, reason: zen-disco-receive(from master [[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]]])
  [2015-05-28 11:31:40,717][INFO ][http                     ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.88.20:9200]}
  [2015-05-28 11:31:40,722][INFO ][node                     ] [ES2] started
  [2015-05-28 11:34:21,024][INFO ][node                     ] [ES2] version[1.5.2], pid[591], build[62ff986/2015-04-27T09:21:06Z]
  [2015-05-28 11:34:21,065][INFO ][node                     ] [ES2] initializing ...
  [2015-05-28 11:34:21,826][INFO ][plugins                  ] [ES2] loaded [marvel], sites [marvel]
  [2015-05-28 11:35:41,387][INFO ][node                     ] [ES2] initialized
  [2015-05-28 11:35:41,401][INFO ][node                     ] [ES2] starting ...
  [2015-05-28 11:35:43,174][INFO ][transport                ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.88.20:9300]}
  [2015-05-28 11:35:43,538][INFO ][discovery                ] [ES2] olympus/FQWotgreS2iTZ_NAJEBmaA
  [2015-05-28 11:35:47,758][INFO ][cluster.service          ] [ES2] detected_master [ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]], added {[ES3][onaC5Y2bSd-nVTDRXsxjBg][es3pi][inet[/192.168.88.21:9300]]{data=false, master=true},[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]],}, reason: zen-disco-receive(from master [[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]]])
  [2015-05-28 11:35:49,442][INFO ][http                     ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.88.20:9200]}
  [2015-05-28 11:35:49,447][INFO ][node                     ] [ES2] started
  [2015-05-28 11:39:40,064][INFO ][node                     ] [ES2] version[1.5.2], pid[655], build[62ff986/2015-04-27T09:21:06Z]
  [2015-05-28 11:39:40,088][INFO ][node                     ] [ES2] initializing ...
  [2015-05-28 11:39:40,880][INFO ][plugins                  ] [ES2] loaded [marvel], sites [marvel]
  [2015-05-28 11:41:03,892][INFO ][node                     ] [ES2] initialized
  [2015-05-28 11:41:03,906][INFO ][node                     ] [ES2] starting ...
  [2015-05-28 11:41:05,665][INFO ][transport                ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.88.20:9300]}
  [2015-05-28 11:41:06,042][INFO ][discovery                ] [ES2] olympus/5JH1JO7PRICesi0tadHZJw
  [2015-05-28 11:41:10,233][INFO ][cluster.service          ] [ES2] detected_master [ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]], added {[ES3][onaC5Y2bSd-nVTDRXsxjBg][es3pi][inet[/192.168.88.21:9300]]{data=false, master=true},[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]],}, reason: zen-disco-receive(from master [[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]]])
  [2015-05-28 11:41:11,844][INFO ][http                     ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.88.20:9200]}
  [2015-05-28 11:41:11,850][INFO ][node                     ] [ES2] started
  [2015-05-28 11:44:47,113][INFO ][node                     ] [ES2] version[1.5.2], pid[763], build[62ff986/2015-04-27T09:21:06Z]
  [2015-05-28 11:44:47,173][INFO ][node                     ] [ES2] initializing ...
  [2015-05-28 11:44:47,985][INFO ][plugins                  ] [ES2] loaded [marvel], sites [marvel]
  [2015-05-28 11:46:08,003][INFO ][node                     ] [ES2] initialized
  [2015-05-28 11:46:08,017][INFO ][node                     ] [ES2] starting ...
  [2015-05-28 11:46:09,818][INFO ][transport                ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.88.20:9300]}
  [2015-05-28 11:46:10,184][INFO ][discovery                ] [ES2] olympus/AjDpUJt1SK2ncVah2ZVQJA
  [2015-05-28 11:46:14,397][INFO ][cluster.service          ] [ES2] detected_master [ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]], added {[ES3][onaC5Y2bSd-nVTDRXsxjBg][es3pi][inet[/192.168.88.21:9300]]{data=false, master=true},[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]],}, reason: zen-disco-receive(from master [[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]]])
  [2015-05-28 11:46:16,056][INFO ][http                     ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.88.20:9200]}
  [2015-05-28 11:46:16,061][INFO ][node                     ] [ES2] started
  [2015-05-28 11:49:49,595][INFO ][node                     ] [ES2] version[1.5.2], pid[824], build[62ff986/2015-04-27T09:21:06Z]
  [2015-05-28 11:49:49,628][INFO ][node                     ] [ES2] initializing ...
  [2015-05-28 11:49:50,394][INFO ][plugins                  ] [ES2] loaded [marvel], sites [marvel]
  [2015-05-28 11:51:17,611][INFO ][node                     ] [ES2] initialized
  [2015-05-28 11:51:17,626][INFO ][node                     ] [ES2] starting ...
  [2015-05-28 11:51:19,409][INFO ][transport                ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.88.20:9300]}
  [2015-05-28 11:51:19,775][INFO ][discovery                ] [ES2] olympus/KVeatmVAR4aGzumHUy4PQA
  [2015-05-28 11:51:24,007][INFO ][cluster.service          ] [ES2] detected_master [ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]], added {[ES3][onaC5Y2bSd-nVTDRXsxjBg][es3pi][inet[/192.168.88.21:9300]]{data=false, master=true},[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]],}, reason: zen-disco-receive(from master [[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]]])
  [2015-05-28 11:51:25,739][INFO ][http                     ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.88.20:9200]}
  [2015-05-28 11:51:25,744][INFO ][node                     ] [ES2] started
  [2015-05-28 11:55:05,150][INFO ][node                     ] [ES2] version[1.5.2], pid[933], build[62ff986/2015-04-27T09:21:06Z]
  [2015-05-28 11:55:05,157][INFO ][node                     ] [ES2] initializing ...
  [2015-05-28 11:55:06,032][INFO ][plugins                  ] [ES2] loaded [marvel], sites [marvel]
  [2015-05-28 11:56:24,370][INFO ][node                     ] [ES2] initialized
  [2015-05-28 11:56:24,384][INFO ][node                     ] [ES2] starting ...
  [2015-05-28 11:56:26,136][INFO ][transport                ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/192.168.88.20:9300]}
  [2015-05-28 11:56:26,496][INFO ][discovery                ] [ES2] olympus/DVO0kHFbQf66qX95DHfzjw
  [2015-05-28 11:56:30,717][INFO ][cluster.service          ] [ES2] detected_master [ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]], added {[ES3][onaC5Y2bSd-nVTDRXsxjBg][es3pi][inet[/192.168.88.21:9300]]{data=false, master=true},[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]],}, reason: zen-disco-receive(from master [[ES1][mZUzucXKTnGUdfXNx1OYEQ][es1][inet[/192.168.88.22:9300]]])
  [2015-05-28 11:56:32,396][INFO ][http                     ] [ES2] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/192.168.88.20:9200]}
  [2015-05-28 11:56:32,410][INFO ][node                     ] [ES2] started
  </code></pre>
  
  <p>Anyone have any ideas on what's going wrong here? It would be much appreciated!</p>