title: RabbitMQ cluster is not reconnecting after network failure
body: <p>I have a RabbitMQ cluster with two nodes in production and the cluster is breaking with these error messages:</p>
  <pre><code>=ERROR REPORT==== 23-Dec-2011::04:21:34 ===
  ** Node rabbit@rabbitmq02 not responding **
  ** Removing (timedout) connection **
  
  =INFO REPORT==== 23-Dec-2011::04:21:35 ===
  node rabbit@rabbitmq02 lost 'rabbit'
  
  =ERROR REPORT==== 23-Dec-2011::04:21:49 ===
  Mnesia(rabbit@rabbitmq01): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, rabbit@rabbitmq02}
  </code></pre>
  <p>I tried to simulate the problem by killing the connection between the two nodes using &quot;tcpkill&quot;. The cluster has disconnected, and surprisingly the two nodes are not trying to reconnect!</p>
  <p>When the cluster breaks, HAProxy load balancer still marks both nodes as active and send requests to both of them, although they are not in a cluster.</p>
  <p>My questions:</p>
  <ol>
  <li><p>If the nodes are configured to work as a cluster, when I get a network failure, why aren't they trying to reconnect afterwards?</p>
  </li>
  <li><p>How can I identify broken cluster and shutdown one of the nodes?  I have consistency problems when working with the two nodes separately.</p>
  </li>
  </ol>