title: Way to determine healthy Cassandra cluster?
body: <p>I've been tasked with re-writing some sub-par Ansible playbooks to stand up a Cassandra cluster in CentOS. Quite frankly, there doesn't seem to be much information on Cassandra out there.</p>
  
  <p>I've managed to get the service running on all three nodes at the same time, using the following configuration file, info scrubbed.</p>
  
  <pre><code>HOSTIP=10.0.0.1
  MSIP=10.10.10.10
  ADMIN_EMAIL=my@email.com
  LICENSE_FILE=/tmp/license.conf
  USE_LDAP_REMOTE_HOST=n
  
  ENABLE_AX=y
  MP_POD=gateway
  
  REGION=test-1
  
  USE_ZK_CLUSTER=y
  ZK_HOSTS="10.0.0.1 10.0.0.2 10.0.0.3"
  ZK_CLIENT_HOSTS="10.0.0.1 10.0.0.2 10.0.0.3"
  
  USE_CASS_CLUSTER=y
  CASS_HOSTS="10.0.0.1:1,1 10.0.0.2:1,1 10.0.0.3:1,1"
  CASS_USERNAME=test
  CASS_PASSWORD=test
  </code></pre>
  
  <p>The HOSTIP changes depending on which node the configuration file is on.</p>
  
  <p>The problem is, when I run <code>nodetool ring</code>, each node says there's only two nodes in the cluster: itself and one other, seemingly random from the other two.</p>
  
  <p>What are some basic sanity checks to determine a "healthy" Cassandra cluster? Why is nodetool saying each one thinks there's a different node missing from the cluster?</p>