title: Why cassandra use up so much of CPU
body: <p>I'm not an expert about Cassandra. Few months ago, i got an order to establish database for a social network application. For many reasons I refers from network, i choose Cassandra to store metadata of all feeds and redis to store list,zset or id (light weight and most changed data). My Cassandra cluster includes 4 nodes with replicate factor is set to 2, 4 nodes in 2 servers(each server has 2 nodes). I configured cassandra.yaml to tune Cassandra cluster as fast as possible.
  My system seems to be good at all till a day I review system stat. It's unbelievable. Let me explain why :</p>
  
  <pre><code>   First: I type "iostat -x 5" to view io stat. The result is as following:
  
  avg-cpu:  %user   %nice %system %iowait  %steal   %idle
             0.35    0.00    0.82    0.08    0.00   98.75
  
  Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
  sda               0.00     0.00    0.00    2.80     0.00    22.40     8.00     0.01    4.50   3.93   1.10
  sdb               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
  sdd               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
  sdc               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
  </code></pre>
  
  <p>The CPU here is 0.35% and while we only have 4000 active users , IO is not my problem yet. The problem is here.</p>
  
  <pre><code>PID  USER      PR   NI VIRT  RES  SHR S  %CPU  %MEM  TIME+   COMMAND
  
  
  4689 root      20   0 13.1g 6.8g 182m S 514.6 21.8  37374:09 java      5029 root      20   0 13.0g 6.7g 174m S  1.7 21.5  10113:45 java
  </code></pre>
  
  <p>The CPU cassandra node use up so much . Even a node can be up to 1400% of CPU and later it 's down to 0.5 %. I really don't know why. Who can tell me why ? and if possible, please tell me the solution to deal with it. Thanks in advance!</p>