title: How to increase Redis performance when 100% CPU? Sharding? Fastest .Net Client?
body: <p>Due to massive load increases on our website redis is now struggling with peak load because the redis server instance is reaching 100% CPU (on one of eight cores) resulting in time outs.</p>
  
  <p>We've updated our client software to ServiceStack V3 (coming from BookSleeve 1.1.0.4) and upgraded the redis server to 2.8.11 (coming from 2.4.x). I chose ServiceStack due to the existence of the <a href="https://github.com/TheCloudlessSky/Harbour.RedisSessionStateStore" rel="noreferrer">Harbour.RedisSessionStateStore</a> that uses ServiceStack.Redis. We used AngiesList.Redis before together with BookSleeve, but we experienced 100% with that too.</p>
  
  <p>We have eight redis servers configured as a master/slave tree. One single server for session state tho. The others are for data cache. One master with two master/slaves connected to two slaves each.</p>
  
  <p>The servers hold about 600 client connections at peak when they start to get clogged at 100% CPU.</p>
  
  <p>What can we do to increase performance? </p>
  
  <p>Sharding and/or StackExchange Redis client (no session state client available to my knowledge...).</p>
  
  <p>Or could it be something else? The session server also hits 100% and it is not connected to any other servers (data and network throughput are low).</p>
  
  <hr>
  
  <p><strong>Update 1: Analysis of redis-cli INFO</strong></p>
  
  <p>Here's the output of the INFO command after one night of running Redis 2.8.</p>
  
  <pre><code># Server
  redis_version:2.8.11
  redis_git_sha1:00000000
  redis_git_dirty:0
  redis_build_id:7a57b118eb75b37f
  redis_mode:standalone
  os:Linux 2.6.32-431.11.2.el6.x86_64 x86_64
  arch_bits:64
  multiplexing_api:epoll
  gcc_version:4.4.7
  process_id:5843
  run_id:d5bb838857d61a9673e36e5bf608fad5a588ac5c
  tcp_port:6379
  uptime_in_seconds:152778
  uptime_in_days:1
  hz:10
  lru_clock:10765770
  config_file:/etc/redis/6379.conf
  
  # Clients
  connected_clients:299
  client_longest_output_list:0
  client_biggest_input_buf:0
  blocked_clients:0
  
  # Memory
  used_memory:80266784
  used_memory_human:76.55M
  used_memory_rss:80719872
  used_memory_peak:1079667208
  used_memory_peak_human:1.01G
  used_memory_lua:33792
  mem_fragmentation_ratio:1.01
  mem_allocator:jemalloc-3.2.0
  
  # Persistence
  loading:0
  rdb_changes_since_last_save:70245
  rdb_bgsave_in_progress:0
  rdb_last_save_time:1403274022
  rdb_last_bgsave_status:ok
  rdb_last_bgsave_time_sec:0
  rdb_current_bgsave_time_sec:-1
  aof_enabled:0
  aof_rewrite_in_progress:0
  aof_rewrite_scheduled:0
  aof_last_rewrite_time_sec:-1
  aof_current_rewrite_time_sec:-1
  aof_last_bgrewrite_status:ok
  aof_last_write_status:ok
  
  # Stats
  total_connections_received:3375
  total_commands_processed:30975281
  instantaneous_ops_per_sec:163
  rejected_connections:0
  sync_full:10
  sync_partial_ok:0
  sync_partial_err:5
  expired_keys:8059370
  evicted_keys:0
  keyspace_hits:97513
  keyspace_misses:46044
  pubsub_channels:2
  pubsub_patterns:0
  latest_fork_usec:22040
  
  # Replication
  role:master
  connected_slaves:2
  slave0:ip=xxx.xxx.xxx.xxx,port=6379,state=online,offset=272643782764,lag=1
  slave1:ip=xxx.xxx.xxx.xxx,port=6379,state=online,offset=272643784216,lag=1
  master_repl_offset:272643811961
  repl_backlog_active:1
  repl_backlog_size:1048576
  repl_backlog_first_byte_offset:272642763386
  repl_backlog_histlen:1048576
  
  # CPU
  used_cpu_sys:20774.19
  used_cpu_user:2458.50
  used_cpu_sys_children:304.17
  used_cpu_user_children:1446.23
  
  # Keyspace
  db0:keys=77863,expires=77863,avg_ttl=3181732
  db6:keys=11855,expires=11855,avg_ttl=3126767
  </code></pre>
  
  <hr>
  
  <p><strong>Update 2: twemproxy (Sharding)</strong></p>
  
  <p>I've discovered an interesting component called <a href="https://github.com/twitter/twemproxy" rel="noreferrer">twemproxy</a>. This component, as I understand it, could Shard across multiple redis instances. </p>
  
  <p>Would this help relieve the CPU?</p>
  
  <p>It would save us a lot of programming time, but it would still take some effort to configure 3 extra instances on each server. So I'm hoping somebody can confirm or debunk this solution before we put in the work.</p>