title: Memcached slow gets, high CPU usage
body: <p>I've got a memcached instance running on a machine to take the stress of the database.
  Currently there are about 350 requests per second via PHP, which should be perfectly doable according to the memcached docs, but I'm seeing ridiculously slow get() times. Averaging around 60 ms, with spikes in both ways (0.1 ms and 250 ms).</p>
  
  <p>The memcached process is also using around 80% CPU all the time. It's getting really problematic as with all the gets combined it takes over 5 seconds for the page to complete.</p>
  
  <p>I'm pretty sure it's the get command as I've commented out in the code and the database takes over, making the memcached process use 0 CPU.</p>
  
  <p>Here are the stats:</p>
  
  <pre><code>stats
  STAT pid 617
  STAT uptime 855901
  STAT time 1370358572
  STAT version 1.4.5
  STAT pointer_size 32
  STAT rusage_user 15472.778988
  STAT rusage_system 38712.971409
  STAT curr_connections 175
  STAT total_connections 4423163
  STAT connection_structures 252
  STAT cmd_get 319670822
  STAT cmd_set 48996864
  STAT cmd_flush 0
  STAT get_hits 233440856
  STAT get_misses 86229966
  STAT delete_misses 11025386
  STAT delete_hits 11131141
  STAT incr_misses 27702934
  STAT incr_hits 19471007
  STAT decr_misses 0
  STAT decr_hits 0
  STAT cas_misses 0
  STAT cas_hits 0
  STAT cas_badval 0
  STAT auth_cmds 0
  STAT auth_errors 0
  STAT bytes_read 25484001864
  STAT bytes_written 77617943971
  STAT limit_maxbytes 201326592
  STAT accepting_conns 1
  STAT listen_disabled_num 0
  STAT threads 4
  STAT conn_yields 0
  STAT bytes 47135355
  STAT curr_items 539471
  STAT total_items 21293860
  STAT evictions 3183365
  STAT reclaimed 3222011
  END
  </code></pre>
  
  <p>And the settings:</p>
  
  <pre><code>stats settings
  STAT maxbytes 201326592
  STAT maxconns 1024
  STAT tcpport 11211
  STAT udpport 11211
  STAT inter 127.0.0.1
  STAT verbosity 0
  STAT oldest 0
  STAT evictions on
  STAT domain_socket NULL
  STAT umask 700
  STAT growth_factor 1.25
  STAT chunk_size 48
  STAT num_threads 4
  STAT stat_key_prefix :
  STAT detail_enabled no
  STAT reqs_per_event 20
  STAT cas_enabled yes
  STAT tcp_backlog 1024
  STAT binding_protocol auto-negotiate
  STAT auth_enabled_sasl no
  STAT item_size_max 1048576
  END
  </code></pre>
  
  <p>Now did I configure memcached wrong? Or is there something else going on?</p>
  
  <p>EDIT:</p>
  
  <p>On request, here's the code with the get (there's not much to it):</p>
  
  <pre><code>function getItem($memcached, $key, $id) {
      $md5key = md5($key.":".$id);
      $v = $memcached-&gt;get($md5key); // changing this to $v = false made the memcached CPU usage go to 0
      if ($v === false) {
          //code that fetches the correct data for each key, stores it in memcached and in $v.
      }
      return $v;
  }
  </code></pre>
  
  <p>And in the main script there's the following:</p>
  
  <pre><code>$memcached = new Memcached;
  $memcached-&gt;addServer('localhost', 11211) or die ("Could not connect to memcached server");
  $memcached-&gt;setOption(Memcached::OPT_COMPRESSION, false);
  
  $myItem = getItem($memcached, "key", "123");
  </code></pre>
  
  <p>EDIT2:
  For some reason I'm still noticing a high load on the database. Now when I manually check for the data to be present in the cache via telnet it's just fine. Could it be that the memcached client thinks the connection to the cache is timing out and hence goed to the database? That would leave me with the question why on earth the connection is timing out...</p>