<p>A redis instance is deployed in k8s cluster, memory limit is 1G:</p>
<pre class="lang-yaml prettyprint-override"><code>resources:
  limits:
    memory: 1Gi
</code></pre>
<p>And also set redis <code>maxmemory</code> to 1G, actual memory usage is ~550M:</p>
<pre><code>info memory
# Memory
used_memory:577791152
used_memory_human:551.02M
used_memory_rss:611942400
used_memory_rss_human:583.59M
used_memory_peak:578877888
used_memory_peak_human:552.06M
used_memory_peak_perc:99.81%
used_memory_overhead:32108388
used_memory_startup:786608
used_memory_dataset:545682764
used_memory_dataset_perc:94.57%
total_system_memory:67555897344
total_system_memory_human:62.92G
used_memory_lua:41984
used_memory_lua_human:41.00K
maxmemory:1000000000
maxmemory_human:953.67M
maxmemory_policy:noeviction
mem_fragmentation_ratio:1.06
mem_allocator:jemalloc-4.0.3
active_defrag_running:0
lazyfree_pending_objects:0
</code></pre>
<p>I observed that the redis container memory usage exceeds 1G from prometheus (metric <code>sum by(container_name) (container_memory_working_set_bytes{namespace=&quot;$namespace&quot;,container_name!=&quot;&quot;,container_name=~&quot;$container&quot;,container_name!=&quot;POD&quot;,pod_name=&quot;$pod&quot;})</code>):</p>
<p><a href="https://i.stack.imgur.com/JkUB4.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/JkUB4.png" alt="enter image description here" /></a></p>
<p>So here is the question:</p>
<ol>
<li>How to free up the memory?</li>
<li>what's the relationship between <code>container_memory_working_set_bytes</code> and redis reported <code>used_memory_rss</code> <code>used_memory</code>?</li>
</ol>

<p>This is a problem relating to a k8s prometheus <a href="https://github.com/grafana/kubernetes-app/issues/44" rel="nofollow noreferrer">issue</a>, prometheus actually got a doubled result:</p>
<pre><code>container_cpu_usage_seconds_total{container=&quot;redis-server&quot;,container_name=&quot;redis-server&quot;,cpu=&quot;total&quot;,endpoint=&quot;https-metrics&quot;,id=&quot;/kubepods/pod149c0943-bc55-4e5d-ac9d-59fedca56138/1e2f5a63ddba49cb13465383e0de66e50ce9f084b52975f9bea8d1d77b966b3c&quot;,image=&quot;sha256:8a74675d2c029efb8c574f6097f2494d0e6acb7427aeea874227be48d96a3fb2&quot;,instance=&quot;192.168.116.119:10250&quot;,job=&quot;expose-kubelets-metrics&quot;,name=&quot;k8s_redis-server_redis-server-0_user-data-service-test_149c0943-bc55-4e5d-ac9d-59fedca56138_0&quot;,namespace=&quot;user-data-service-test&quot;,node=&quot;k8sworker03&quot;,pod=&quot;redis-server-0&quot;,pod_name=&quot;redis-server-0&quot;,service=&quot;expose-kubelets-metrics&quot;}

container_cpu_usage_seconds_total{container=&quot;redis-server&quot;,container_name=&quot;redis-server&quot;,endpoint=&quot;https-metrics&quot;,instance=&quot;192.168.116.119:10250&quot;,job=&quot;expose-kubelets-metrics&quot;,namespace=&quot;user-data-service-test&quot;,node=&quot;k8sworker03&quot;,pod=&quot;redis-server-0&quot;,pod_name=&quot;redis-server-0&quot;,service=&quot;expose-kubelets-metrics&quot;}    
</code></pre>
<p>So the result is doubbled.</p>
