title: What is the difference between “container_memory_working_set_bytes” and “container_memory_rss” metric on the container
body: <p>I need to monitor my container memory usage running on kubernetes cluster. After read some  articles there're two recommendations: &quot;container_memory_rss&quot;, &quot;container_memory_working_set_bytes&quot;</p>
  <p>The definitions of both metrics are said (from the cAdvisor code)</p>
  <ul>
  <li>&quot;container_memory_rss&quot; : The amount of anonymous and swap cache memory</li>
  <li>&quot;container_memory_working_set_bytes&quot;: The amount of working set memory, this includes recently accessed memory, dirty memory, and kernel memory</li>
  </ul>
  <p>I think both metrics are represent the bytes size on the physical memory that process uses. But there are some differences between the two values from my grafana dashboard.</p>
  <p><strong>My question is:</strong></p>
  <ul>
  <li>What is the difference between two metrics?</li>
  <li>Which metrics are much proper to monitor memory usage? Some post said both because one of those metrics reaches to the limit, then that container is oom killed.</li>
  </ul>