{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/28619","repository_url":"https://api.github.com/repos/kubernetes/kubernetes","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/28619/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/28619/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/28619/events","html_url":"https://github.com/kubernetes/kubernetes/issues/28619","id":164377758,"number":28619,"title":"Eviction manager's observed memory.available is vastly different from /proc/meminfo MemAvailable","user":{"login":"mtaufen","id":903617,"avatar_url":"https://avatars.githubusercontent.com/u/903617?v=3","gravatar_id":"","url":"https://api.github.com/users/mtaufen","html_url":"https://github.com/mtaufen","followers_url":"https://api.github.com/users/mtaufen/followers","following_url":"https://api.github.com/users/mtaufen/following{/other_user}","gists_url":"https://api.github.com/users/mtaufen/gists{/gist_id}","starred_url":"https://api.github.com/users/mtaufen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mtaufen/subscriptions","organizations_url":"https://api.github.com/users/mtaufen/orgs","repos_url":"https://api.github.com/users/mtaufen/repos","events_url":"https://api.github.com/users/mtaufen/events{/privacy}","received_events_url":"https://api.github.com/users/mtaufen/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/kind/bug","name":"kind/bug","color":"e11d21"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/priority/P0","name":"priority/P0","color":"e11d21"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/node","name":"team/node","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-07-07T18:35:05Z","updated_at":"2016-07-08T00:55:28Z","closed_at":null,"body":"I'm working on a node e2e memory eviction test, and even though I set the aggressive hard eviction threshold of ` --eviction-hard memory.available<2.5Gi`, I noticed that my pods were chewing up memory without getting evicted, so I started digging.\r\n\r\nI logged the `observations` made from the `synchronize` function in `kubernetes/pkg/kubelet/eviction/eviction_manager.go`, with `glog.Infof(\"eviction manager: observations: %s\", strings.Replace(spew.Sdump(observations), \"\\n\", \" \", -1))`, and I saw the following:\r\n\r\n`eviction manager: observations: (eviction.signalObservations) (len=1) {  (eviction.Signal) (len=16) \"memory.available\": (*resource.Quantity)(0xc820d61940)(3377236Ki) }`\r\n\r\nsampling `/proc/meminfo` about two seconds later reveals:\r\n```\r\nMemTotal:        3801016 kB\r\nMemFree:          826748 kB\r\nMemAvailable:    1213620 kB\r\n```\r\n\r\nAssuming kB and Ki are equivalent units (they seem to both be kibibytes), that's a 2163616Ki difference in reporting between `memory.available` and `MemAvailable`.\r\n\r\nMy test collectively chews up memory in steps of about 40MB every 10 seconds, so they could show up as different, but shouldn't be THAT different.\r\n\r\nTagging @vishh @derekwaynecarr "},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/231249303","html_url":"https://github.com/kubernetes/kubernetes/issues/28619#issuecomment-231249303","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/28619","id":231249303,"user":{"login":"vishh","id":7063592,"avatar_url":"https://avatars.githubusercontent.com/u/7063592?v=3","gravatar_id":"","url":"https://api.github.com/users/vishh","html_url":"https://github.com/vishh","followers_url":"https://api.github.com/users/vishh/followers","following_url":"https://api.github.com/users/vishh/following{/other_user}","gists_url":"https://api.github.com/users/vishh/gists{/gist_id}","starred_url":"https://api.github.com/users/vishh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vishh/subscriptions","organizations_url":"https://api.github.com/users/vishh/orgs","repos_url":"https://api.github.com/users/vishh/repos","events_url":"https://api.github.com/users/vishh/events{/privacy}","received_events_url":"https://api.github.com/users/vishh/received_events","type":"User","site_admin":false},"created_at":"2016-07-08T00:55:28Z","updated_at":"2016-07-08T00:55:28Z","body":"@mtaufen and I looked at this issue together briefly today and we suspect that cAdvisor is not accounting for tmpfs as part of `working_set`. We need to isolate and confirm this issue. \r\n@derekwaynecarr since your day starts earlier, do take a look at cAdvisor tmpfs handling if you get a chance. "}}
{"actor":{"display_login":"vishh"}}