{"action":"created","issue":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4698","repository_url":"https://api.github.com/repos/zfsonlinux/zfs","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4698/labels{/name}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4698/comments","events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4698/events","html_url":"https://github.com/zfsonlinux/zfs/issues/4698","id":156796904,"number":4698,"title":"memory leaks until kmalloc-512 slab is full","user":{"login":"ptx0","id":12914599,"avatar_url":"https://avatars.githubusercontent.com/u/12914599?v=3","gravatar_id":"","url":"https://api.github.com/users/ptx0","html_url":"https://github.com/ptx0","followers_url":"https://api.github.com/users/ptx0/followers","following_url":"https://api.github.com/users/ptx0/following{/other_user}","gists_url":"https://api.github.com/users/ptx0/gists{/gist_id}","starred_url":"https://api.github.com/users/ptx0/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ptx0/subscriptions","organizations_url":"https://api.github.com/users/ptx0/orgs","repos_url":"https://api.github.com/users/ptx0/repos","events_url":"https://api.github.com/users/ptx0/events{/privacy}","received_events_url":"https://api.github.com/users/ptx0/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-05-25T16:39:50Z","updated_at":"2016-05-25T17:35:00Z","closed_at":null,"body":"* Linux 4.4.11\r\n* Memory: 32GB\r\n* ARC max: defaults (~15GB)\r\n* Root filesystem: ext4+mdraid\r\n* Pool layout: mirrored pair, 2x2TB HGST (partitions)\r\n* Features: Compression (lz4), no deduplication\r\n* Workload: 7 x Linux KVM on ZVOL direct and via iSCSI (512MB RAM each)\r\n\r\nI had this box running for 200+ days with its current workload on zfs+spl `0.6.5.4` with Linux `3.16`.\r\n* SLUB allocator\r\n* ARC usually stays at 15GB\r\n* VMs can balloon and shrink as needed\r\n* drop_caches frees ARC\r\n\r\nI recently updated the kernel to `4.4.11` and simultaneously updated zfs+spl to `0.6.5.7`\r\n* SLAB / SLUB allocators tested\r\n* Memory is consumed until it is all used, and then starts swapping\r\n* Applications are not using memory\r\n* ARC was stuck at 100MB\r\n* `ps auxw --sort -rss` numbers do not add up to the total used memory\r\n* `/proc/slabinfo` lists `kmalloc-512` as 30GB\r\n* `drop_caches` does nothing\r\n\r\nThe system does not hang, unlike other reclaim issue reports I have seen. Stopping virtual machines didn't free up any memory, either.\r\n\r\nI have Linux sysctl for overcommit set to defaults."},"comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments/221649187","html_url":"https://github.com/zfsonlinux/zfs/issues/4698#issuecomment-221649187","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/4698","id":221649187,"user":{"login":"behlendorf","id":148917,"avatar_url":"https://avatars.githubusercontent.com/u/148917?v=3","gravatar_id":"","url":"https://api.github.com/users/behlendorf","html_url":"https://github.com/behlendorf","followers_url":"https://api.github.com/users/behlendorf/followers","following_url":"https://api.github.com/users/behlendorf/following{/other_user}","gists_url":"https://api.github.com/users/behlendorf/gists{/gist_id}","starred_url":"https://api.github.com/users/behlendorf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/behlendorf/subscriptions","organizations_url":"https://api.github.com/users/behlendorf/orgs","repos_url":"https://api.github.com/users/behlendorf/repos","events_url":"https://api.github.com/users/behlendorf/events{/privacy}","received_events_url":"https://api.github.com/users/behlendorf/received_events","type":"User","site_admin":false},"created_at":"2016-05-25T17:35:00Z","updated_at":"2016-05-25T17:35:00Z","body":"@ptx0 can you try either reverting to 0.6.5.4 with the new kernel, or reverting to the old kernel with 0.6.5.7.  This way we can narrow down where this issue was introduced."}}
{"actor":{"display_login":"behlendorf"}}