{"action":"created","issue":{"url":"https://api.github.com/repos/lxc/lxd/issues/1961","repository_url":"https://api.github.com/repos/lxc/lxd","labels_url":"https://api.github.com/repos/lxc/lxd/issues/1961/labels{/name}","comments_url":"https://api.github.com/repos/lxc/lxd/issues/1961/comments","events_url":"https://api.github.com/repos/lxc/lxd/issues/1961/events","html_url":"https://github.com/lxc/lxd/issues/1961","id":151556448,"number":1961,"title":"lxc_utils - utils.c:mkdir_p:253 - No space left on device - failed to create directory '/sys/fs/cgroup/memory//lxc/test123'","user":{"login":"rchincholkar","id":11195635,"avatar_url":"https://avatars.githubusercontent.com/u/11195635?v=3","gravatar_id":"","url":"https://api.github.com/users/rchincholkar","html_url":"https://github.com/rchincholkar","followers_url":"https://api.github.com/users/rchincholkar/followers","following_url":"https://api.github.com/users/rchincholkar/following{/other_user}","gists_url":"https://api.github.com/users/rchincholkar/gists{/gist_id}","starred_url":"https://api.github.com/users/rchincholkar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rchincholkar/subscriptions","organizations_url":"https://api.github.com/users/rchincholkar/orgs","repos_url":"https://api.github.com/users/rchincholkar/repos","events_url":"https://api.github.com/users/rchincholkar/events{/privacy}","received_events_url":"https://api.github.com/users/rchincholkar/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/lxc/lxd/labels/Incomplete","name":"Incomplete","color":"5319e7"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-04-28T06:05:04Z","updated_at":"2016-04-28T16:49:55Z","closed_at":"2016-04-28T16:49:54Z","body":"Hi,\r\n\r\nI am getting following error when I try to start any stopped container and when I try to launch a new container.\r\n\r\nroot@lxd-prod3:~# lxc start test123\r\nerror: Error calling 'lxd forkstart test123 /var/lib/lxd/containers /var/log/lxd/test123/lxc.conf': err='exit status 1'\r\nTry `lxc info --show-log test123` for more info\r\n\r\n\r\nroot@lxd-prod3:~# lxc info --show-log test123\r\nName: test123\r\nArchitecture: x86_64\r\nCreated: 2016/04/28 05:42 UTC\r\nStatus: Stopped\r\nType: persistent\r\nProfiles: default\r\n\r\nLog:\r\n\r\n            lxc 20160428111210.781 INFO     lxc_confile - confile.c:config_idmap:1495 - read uid map: type u nsid 0 hostid 165536 range 65536\r\n            lxc 20160428111210.781 INFO     lxc_confile - confile.c:config_idmap:1495 - read uid map: type g nsid 0 hostid 165536 range 65536\r\n            lxc 20160428111214.394 INFO     lxc_confile - confile.c:config_idmap:1495 - read uid map: type u nsid 0 hostid 165536 range 65536\r\n            lxc 20160428111214.394 INFO     lxc_confile - confile.c:config_idmap:1495 - read uid map: type g nsid 0 hostid 165536 range 65536\r\n            lxc 20160428111214.615 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 3 \r\n            lxc 20160428111214.615 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 7 \r\n            lxc 20160428111214.617 INFO     lxc_container - lxccontainer.c:do_lxcapi_start:797 - Attempting to set proc title to [lxc monitor] /var/lib/lxd/containers test123\r\n            lxc 20160428111214.617 INFO     lxc_utils - utils.c:setproctitle:1460 - setting cmdline failed - Invalid argument\r\n            lxc 20160428111214.617 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 7 \r\n            lxc 20160428111214.617 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor\r\n            lxc 20160428111214.617 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .reject_force_umount  # comment this to allow umount -f;  not recommended.\r\n            lxc 20160428111214.617 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for reject_force_umount action 0\r\n            lxc 20160428111214.617 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts\r\n\r\n            lxc 20160428111214.617 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for reject_force_umount action 0\r\n            lxc 20160428111214.617 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts\r\n\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .[all].\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .kexec_load errno 1.\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for kexec_load action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for kexec_load action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .open_by_handle_at errno 1.\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for open_by_handle_at action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for open_by_handle_at action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .init_module errno 1.\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for init_module action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for init_module action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .finit_module errno 1.\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for finit_module action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for finit_module action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for finit_module action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .delete_module errno 1.\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for delete_module action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for delete_module action 327681\r\n            lxc 20160428111214.618 INFO     lxc_seccomp - seccomp.c:parse_config_v2:456 - Merging in the compat seccomp ctx into the main one\r\n            lxc 20160428111214.618 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/bin/lxd callhook /var/lib/lxd 21 start' for container 'test123', config section 'lxc'\r\n            lxc 20160428111214.618 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 3\r\n            lxc 20160428111214.618 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 7\r\n            lxc 20160428111214.620 INFO     lxc_monitor - monitor.c:lxc_monitor_sock_name:178 - using monitor sock name lxc/d78a9d7e97b4b375//var/lib/lxd/containers\r\n            lxc 20160428111214.733 DEBUG    lxc_start - start.c:setup_signal_fd:289 - sigchild handler set\r\n            lxc 20160428111214.734 DEBUG    lxc_console - console.c:lxc_console_peer_default:473 - no console peer\r\n            lxc 20160428111214.734 INFO     lxc_start - start.c:lxc_init:488 - 'test123' is initialized\r\n            lxc 20160428111214.734 DEBUG    lxc_start - start.c:__lxc_start:1259 - Not dropping cap_sys_boot or watching utmp\r\n            lxc 20160428111214.735 INFO     lxc_start - start.c:resolve_clone_flags:956 - Cloning a new user namespace\r\n            lxc 20160428111214.736 DEBUG    lxc_conf - conf.c:instantiate_macvlan:2685 - instantiated macvlan 'mc7JSSIO', index is '470' and mode '4'\r\n            lxc 20160428111214.736 INFO     lxc_cgroup - cgroup.c:cgroup_init:68 - cgroup driver cgroupfs-ng initing for test123\r\n            lxc 20160428111214.737 ERROR    lxc_utils - utils.c:mkdir_p:253 - No space left on device - failed to create directory '/sys/fs/cgroup/memory//lxc/test123'\r\n            lxc 20160428111214.737 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_create:1086 - No space left on device - Failed to create /sys/fs/cgroup/memory//lxc/test123: No space left on device\r\n            lxc 20160428111214.737 ERROR    lxc_utils - utils.c:mkdir_p:253 - No space left on device - failed to create directory '/sys/fs/cgroup/memory//lxc/test123-1'\r\n            lxc 20160428111214.737 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_create:1086 - No space left on device - Failed to create /sys/fs/cgroup/memory//lxc/test123-1: No space left on device\r\n            lxc 20160428111214.737 ERROR    lxc_utils - utils.c:mkdir_p:253 - No space left on device - failed to create directory '/sys/fs/cgroup/memory//lxc/test123-2'\r\n            lxc 20160428111214.737 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_create:1086 - No space left on device - Failed to create /sys/fs/cgroup/memory//lxc/test123-2: No space left on device\r\n            lxc 20160428111214.738 ERROR    lxc_utils - utils.c:mkdir_p:253 - No space left on device - failed to create directory '/sys/fs/cgroup/memory//lxc/test123-3'\r\n            lxc 20160428111214.738 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_create:1086 - No space left on device - Failed to create /sys/fs/cgroup/memory//lxc/test123-3: No space left on device\r\n            lxc 20160428111214.738 ERROR    lxc_utils - utils.c:mkdir_p:253 - No space left on device - failed to create directory '/sys/fs/cgroup/memory//lxc/test123-4'\r\n            lxc 20160428111214.738 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_create:1086 - No space left on device - Failed to create /sys/fs/cgroup/memory//lxc/test123-4: No space left on device\r\n\r\n\r\n\r\nWe cant upgrade this server now, Its in production. \r\nIs there any workaround for this?\r\n\r\nThanks! "},"comment":{"url":"https://api.github.com/repos/lxc/lxd/issues/comments/215491788","html_url":"https://github.com/lxc/lxd/issues/1961#issuecomment-215491788","issue_url":"https://api.github.com/repos/lxc/lxd/issues/1961","id":215491788,"user":{"login":"stgraber","id":2230599,"avatar_url":"https://avatars.githubusercontent.com/u/2230599?v=3","gravatar_id":"","url":"https://api.github.com/users/stgraber","html_url":"https://github.com/stgraber","followers_url":"https://api.github.com/users/stgraber/followers","following_url":"https://api.github.com/users/stgraber/following{/other_user}","gists_url":"https://api.github.com/users/stgraber/gists{/gist_id}","starred_url":"https://api.github.com/users/stgraber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stgraber/subscriptions","organizations_url":"https://api.github.com/users/stgraber/orgs","repos_url":"https://api.github.com/users/stgraber/repos","events_url":"https://api.github.com/users/stgraber/events{/privacy}","received_events_url":"https://api.github.com/users/stgraber/received_events","type":"User","site_admin":false},"created_at":"2016-04-28T16:49:54Z","updated_at":"2016-04-28T16:49:54Z","body":"Nope, those would only have been useful on a system currently exhibiting the problem.\r\n\r\nI'll close the issue for now, feel free to update it if you ever run into this again. "}}