{"action":"created","issue":{"url":"https://api.github.com/repos/habitat-sh/builder/issues/93","repository_url":"https://api.github.com/repos/habitat-sh/builder","labels_url":"https://api.github.com/repos/habitat-sh/builder/issues/93/labels{/name}","comments_url":"https://api.github.com/repos/habitat-sh/builder/issues/93/comments","events_url":"https://api.github.com/repos/habitat-sh/builder/issues/93/events","html_url":"https://github.com/habitat-sh/builder/issues/93","id":291661814,"number":93,"title":"Docker image export not working?","user":{"login":"elliott-davis","id":1130349,"avatar_url":"https://avatars2.githubusercontent.com/u/1130349?v=4","gravatar_id":"","url":"https://api.github.com/users/elliott-davis","html_url":"https://github.com/elliott-davis","followers_url":"https://api.github.com/users/elliott-davis/followers","following_url":"https://api.github.com/users/elliott-davis/following{/other_user}","gists_url":"https://api.github.com/users/elliott-davis/gists{/gist_id}","starred_url":"https://api.github.com/users/elliott-davis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/elliott-davis/subscriptions","organizations_url":"https://api.github.com/users/elliott-davis/orgs","repos_url":"https://api.github.com/users/elliott-davis/repos","events_url":"https://api.github.com/users/elliott-davis/events{/privacy}","received_events_url":"https://api.github.com/users/elliott-davis/received_events","type":"User","site_admin":false},"labels":[{"id":830264242,"url":"https://api.github.com/repos/habitat-sh/builder/labels/STABILITY","name":"STABILITY","color":"b60205","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-01-25T18:00:09Z","updated_at":"2018-02-07T19:14:57Z","closed_at":null,"author_association":"OWNER","body":"@indradhanush commented on [Fri Jan 05 2018](https://github.com/habitat-sh/habitat/issues/4418)\n\nDocker image exports are currently not working for us. We run on top of master (6e39b7a) with the Vagrant setup from [PR](https://github.com/habitat-sh/habitat/pull/4161) and the pending bug fix from [here](https://github.com/habitat-sh/habitat/pull/4392). After connecting a new plan with Docker integration, we see the docker exporter failing silently midway through building the image. We see the following logs:\r\n\r\n**Web UI logs**\r\n\r\n```\r\nSending build context to Docker daemon  158.6MB\r\nStep 1/11 : FROM scratch\r\n ---> \r\nStep 2/11 : ENV PATH /bin\r\n ---> Running in 77481113cc61\r\n ---> 4148ca6b6ac1\r\nRemoving intermediate container 77481113cc61\r\nStep 3/11 : ADD rootfs /\r\n ---> 8c0dada5a645\r\nStep 4/11 : USER 0:0\r\n ---> Running in 6201a153ede1\r\n ---> 27a5fb8962f3\r\nRemoving intermediate container 6201a153ede1\r\nStep 5/11 : RUN mkdir -p /hab/svc/zero-dep/data\r\n ---> Running in 14cd81545099\r\nRemoving intermediate container 14cd81545099\r\n--- END: Docker export ---\r\n```\r\n\r\n**Worker logs**\r\n\r\n```\r\nworker.1    | DEBUG:habitat_builder_worker::runner: Found runnable package, running docker export\r\nworker.1    | DEBUG:habitat_core::fs: resolved absolute path to program, program=dockerd, ident=core/docker, abs_path=/hab/pkgs/core/docker/17.09.0/20171001205930/bin/dockerd\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: building dockerd command, cmd=\"/hab/pkgs/core/docker/17.09.0/20171001205930/bin/dockerd\" \"-H\" \"unix:///hab/svc/builder-worker/data/888124392435916800/dockerd/var/run/docker.sock\" \"--pidfile\" \"/hab/svc/builder-worker/data/888124392435916800/dockerd/var/run/docker.pid\" \"--data-root\" \"/hab/svc/builder-worker/data/888124392435916800/dockerd/var/lib/docker\" \"--exec-root\" \"/hab/svc/builder-worker/data/888124392435916800/dockerd/var/run/docker\" \"--config-file\" \"/hab/svc/builder-worker/data/888124392435916800/dockerd/etc/daemon.json\" \"--group\" \"hab\" \"--iptables=false\" \"--ip-masq=false\" \"--ipv6=false\" \"--raw-logs\"\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: setting docker export command env, PATH=/hab/pkgs/core/docker/17.09.0/20171001205930/bin\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: spawning dockerd export command\r\napi.1       | DEBUG:builder_http_gateway::http::helpers: extract_pagination range: (start, end): (0, 49)\r\nworker.1    | DEBUG:habitat_core::fs: resolved absolute path to program, program=hab-pkg-export-docker, ident=core/hab-pkg-export-docker, abs_path=/hab/pkgs/core/hab-pkg-export-docker/0.51.0/20171218213726/bin/hab-pkg-export-docker\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: building docker export command, cmd=building docker export command, cmd=\"/hab/pkgs/core/hab-pkg-export-docker/0.51.0/20171218213726/bin/hab-pkg-export-docker\" \"--image-name\" \"<username-redacted>/zero-dep\" \"--base-pkgs-url\" \"http://localhost:9636\" \"--url\" \"http://localhost:9636\" \"--tag-latest\" \"--tag-version\" \"--tag-version-release\" \"--push-image\" \"--username\" \"<username-redacted>\" \"--password\" \"<password-redacted>\" \"--rm-image\" \"--registry-type\" \"docker\" \"/hab/svc/builder-worker/data/888124392435916800/out/<username-redacted>-zero-dep-0.1.0-20180105075532-x86_64-linux.hart\"\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: setting docker export command env, DOCKER_HOST=unix:///hab/svc/builder-worker/data/888124392435916800/dockerd/var/run/docker.sock\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: spawning docker export command\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\napi.1       | DEBUG:habitat_depot::server: Headers = hab-pkg-export-docker/0.51.0/20171218213726 (x86_64-linux; 4.10.0-19-generic)\r\nworker.1    | DEBUG:habitat_builder_worker::heartbeat: heartbeat pulsed: endpoint: \"29614@vagrant\" os: Linux state: Busy\r\njobsrv.1    | DEBUG:habitat_builder_jobsrv::server::worker_manager: Got heartbeat: endpoint: \"29614@vagrant\" os: Linux state: Busy\r\nworker.1    | DEBUG:habitat_builder_worker::heartbeat: heartbeat pulsed: endpoint: \"29614@vagrant\" os: Linux state: Busy\r\njobsrv.1    | DEBUG:habitat_builder_jobsrv::server::worker_manager: Got heartbeat: endpoint: \"29614@vagrant\" os: Linux state: Busy\r\njobsrv.1    | DEBUG:habitat_builder_jobsrv::server::scheduler: Process status: found 0 updated jobs\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: completed docker export command, status=ExitStatus(ExitStatus(256))\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: signaling dockerd to shutdown pid=30869, sig=TERM\r\nworker.1    | DEBUG:habitat_builder_worker::runner::docker: terminated dockerd\r\nworker.1    | DEBUG:habitat_builder_worker::runner::log_pipe: completing log_forwarder, job_id=888124392435916800\r\njobsrv.1    | DEBUG:habitat_builder_jobsrv::server::log_ingester: Log complete for job 888124392435916800\r\njobsrv.1    | DEBUG:habitat_builder_jobsrv::server::log_ingester: Archived log for job 888124392435916800\r\nworker.1    | DEBUG:habitat_builder_worker::runner: Failed studio build for indradhanush/zero-dep, err=BuildFailure(1)\r\nworker.1    | WARN:habitat_builder_worker::runner: Failed to remove studio dir /hab/svc/builder-worker/data/888124392435916800/studio, err: Error { repr: Os { code: 2, message: \"No such file or directory\" } }\r\n```\r\n\r\n**dockerd logs**\r\n\r\n```\r\ntime=\"2018-01-05T09:06:09.783395854Z\" level=info msg=\"libcontainerd: new containerd process, pid: 1587\"\r\ntime=\"2018-01-05T09:06:09.805879171Z\" level=warning msg=\"containerd: low RLIMIT_NOFILE changing to max\" current=1024 max=1048576\r\ntime=\"2018-01-05T09:06:10.790901720Z\" level=warning msg=\"failed to rename /hab/svc/builder-worker/data/888159704012914688/dockerd/var/lib/docker/tmp for background deletion: rename /hab/svc/builder-worker/data/888159704012914688/dockerd/var/lib/docker/tmp /hab/svc/builder-worker/data/888159704012914688/dockerd/var/lib/docker/tmp-old: no such file or directory. Deleting synchronously\"\r\ntime=\"2018-01-05T09:06:10.793746824Z\" level=error msg=\"AppArmor enabled on system but the docker-default profile could not be loaded: running `apparmor_parser apparmor_parser --version` failed with output:\r\nerror: exec: \\\"apparmor_parser\\\": executable file not found in $PATH\"\r\ntime=\"2018-01-05T09:06:12.024688467Z\" level=info msg=\"Graph migration to content-addressability took 0.00 seconds\"\r\ntime=\"2018-01-05T09:06:12.025451076Z\" level=warning msg=\"Your kernel does not support swap memory limit\"\r\ntime=\"2018-01-05T09:06:12.025565211Z\" level=warning msg=\"Your kernel does not support cgroup rt period\"\r\ntime=\"2018-01-05T09:06:12.025602010Z\" level=warning msg=\"Your kernel does not support cgroup rt runtime\"\r\ntime=\"2018-01-05T09:06:12.026884773Z\" level=info msg=\"Loading containers: start.\"\r\ntime=\"2018-01-05T09:06:12.086503769Z\" level=warning msg=\"Running modprobe nf_nat failed with message: ``, error: exec: \\\"modprobe\\\": executable file not found in $PATH\"\r\ntime=\"2018-01-05T09:06:12.086686628Z\" level=warning msg=\"Running modprobe xt_conntrack failed with message: ``, error: exec: \\\"modprobe\\\": executable file not found in $PATH\"\r\ntime=\"2018-01-05T09:06:12.095347526Z\" level=warning msg=\"Failed to create DOCKER-USER chain: Iptables not found\"\r\ntime=\"2018-01-05T09:06:12.108064234Z\" level=warning msg=\"Failed to create DOCKER-USER chain: Iptables not found\"\r\ntime=\"2018-01-05T09:06:12.109124276Z\" level=info msg=\"Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address\"\r\ntime=\"2018-01-05T09:06:12.121900452Z\" level=warning msg=\"Failed to create DOCKER-USER chain: Iptables not found\"\r\ntime=\"2018-01-05T09:06:12.121960134Z\" level=info msg=\"Loading containers: done.\"\r\ntime=\"2018-01-05T09:06:12.176709096Z\" level=info msg=\"Docker daemon\" commit=afdb6d4 graphdriver(s)=overlay2 version=17.09.0-ce\r\ntime=\"2018-01-05T09:06:12.177189834Z\" level=info msg=\"Daemon has completed initialization\"\r\ntime=\"2018-01-05T09:06:12.217401295Z\" level=info msg=\"API listen on /hab/svc/builder-worker/data/888159704012914688/dockerd/var/run/docker.sock\"\r\ntime=\"2018-01-05T09:06:28.363454302Z\" level=info msg=\"Layer sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef cleaned up\"\r\ntime=\"2018-01-05T09:06:34.837588807Z\" level=info msg=\"Layer sha256:ec3b88c0faddf0baefd51243f96b92697aa95b4f03aad930f6ea08dbde9046af cleaned up\"\r\ntime=\"2018-01-05T09:06:35.692015969Z\" level=info msg=\"Processing signal 'terminated'\"\r\ntime=\"2018-01-05T09:06:35.74920319Z\" level=info msg=\"stopping containerd after receiving terminated\"\r\n```\r\n\r\nWhat could be missing here?\r\n  \n\n---\n\n@eeyun commented on [Tue Jan 16 2018](https://github.com/habitat-sh/habitat/issues/4418#issuecomment-358059368)\n\nWe're going to sort this into accepted minor and try to validate/repro in our acceptance environment! Thank you for submitting!\n\n"},"comment":{"url":"https://api.github.com/repos/habitat-sh/builder/issues/comments/363878370","html_url":"https://github.com/habitat-sh/builder/issues/93#issuecomment-363878370","issue_url":"https://api.github.com/repos/habitat-sh/builder/issues/93","id":363878370,"user":{"login":"chefsalim","id":13542112,"avatar_url":"https://avatars2.githubusercontent.com/u/13542112?v=4","gravatar_id":"","url":"https://api.github.com/users/chefsalim","html_url":"https://github.com/chefsalim","followers_url":"https://api.github.com/users/chefsalim/followers","following_url":"https://api.github.com/users/chefsalim/following{/other_user}","gists_url":"https://api.github.com/users/chefsalim/gists{/gist_id}","starred_url":"https://api.github.com/users/chefsalim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chefsalim/subscriptions","organizations_url":"https://api.github.com/users/chefsalim/orgs","repos_url":"https://api.github.com/users/chefsalim/repos","events_url":"https://api.github.com/users/chefsalim/events{/privacy}","received_events_url":"https://api.github.com/users/chefsalim/received_events","type":"User","site_admin":false},"created_at":"2018-02-07T19:14:57Z","updated_at":"2018-02-07T19:14:57Z","author_association":"OWNER","body":"This is being tracked more explicitly in  https://app.zenhub.com/workspace/o/habitat-sh/builder/issues/159\n\nHowever, we should be surfacing up the Docker stderr to the streaming logs"}}
{"actor":{"display_login":"chefsalim"}}