{"action":"created","issue":{"url":"https://api.github.com/repos/fabric8io/gofabric8/issues/281","repository_url":"https://api.github.com/repos/fabric8io/gofabric8","labels_url":"https://api.github.com/repos/fabric8io/gofabric8/issues/281/labels{/name}","comments_url":"https://api.github.com/repos/fabric8io/gofabric8/issues/281/comments","events_url":"https://api.github.com/repos/fabric8io/gofabric8/issues/281/events","html_url":"https://github.com/fabric8io/gofabric8/issues/281","id":187849245,"number":281,"title":"fluentd Crashing","user":{"login":"jeremy-wilson","id":8602317,"avatar_url":"https://avatars.githubusercontent.com/u/8602317?v=3","gravatar_id":"","url":"https://api.github.com/users/jeremy-wilson","html_url":"https://github.com/jeremy-wilson","followers_url":"https://api.github.com/users/jeremy-wilson/followers","following_url":"https://api.github.com/users/jeremy-wilson/following{/other_user}","gists_url":"https://api.github.com/users/jeremy-wilson/gists{/gist_id}","starred_url":"https://api.github.com/users/jeremy-wilson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeremy-wilson/subscriptions","organizations_url":"https://api.github.com/users/jeremy-wilson/orgs","repos_url":"https://api.github.com/users/jeremy-wilson/repos","events_url":"https://api.github.com/users/jeremy-wilson/events{/privacy}","received_events_url":"https://api.github.com/users/jeremy-wilson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2016-11-07T22:34:10Z","updated_at":"2016-11-21T21:22:19Z","closed_at":null,"body":"I deployed the latest fabric8 release. Things are much more stable.\r\n\r\nThat said, fluentd is still crashing after installing the management components on GKE. Also, for some reason there are two fluentd pods instead of the single one I saw before.\r\n\r\n```\r\nkubectl get pods\r\nNAME                                        READY     STATUS             RESTARTS   AGE\r\nconfigmapcontroller-2401828919-krwbe        1/1       Running            0          3h\r\nelasticsearch-426176676-ojwdr               2/2       Running            0          3m\r\nexposecontroller-1126175316-pcf1e           1/1       Running            0          3h\r\nfabric8-1449330595-ixfeo                    2/2       Running            0          3h\r\nfabric8-docker-registry-4185129253-3lgzw    1/1       Running            0          3h\r\nfabric8-forge-1425626965-2hso7              1/1       Running            0          3h\r\nfluentd-iwzbd                               0/1       CrashLoopBackOff   5          3m\r\nfluentd-spk75                               0/1       CrashLoopBackOff   5          3m\r\ngogs-3048373565-6o62k                       1/1       Running            0          3h\r\ngrafana-225606179-t58x0                     1/1       Running            0          3m\r\njenkins-2297698455-8550g                    1/1       Running            0          3h\r\nkibana-3678948785-wqqei                     2/2       Running            0          3m\r\nmessage-broker-1045034239-demg6             1/1       Running            0          10m\r\nmessage-gateway-474760680-clygh             1/1       Running            0          10m\r\nnexus-4216203902-yyam6                      1/1       Running            0          3h\r\nnode-exporter-0yxge                         1/1       Running            0          3m\r\nnode-exporter-8rgzj                         1/1       Running            0          3m\r\nprometheus-3924772489-dkkk4                 2/2       Running            0          3m\r\nprometheus-blackbox-expo-1529649959-du92x   1/1       Running            0          3m\r\nzookeeper-3695684073-snpie                  1/1       Running            0          10m\r\n```\r\n\r\nHere is the dump from the fluentd pod(s)\r\n\r\n```\r\nkubectl describe pods fluentd-iwzbd\r\nName:           fluentd-iwzbd\r\nNamespace:      forge-paas-ns\r\nNode:           gke-forge-paas-default-pool-2f6c1994-iu1d/10.128.0.2\r\nStart Time:     Mon, 07 Nov 2016 15:24:24 -0700\r\nLabels:         group=io.fabric8.devops.apps\r\n                project=fluentd\r\n                provider=fabric8\r\n                version=2.2.296\r\nStatus:         Running\r\nIP:             10.0.1.11\r\nControllers:    DaemonSet/fluentd\r\nContainers:\r\n  fluentd:\r\n    Container ID:       docker://05a70fc5fa2ad70a1f707b821cb6d41e70dda48bc647521efac62017deb2721f\r\n    Image:              fabric8/fluentd-kubernetes:v1.19\r\n    Image ID:           docker://sha256:c4d9030b93687aef0c1b5ed635ce7b009c50e77692769819af3d7367cf8cc05c\r\n    Port:               24231/TCP\r\n    Limits:\r\n      cpu:      100m\r\n    Requests:\r\n      cpu:              100m\r\n    State:              Waiting\r\n      Reason:           CrashLoopBackOff\r\n    Last State:         Terminated\r\n      Reason:           ContainerCannotRun\r\n      Exit Code:        128\r\n      Started:          Mon, 07 Nov 2016 15:30:09 -0700\r\n      Finished:         Mon, 07 Nov 2016 15:30:09 -0700\r\n    Ready:              False\r\n    Restart Count:      6\r\n    Volume Mounts:\r\n      /mnt/ephemeral/docker/containers from awsdocker (ro)\r\n      /mnt/sda1/var/lib/docker/containers from minikubedocker (ro)\r\n      /var/lib/docker/containers from defaultdocker (ro)\r\n      /var/log from varlog (rw)\r\n      /var/run/secrets/kubernetes.io/serviceaccount from fluentd-token-qaddh (ro)\r\n    Environment Variables:\r\n      ELASTICSEARCH_HOST:       elasticsearch\r\n      ELASTICSEARCH_PORT:       9200\r\nConditions:\r\n  Type          Status\r\n  Initialized   True \r\n  Ready         False \r\n  PodScheduled  True \r\nVolumes:\r\n  varlog:\r\n    Type:       HostPath (bare host directory volume)\r\n    Path:       /var/log\r\n  defaultdocker:\r\n    Type:       HostPath (bare host directory volume)\r\n    Path:       /var/lib/docker/containers\r\n  awsdocker:\r\n    Type:       HostPath (bare host directory volume)\r\n    Path:       /mnt/ephemeral/docker/containers\r\n  minikubedocker:\r\n    Type:       HostPath (bare host directory volume)\r\n    Path:       /mnt/sda1/var/lib/docker/containers\r\n  fluentd-token-qaddh:\r\n    Type:       Secret (a volume populated by a Secret)\r\n    SecretName: fluentd-token-qaddh\r\nQoS Class:      Burstable\r\nTolerations:    <none>\r\nEvents:\r\n  FirstSeen     LastSeen        Count   From                                                    SubobjectPath                   Type            Reason          Message\r\n  ---------     --------        -----   ----                                                    -------------                   --------        ------          -------\r\n  7m            7m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal          Pulling         pulling image \"fabric8/fluentd-kubernetes:v1.19\"\r\n  6m            6m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal          Pulled          Successfully pulled image \"fabric8/fluentd-kubernetes:v1.19\"\r\n  6m            6m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal          Created         Created container with docker id 4a82f3ac063b; Security:[seccomp=unconfined]\r\n  6m            6m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Warning         Failed          Failed to start container with docker id 4a82f3ac063b with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  6m            6m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal          Created         Created container with docker id 25527ff66e48; Security:[seccomp=unconfined]\r\n  6m            6m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Warning         Failed          Failed to start container with docker id 25527ff66e48 with error: Error response from daemon: mkdir /mnt/sda1: read-only file system\r\n  6m            6m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}                                     Warning         FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with RunContainerError: \"runContainer: Error response from daemon: mkdir /mnt/sda1: read-only file system\"\r\n\r\n  6m    6m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id 9d0e71280a08 with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  6m    6m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal  Created         Created container with docker id 9d0e71280a08; Security:[seccomp=unconfined]\r\n  6m    6m      2       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 20s restarting failed container=fluentd pod=fluentd-iwzbd_forge-paas-ns(efd6c1c8-a538-11e6-90b0-42010a800fd0)\"\r\n\r\n  6m    6m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal  Created         Created container with docker id 1636d00bdba0; Security:[seccomp=unconfined]\r\n  6m    6m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id 1636d00bdba0 with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  5m    5m      2       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 40s restarting failed container=fluentd pod=fluentd-iwzbd_forge-paas-ns(efd6c1c8-a538-11e6-90b0-42010a800fd0)\"\r\n\r\n  5m    5m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal  Created         Created container with docker id e820f8c142db; Security:[seccomp=unconfined]\r\n  5m    5m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id e820f8c142db with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  5m    4m      7       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 1m20s restarting failed container=fluentd pod=fluentd-iwzbd_forge-paas-ns(efd6c1c8-a538-11e6-90b0-42010a800fd0)\"\r\n\r\n  4m    4m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal  Created         Created container with docker id 38194fa0310a; Security:[seccomp=unconfined]\r\n  4m    4m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id 38194fa0310a with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  3m    1m      12      {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 2m40s restarting failed container=fluentd pod=fluentd-iwzbd_forge-paas-ns(efd6c1c8-a538-11e6-90b0-42010a800fd0)\"\r\n\r\n  6m    1m      6       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal  Pulled          Container image \"fabric8/fluentd-kubernetes:v1.19\" already present on machine\r\n  6m    1m      6       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with RunContainerError: \"runContainer: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\"\r\n\r\n  1m    1m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id 05a70fc5fa2a with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  1m    1m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Normal  Created         Created container with docker id 05a70fc5fa2a; Security:[seccomp=unconfined]\r\n  6m    11s     29      {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}     spec.containers{fluentd}        Warning BackOff         Back-off restarting failed docker container\r\n  1m    11s     6       {kubelet gke-forge-paas-default-pool-2f6c1994-iu1d}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 5m0s restarting failed container=fluentd pod=fluentd-iwzbd_forge-paas-ns(efd6c1c8-a538-11e6-90b0-42010a800fd0)\"\r\n```\r\n\r\nAnd from the second pod\r\n\r\n```\r\nkubectl describe pods fluentd-spk75\r\nName:           fluentd-spk75\r\nNamespace:      forge-paas-ns\r\nNode:           gke-forge-paas-default-pool-2f6c1994-6ho0/10.128.0.3\r\nStart Time:     Mon, 07 Nov 2016 15:24:24 -0700\r\nLabels:         group=io.fabric8.devops.apps\r\n                project=fluentd\r\n                provider=fabric8\r\n                version=2.2.296\r\nStatus:         Running\r\nIP:             10.0.0.19\r\nControllers:    DaemonSet/fluentd\r\nContainers:\r\n  fluentd:\r\n    Container ID:       docker://63b6743a1e3f9b08d8b1e5ddd68eb7098951ff7817ff4434555a6188ae7e6f73\r\n    Image:              fabric8/fluentd-kubernetes:v1.19\r\n    Image ID:           docker://sha256:c4d9030b93687aef0c1b5ed635ce7b009c50e77692769819af3d7367cf8cc05c\r\n    Port:               24231/TCP\r\n    Limits:\r\n      cpu:      100m\r\n    Requests:\r\n      cpu:              100m\r\n    State:              Waiting\r\n      Reason:           CrashLoopBackOff\r\n    Last State:         Terminated\r\n      Reason:           ContainerCannotRun\r\n      Exit Code:        128\r\n      Started:          Mon, 07 Nov 2016 15:30:15 -0700\r\n      Finished:         Mon, 07 Nov 2016 15:30:15 -0700\r\n    Ready:              False\r\n    Restart Count:      6\r\n    Volume Mounts:\r\n      /mnt/ephemeral/docker/containers from awsdocker (ro)\r\n      /mnt/sda1/var/lib/docker/containers from minikubedocker (ro)\r\n      /var/lib/docker/containers from defaultdocker (ro)\r\n      /var/log from varlog (rw)\r\n      /var/run/secrets/kubernetes.io/serviceaccount from fluentd-token-qaddh (ro)\r\n    Environment Variables:\r\n      ELASTICSEARCH_HOST:       elasticsearch\r\n      ELASTICSEARCH_PORT:       9200\r\nConditions:\r\n  Type          Status\r\n  Initialized   True \r\n  Ready         False \r\n  PodScheduled  True \r\nVolumes:\r\n  varlog:\r\n    Type:       HostPath (bare host directory volume)\r\n    Path:       /var/log\r\n  defaultdocker:\r\n    Type:       HostPath (bare host directory volume)\r\n    Path:       /var/lib/docker/containers\r\n  awsdocker:\r\n    Type:       HostPath (bare host directory volume)\r\n    Path:       /mnt/ephemeral/docker/containers\r\n  minikubedocker:\r\n    Type:       HostPath (bare host directory volume)\r\n    Path:       /mnt/sda1/var/lib/docker/containers\r\n  fluentd-token-qaddh:\r\n    Type:       Secret (a volume populated by a Secret)\r\n    SecretName: fluentd-token-qaddh\r\nQoS Class:      Burstable\r\nTolerations:    <none>\r\nEvents:\r\n  FirstSeen     LastSeen        Count   From                                                    SubobjectPath                   Type            Reason          Message\r\n  ---------     --------        -----   ----                                                    -------------                   --------        ------          -------\r\n  8m            8m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal          Pulling         pulling image \"fabric8/fluentd-kubernetes:v1.19\"\r\n  8m            8m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal          Pulled          Successfully pulled image \"fabric8/fluentd-kubernetes:v1.19\"\r\n  8m            8m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal          Created         Created container with docker id dce2709cd6b3; Security:[seccomp=unconfined]\r\n  8m            8m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Warning         Failed          Failed to start container with docker id dce2709cd6b3 with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  8m            8m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal          Created         Created container with docker id 912c7d020859; Security:[seccomp=unconfined]\r\n  8m            8m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Warning         Failed          Failed to start container with docker id 912c7d020859 with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  8m            8m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal          Created         Created container with docker id 6b2b738ce354; Security:[seccomp=unconfined]\r\n  8m            8m              1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Warning         Failed          Failed to start container with docker id 6b2b738ce354 with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  8m            8m              3       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}                                     Warning         FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 20s restarting failed container=fluentd pod=fluentd-spk75_forge-paas-ns(efd6a1ed-a538-11e6-90b0-42010a800fd0)\"\r\n\r\n  7m    7m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id 2e508583849a with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  7m    7m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal  Created         Created container with docker id 2e508583849a; Security:[seccomp=unconfined]\r\n  7m    7m      3       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 40s restarting failed container=fluentd pod=fluentd-spk75_forge-paas-ns(efd6a1ed-a538-11e6-90b0-42010a800fd0)\"\r\n\r\n  7m    7m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal  Created         Created container with docker id 5d51a9b0b79d; Security:[seccomp=unconfined]\r\n  7m    7m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id 5d51a9b0b79d with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  6m    5m      6       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 1m20s restarting failed container=fluentd pod=fluentd-spk75_forge-paas-ns(efd6a1ed-a538-11e6-90b0-42010a800fd0)\"\r\n\r\n  5m    5m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal  Created         Created container with docker id fc1fdbb29b05; Security:[seccomp=unconfined]\r\n  5m    5m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id fc1fdbb29b05 with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  5m    3m      12      {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 2m40s restarting failed container=fluentd pod=fluentd-spk75_forge-paas-ns(efd6a1ed-a538-11e6-90b0-42010a800fd0)\"\r\n\r\n  2m    2m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Warning Failed          Failed to start container with docker id 63b6743a1e3f with error: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\r\n  8m    2m      6       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal  Pulled          Container image \"fabric8/fluentd-kubernetes:v1.19\" already present on machine\r\n  8m    2m      7       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with RunContainerError: \"runContainer: Error response from daemon: mkdir /mnt/ephemeral: read-only file system\"\r\n\r\n  2m    2m      1       {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Normal  Created         Created container with docker id 63b6743a1e3f; Security:[seccomp=unconfined]\r\n  8m    0s      38      {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}     spec.containers{fluentd}        Warning BackOff         Back-off restarting failed docker container\r\n  2m    0s      14      {kubelet gke-forge-paas-default-pool-2f6c1994-6ho0}                                     Warning FailedSync      Error syncing pod, skipping: failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"Back-off 5m0s restarting failed container=fluentd pod=fluentd-spk75_forge-paas-ns(efd6a1ed-a538-11e6-90b0-42010a800fd0)\"\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/fabric8io/gofabric8/issues/comments/262071132","html_url":"https://github.com/fabric8io/gofabric8/issues/281#issuecomment-262071132","issue_url":"https://api.github.com/repos/fabric8io/gofabric8/issues/281","id":262071132,"user":{"login":"jeremy-wilson","id":8602317,"avatar_url":"https://avatars.githubusercontent.com/u/8602317?v=3","gravatar_id":"","url":"https://api.github.com/users/jeremy-wilson","html_url":"https://github.com/jeremy-wilson","followers_url":"https://api.github.com/users/jeremy-wilson/followers","following_url":"https://api.github.com/users/jeremy-wilson/following{/other_user}","gists_url":"https://api.github.com/users/jeremy-wilson/gists{/gist_id}","starred_url":"https://api.github.com/users/jeremy-wilson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeremy-wilson/subscriptions","organizations_url":"https://api.github.com/users/jeremy-wilson/orgs","repos_url":"https://api.github.com/users/jeremy-wilson/repos","events_url":"https://api.github.com/users/jeremy-wilson/events{/privacy}","received_events_url":"https://api.github.com/users/jeremy-wilson/received_events","type":"User","site_admin":false},"created_at":"2016-11-21T21:22:19Z","updated_at":"2016-11-21T21:22:19Z","body":"Ok, so I fixed the issue. I edited the daemon set as recommended. But I needed to change the volumes and volume mounts as follows in the daemon set:\r\n\r\n```\r\n        volumeMounts:\r\n        - mountPath: /var/log\r\n          name: varlog\r\n        - mountPath: /var/lib/docker/containers\r\n          name: defaultdocker\r\n          readOnly: true\r\n        - mountPath: /var/log/journal\r\n          name: journaldir\r\n        - mountPath: /host/lib\r\n          name: libsystemddir\r\n```\r\n\r\nAnd...\r\n\r\n```\r\n      volumes:\r\n      - hostPath:\r\n          path: /var/log\r\n        name: varlog\r\n      - hostPath:\r\n          path: /var/lib/docker/containers\r\n        name: defaultdocker\r\n      - hostPath:\r\n          path: /var/log/journal\r\n        name: journaldir\r\n      - hostPath:\r\n          path: /usr/lib64\r\n        name: libsystemddir\r\n```\r\n\r\nAfter changing that in the daemon set, the pods come up after I delete them and they auto create again."}}
{"actor":{"display_login":"jeremy-wilson"}}