{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/65835","repository_url":"https://api.github.com/repos/kubernetes/kubernetes","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/65835/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/65835/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/65835/events","html_url":"https://github.com/kubernetes/kubernetes/issues/65835","id":338344938,"node_id":"MDU6SXNzdWUzMzgzNDQ5Mzg=","number":65835,"title":"pod crashes if /run/secrets is used as a mount target","user":{"login":"trondhindenes","id":1747120,"node_id":"MDQ6VXNlcjE3NDcxMjA=","avatar_url":"https://avatars0.githubusercontent.com/u/1747120?v=4","gravatar_id":"","url":"https://api.github.com/users/trondhindenes","html_url":"https://github.com/trondhindenes","followers_url":"https://api.github.com/users/trondhindenes/followers","following_url":"https://api.github.com/users/trondhindenes/following{/other_user}","gists_url":"https://api.github.com/users/trondhindenes/gists{/gist_id}","starred_url":"https://api.github.com/users/trondhindenes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/trondhindenes/subscriptions","organizations_url":"https://api.github.com/users/trondhindenes/orgs","repos_url":"https://api.github.com/users/trondhindenes/repos","events_url":"https://api.github.com/users/trondhindenes/events{/privacy}","received_events_url":"https://api.github.com/users/trondhindenes/received_events","type":"User","site_admin":false},"labels":[{"id":105146071,"node_id":"MDU6TGFiZWwxMDUxNDYwNzE=","url":"https://api.github.com/repos/kubernetes/kubernetes/labels/kind/bug","name":"kind/bug","color":"e11d21","default":false},{"id":173493665,"node_id":"MDU6TGFiZWwxNzM0OTM2NjU=","url":"https://api.github.com/repos/kubernetes/kubernetes/labels/sig/node","name":"sig/node","color":"d2b48c","default":false},{"id":169428334,"node_id":"MDU6TGFiZWwxNjk0MjgzMzQ=","url":"https://api.github.com/repos/kubernetes/kubernetes/labels/sig/storage","name":"sig/storage","color":"d2b48c","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2018-07-04T17:37:43Z","updated_at":"2018-08-08T13:36:05Z","closed_at":null,"author_association":"NONE","body":"/kind bug\r\n\r\n\r\n**What happened**:\r\nAfter upgrading from 1.9.0 to 1.10.2 / 1.10.5 (tried both), we started to get multiple containers in crash loop. This turned out to be caused by the fact that we usually map Kubernetes secrets to the container path `/run/secrets` and use company-internal libraries to populate application config from these secrets. I don't know what the change is from 1.9.0 to 1.10.x (the order of which the mounts are happening maybe?)\r\n\r\n**What you expected to happen**:\r\nThe pod should start\r\n\r\n**How to reproduce it (as minimally and precisely as possible)**:\r\ncreate a Secret, mount it into `/run/secrets` on a pod\r\n\r\n**Environment**:\r\n- Kubernetes version (use `kubectl version`): v1.10.2\r\n- Cloud provider or hardware configuration: aws (custom)\r\n- OS (e.g. from /etc/os-release): Ubuntu 16.04.4 LTS\r\n- Kernel (e.g. `uname -a`): 4.4.0-1057-aws\r\n- Install tools: custom\r\n- Others: docker://1.13.1\r\n\r\n\r\nlog output from `kubectl logs`:\r\n```\r\ncontainer_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:54: mounting \\\\\\\"/var/lib/kubelet/pods/63a4bfea-7fa3-11e8-9f7b-06589ff22a28/volumes/kubernetes.io~secret/ansiblejobservice-token-dsbdf\\\\\\\" to rootfs \\\\\\\"/dockerdata/overlay2/ece45c15fd86a5676d99ac0dae3559f7bda882d7f7b0f9862d572f2ad52949a3/merged\\\\\\\" at \\\\\\\"/dockerdata/overlay2/ece45c15fd86a5676d99ac0dae3559f7bda882d7f7b0f9862d572f2ad52949a3/merged/run/secrets/kubernetes.io/serviceaccount\\\\\\\" caused \\\\\\\"mkdir /dockerdata/overlay2/ece45c15fd86a5676d99ac0dae3559f7bda882d7f7b0f9862d572f2ad52949a3/merged/run/secrets/kubernetes.io: read-only file system\\\\\\\"\\\"\"\r\n```"},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/411407583","html_url":"https://github.com/kubernetes/kubernetes/issues/65835#issuecomment-411407583","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/65835","id":411407583,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMTQwNzU4Mw==","user":{"login":"AntonFriberg","id":14290517,"node_id":"MDQ6VXNlcjE0MjkwNTE3","avatar_url":"https://avatars1.githubusercontent.com/u/14290517?v=4","gravatar_id":"","url":"https://api.github.com/users/AntonFriberg","html_url":"https://github.com/AntonFriberg","followers_url":"https://api.github.com/users/AntonFriberg/followers","following_url":"https://api.github.com/users/AntonFriberg/following{/other_user}","gists_url":"https://api.github.com/users/AntonFriberg/gists{/gist_id}","starred_url":"https://api.github.com/users/AntonFriberg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AntonFriberg/subscriptions","organizations_url":"https://api.github.com/users/AntonFriberg/orgs","repos_url":"https://api.github.com/users/AntonFriberg/repos","events_url":"https://api.github.com/users/AntonFriberg/events{/privacy}","received_events_url":"https://api.github.com/users/AntonFriberg/received_events","type":"User","site_admin":false},"created_at":"2018-08-08T13:36:05Z","updated_at":"2018-08-08T13:36:05Z","author_association":"NONE","body":"Ok I have found a workaround using \"subPath\"\r\n\r\n```yaml\r\napiVersion: v1\r\nkind: Secret\r\nmetadata:\r\n  name: test-secret\r\ndata:\r\n  test: TS1oeUtKYkNrQWdZYlBZSGlnWF9kcWRNZUFiME9IdWlxN0xQTjFHWG9oUQo=\r\ntype: Opaque\r\n---\r\napiVersion: apps/v1\r\nkind: Deployment\r\nmetadata:\r\n  name: test\r\nspec:\r\n  selector:\r\n    matchLabels:\r\n      app: test-nginx\r\n  replicas: 1\r\n  template:\r\n    metadata:\r\n      labels:\r\n        app: test-nginx\r\n    spec:\r\n      containers:\r\n      - name: nginx\r\n        image: nginx\r\n        imagePullPolicy: Always\r\n        volumeMounts:\r\n        - name: secret-volume\r\n          mountPath: \"/run/secrets/tmp\"\r\n          subPath: ..secrets\r\n      volumes:\r\n      - name: secret-volume\r\n        secret:\r\n          secretName: test-secret\r\n          items:\r\n          - key: test\r\n            path: \"secrets/test\"\r\n```"}}
{"actor":{"display_login":"AntonFriberg"}}