{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/23017","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/23017/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/23017/comments","events_url":"https://api.github.com/repos/apache/spark/issues/23017/events","html_url":"https://github.com/apache/spark/pull/23017","id":379871748,"node_id":"MDExOlB1bGxSZXF1ZXN0MjMwMjA2OTU0","number":23017,"title":"[WIP][SPARK-26015][K8S] Set a default UID for Spark on K8S Images","user":{"login":"rvesse","id":2104864,"node_id":"MDQ6VXNlcjIxMDQ4NjQ=","avatar_url":"https://avatars0.githubusercontent.com/u/2104864?v=4","gravatar_id":"","url":"https://api.github.com/users/rvesse","html_url":"https://github.com/rvesse","followers_url":"https://api.github.com/users/rvesse/followers","following_url":"https://api.github.com/users/rvesse/following{/other_user}","gists_url":"https://api.github.com/users/rvesse/gists{/gist_id}","starred_url":"https://api.github.com/users/rvesse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rvesse/subscriptions","organizations_url":"https://api.github.com/users/rvesse/orgs","repos_url":"https://api.github.com/users/rvesse/repos","events_url":"https://api.github.com/users/rvesse/events{/privacy}","received_events_url":"https://api.github.com/users/rvesse/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-11-12T17:07:47Z","updated_at":"2018-11-13T17:57:48Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/23017","html_url":"https://github.com/apache/spark/pull/23017","diff_url":"https://github.com/apache/spark/pull/23017.diff","patch_url":"https://github.com/apache/spark/pull/23017.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nAdds USER directives to the Dockerfiles which is configurable via build argument (`spark_uid`) for easy customisation.  A `-u` flag is added to `bin/docker-image-tool.sh` to make it easy to customise this e.g.\r\n\r\n```\r\n> bin/docker-image-tool.sh -r rvesse -t uid -u 185 build\r\n> bin/docker-image-tool.sh -r rvesse -t uid push\r\n```\r\n\r\nIf no UID is explicitly specified it defaults to `185` - this is per @skonto's suggestion to align with the OpenShift standard reserved UID for Java apps ( \r\nhttps://lists.openshift.redhat.com/openshift-archives/users/2016-March/msg00283.html)\r\n\r\nNotes:\r\n- We have to make the `WORKDIR` writable by the root group or otherwise jobs will fail with `AccessDeniedException`\r\n\r\nTo Do:\r\n- [ ] Debug and resolve issue with client mode test\r\n- [ ] Consider whether to always propagate `SPARK_USER_NAME` to environment of driver and executor pods so `entrypoint.sh` can insert that into `/etc/passwd` entry\r\n- [ ] Rebase once PR #23013 is merged and update documentation accordingly\r\n\r\n## How was this patch tested?\r\n\r\nBuilt the Docker images with the new Dockerfiles that include the `USER` directives.  Ran the Spark on K8S integration tests against the new images.  All pass except client mode which I am currently debugging further.\r\n\r\nAlso manually dropped myself into the resulting container images via `docker run` and checked `id -u` output to see that UID is as expected.\r\n\r\nTried customising the UID from the default via the new `-u` argument to `docker-image-tool.sh` and again checked the resulting image for the correct runtime UID.\r\n\r\ncc @felixcheung @skonto @vanzin "},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/438373379","html_url":"https://github.com/apache/spark/pull/23017#issuecomment-438373379","issue_url":"https://api.github.com/repos/apache/spark/issues/23017","id":438373379,"node_id":"MDEyOklzc3VlQ29tbWVudDQzODM3MzM3OQ==","user":{"login":"felixcheung","id":8969467,"node_id":"MDQ6VXNlcjg5Njk0Njc=","avatar_url":"https://avatars3.githubusercontent.com/u/8969467?v=4","gravatar_id":"","url":"https://api.github.com/users/felixcheung","html_url":"https://github.com/felixcheung","followers_url":"https://api.github.com/users/felixcheung/followers","following_url":"https://api.github.com/users/felixcheung/following{/other_user}","gists_url":"https://api.github.com/users/felixcheung/gists{/gist_id}","starred_url":"https://api.github.com/users/felixcheung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/felixcheung/subscriptions","organizations_url":"https://api.github.com/users/felixcheung/orgs","repos_url":"https://api.github.com/users/felixcheung/repos","events_url":"https://api.github.com/users/felixcheung/events{/privacy}","received_events_url":"https://api.github.com/users/felixcheung/received_events","type":"User","site_admin":false},"created_at":"2018-11-13T17:57:48Z","updated_at":"2018-11-13T17:57:48Z","author_association":"MEMBER","body":"noted test issue. let's kick off test though"}}
{"actor":{"display_login":"felixcheung"}}