{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/21669","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/21669/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/21669/comments","events_url":"https://api.github.com/repos/apache/spark/issues/21669/events","html_url":"https://github.com/apache/spark/pull/21669","id":336915122,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk4MjUyODQz","number":21669,"title":"[SPARK-23257][K8S] Kerberos Support for Spark on K8S","user":{"login":"ifilonenko","id":4926714,"node_id":"MDQ6VXNlcjQ5MjY3MTQ=","avatar_url":"https://avatars1.githubusercontent.com/u/4926714?v=4","gravatar_id":"","url":"https://api.github.com/users/ifilonenko","html_url":"https://github.com/ifilonenko","followers_url":"https://api.github.com/users/ifilonenko/followers","following_url":"https://api.github.com/users/ifilonenko/following{/other_user}","gists_url":"https://api.github.com/users/ifilonenko/gists{/gist_id}","starred_url":"https://api.github.com/users/ifilonenko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ifilonenko/subscriptions","organizations_url":"https://api.github.com/users/ifilonenko/orgs","repos_url":"https://api.github.com/users/ifilonenko/repos","events_url":"https://api.github.com/users/ifilonenko/events{/privacy}","received_events_url":"https://api.github.com/users/ifilonenko/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":69,"created_at":"2018-06-29T08:47:21Z","updated_at":"2018-10-09T22:19:53Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/21669","html_url":"https://github.com/apache/spark/pull/21669","diff_url":"https://github.com/apache/spark/pull/21669.diff","patch_url":"https://github.com/apache/spark/pull/21669.patch"},"body":"## What changes were proposed in this pull request?\r\nThis is the work on setting up Secure HDFS interaction with Spark-on-K8S.\r\nThe architecture is discussed in this community-wide google [doc](https://docs.google.com/document/d/1RBnXD9jMDjGonOdKJ2bA1lN4AAV_1RwpU_ewFuCNWKg)\r\nThis initiative can be broken down into 4 Stages\r\n\r\n**STAGE 1**\r\n- [x] Detecting `HADOOP_CONF_DIR` environmental variable and using Config Maps to store all Hadoop config files locally, while also setting `HADOOP_CONF_DIR` locally in the driver / executors\r\n\r\n**STAGE 2**\r\n- [x] Grabbing `TGT` from `LTC` or using keytabs+principle and creating a `DT` that will be mounted as a secret or using a pre-populated secret\r\n\r\n**STAGE 3**\r\n- [x] Driver \r\n\r\n**STAGE 4**\r\n- [x] Executor \r\n\r\n## How was this patch tested?\r\nLocally tested on a single-noded, pseudo-distributed Kerberized Hadoop Cluster\r\n- [x] E2E Integration tests https://github.com/apache/spark/pull/22608 \r\n- [ ] Unit tests\r\n\r\n## Docs and Error Handling?\r\n- [x] Docs\r\n- [x] Error Handling\r\n\r\n## Contribution Credit\r\n@kimoonkim @skonto "},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/428372953","html_url":"https://github.com/apache/spark/pull/21669#issuecomment-428372953","issue_url":"https://api.github.com/repos/apache/spark/issues/21669","id":428372953,"node_id":"MDEyOklzc3VlQ29tbWVudDQyODM3Mjk1Mw==","user":{"login":"SparkQA","id":8108735,"node_id":"MDQ6VXNlcjgxMDg3MzU=","avatar_url":"https://avatars1.githubusercontent.com/u/8108735?v=4","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2018-10-09T22:19:53Z","updated_at":"2018-10-09T22:19:53Z","author_association":"NONE","body":"**[Test build #97175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97175/testReport)** for PR 21669 at commit [`e2f8063`](https://github.com/apache/spark/commit/e2f806386dfd836d12804997c748076083977378)."}}
{"actor":{"display_login":"SparkQA"}}