{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/21555","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/21555/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/21555/comments","events_url":"https://api.github.com/repos/apache/spark/issues/21555/events","html_url":"https://github.com/apache/spark/pull/21555","id":331955654,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk0NTU2MTQ4","number":21555,"title":"[SPARK-24547][K8S] Allow for building spark on k8s docker images without cache and don't forget to push spark-py container.","user":{"login":"rayburgemeestre","id":498257,"node_id":"MDQ6VXNlcjQ5ODI1Nw==","avatar_url":"https://avatars1.githubusercontent.com/u/498257?v=4","gravatar_id":"","url":"https://api.github.com/users/rayburgemeestre","html_url":"https://github.com/rayburgemeestre","followers_url":"https://api.github.com/users/rayburgemeestre/followers","following_url":"https://api.github.com/users/rayburgemeestre/following{/other_user}","gists_url":"https://api.github.com/users/rayburgemeestre/gists{/gist_id}","starred_url":"https://api.github.com/users/rayburgemeestre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rayburgemeestre/subscriptions","organizations_url":"https://api.github.com/users/rayburgemeestre/orgs","repos_url":"https://api.github.com/users/rayburgemeestre/repos","events_url":"https://api.github.com/users/rayburgemeestre/events{/privacy}","received_events_url":"https://api.github.com/users/rayburgemeestre/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2018-06-13T11:34:35Z","updated_at":"2018-06-19T21:12:00Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/21555","html_url":"https://github.com/apache/spark/pull/21555","diff_url":"https://github.com/apache/spark/pull/21555.diff","patch_url":"https://github.com/apache/spark/pull/21555.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nhttps://issues.apache.org/jira/browse/SPARK-24547\r\n\r\nTL;DR from JIRA issue:\r\n\r\n- First time I generated images for 2.4.0 Docker was using it's cache, so actually when running jobs, old jars where still in the Docker image. This produces errors like this in the executors:\r\n\r\n`java.io.InvalidClassException: org.apache.spark.storage.BlockManagerId; local class incompatible: stream classdesc serialVersionUID = 6155820641931972169, local class serialVersionUID = -3720498261147521051`\r\n\r\n- The second problem was that the spark container is pushed, but the spark-py container wasn't yet. This was just forgotten in the initial PR.\r\n\r\n- A third problem I also ran into because I had an older docker was https://github.com/apache/spark/pull/21551 so I have not included a fix for that in this ticket.\r\n\r\n\r\n## How was this patch tested?\r\n\r\nI've tested it on my own Spark on k8s deployment.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/398547536","html_url":"https://github.com/apache/spark/pull/21555#issuecomment-398547536","issue_url":"https://api.github.com/repos/apache/spark/issues/21555","id":398547536,"node_id":"MDEyOklzc3VlQ29tbWVudDM5ODU0NzUzNg==","user":{"login":"SparkQA","id":8108735,"node_id":"MDQ6VXNlcjgxMDg3MzU=","avatar_url":"https://avatars1.githubusercontent.com/u/8108735?v=4","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2018-06-19T21:12:00Z","updated_at":"2018-06-19T21:12:00Z","author_association":"NONE","body":"Kubernetes integration test starting\nURL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/330/\n"}}
{"actor":{"display_login":"SparkQA"}}