{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/23846","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/23846/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/23846/comments","events_url":"https://api.github.com/repos/apache/spark/issues/23846/events","html_url":"https://github.com/apache/spark/pull/23846","id":412587367,"node_id":"MDExOlB1bGxSZXF1ZXN0MjU0NzYyMDc4","number":23846,"title":"[SPARK-26729][K8S] Make image names under test configurable","user":{"login":"rvesse","id":2104864,"node_id":"MDQ6VXNlcjIxMDQ4NjQ=","avatar_url":"https://avatars0.githubusercontent.com/u/2104864?v=4","gravatar_id":"","url":"https://api.github.com/users/rvesse","html_url":"https://github.com/rvesse","followers_url":"https://api.github.com/users/rvesse/followers","following_url":"https://api.github.com/users/rvesse/following{/other_user}","gists_url":"https://api.github.com/users/rvesse/gists{/gist_id}","starred_url":"https://api.github.com/users/rvesse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rvesse/subscriptions","organizations_url":"https://api.github.com/users/rvesse/orgs","repos_url":"https://api.github.com/users/rvesse/repos","events_url":"https://api.github.com/users/rvesse/events{/privacy}","received_events_url":"https://api.github.com/users/rvesse/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":8,"created_at":"2019-02-20T19:21:06Z","updated_at":"2019-03-05T16:11:17Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/23846","html_url":"https://github.com/apache/spark/pull/23846","diff_url":"https://github.com/apache/spark/pull/23846.diff","patch_url":"https://github.com/apache/spark/pull/23846.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nAllow specifying system properties to customise the image names for the images used in the integration testing.  Useful if your CI/CD pipeline or policy requires using a different naming format.\r\n\r\nThis is one part of addressing SPARK-26729, I plan to have a follow up patch that will also make the names configurable when using `docker-image-tool.sh`\r\n\r\n## How was this patch tested?\r\n\r\nRan integration tests against custom images generated by our CI/CD pipeline that do not follow Spark's existing hardcoded naming conventions using the new system properties to override the image names appropriately:\r\n\r\n```\r\nmvn clean integration-test -pl :spark-kubernetes-integration-tests_${SCALA_VERSION} \\\r\n            -Pkubernetes -Pkubernetes-integration-tests \\\r\n            -P${SPARK_HADOOP_PROFILE} -Dhadoop.version=${HADOOP_VERSION} \\\r\n            -Dspark.kubernetes.test.sparkTgz=${TARBALL} \\\r\n            -Dspark.kubernetes.test.imageTag=${TAG} \\\r\n            -Dspark.kubernetes.test.imageRepo=${REPO} \\\r\n            -Dspark.kubernetes.test.namespace=${K8S_NAMESPACE} \\\r\n            -Dspark.kubernetes.test.kubeConfigContext=${K8S_CONTEXT} \\\r\n            -Dspark.kubernetes.test.deployMode=${K8S_TEST_DEPLOY_MODE} \\\r\n            -Dspark.kubernetes.test.jvmImage=apache-spark \\\r\n            -Dspark.kubernetes.test.pythonImage=apache-spark-py \\\r\n            -Dspark.kubernetes.test.rImage=apache-spark-r \\\r\n            -Dtest.include.tags=k8s\r\n...\r\n[INFO] --- scalatest-maven-plugin:1.0:test (integration-test) @ spark-kubernetes-integration-tests_2.12 ---\r\nDiscovery starting.\r\nDiscovery completed in 230 milliseconds.\r\nRun starting. Expected test count is: 15\r\nKubernetesSuite:\r\n- Run SparkPi with no resources\r\n- Run SparkPi with a very long application name.\r\n- Use SparkLauncher.NO_RESOURCE\r\n- Run SparkPi with a master URL without a scheme.\r\n- Run SparkPi with an argument.\r\n- Run SparkPi with custom labels, annotations, and environment variables.\r\n- Run extraJVMOptions check on driver\r\n- Run SparkRemoteFileTest using a remote data file\r\n- Run SparkPi with env and mount secrets.\r\n- Run PySpark on simple pi.py example\r\n- Run PySpark with Python2 to test a pyfiles example\r\n- Run PySpark with Python3 to test a pyfiles example\r\n- Run PySpark with memory customization\r\n- Run in client mode.\r\n- Start pod creation from template\r\nRun completed in 8 minutes, 33 seconds.\r\nTotal number of tests run: 15\r\nSuites: completed 2, aborted 0\r\nTests: succeeded 15, failed 0, canceled 0, ignored 0, pending 0\r\nAll tests passed.\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/469740729","html_url":"https://github.com/apache/spark/pull/23846#issuecomment-469740729","issue_url":"https://api.github.com/repos/apache/spark/issues/23846","id":469740729,"node_id":"MDEyOklzc3VlQ29tbWVudDQ2OTc0MDcyOQ==","user":{"login":"SparkQA","id":8108735,"node_id":"MDQ6VXNlcjgxMDg3MzU=","avatar_url":"https://avatars1.githubusercontent.com/u/8108735?v=4","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2019-03-05T16:11:17Z","updated_at":"2019-03-05T16:11:17Z","author_association":"NONE","body":"**[Test build #103055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103055/testReport)** for PR 23846 at commit [`00fcf90`](https://github.com/apache/spark/commit/00fcf90c1cfef965df20ea576dea2231e14d7558)."}}
{"actor":{"display_login":"SparkQA"}}