{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/20909","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/20909/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/20909/comments","events_url":"https://api.github.com/repos/apache/spark/issues/20909/events","html_url":"https://github.com/apache/spark/pull/20909","id":308763214,"node_id":"MDExOlB1bGxSZXF1ZXN0MTc3NTg5Mzk4","number":20909,"title":"[SPARK-23776][python][test] Check for needed components/files before running pyspark-sql tests","user":{"login":"bersprockets","id":21131848,"node_id":"MDQ6VXNlcjIxMTMxODQ4","avatar_url":"https://avatars2.githubusercontent.com/u/21131848?v=4","gravatar_id":"","url":"https://api.github.com/users/bersprockets","html_url":"https://github.com/bersprockets","followers_url":"https://api.github.com/users/bersprockets/followers","following_url":"https://api.github.com/users/bersprockets/following{/other_user}","gists_url":"https://api.github.com/users/bersprockets/gists{/gist_id}","starred_url":"https://api.github.com/users/bersprockets/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bersprockets/subscriptions","organizations_url":"https://api.github.com/users/bersprockets/orgs","repos_url":"https://api.github.com/users/bersprockets/repos","events_url":"https://api.github.com/users/bersprockets/events{/privacy}","received_events_url":"https://api.github.com/users/bersprockets/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":17,"created_at":"2018-03-26T22:54:15Z","updated_at":"2018-06-09T14:08:15Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/20909","html_url":"https://github.com/apache/spark/pull/20909","diff_url":"https://github.com/apache/spark/pull/20909.diff","patch_url":"https://github.com/apache/spark/pull/20909.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nChange pyspark-sql tests to check the following:\r\n- Spark was built with the Hive profile\r\n- Spark scala tests were compiled\r\n\r\nIf either condition is not met, throw an exception with a message explaining how to appropriately build Spark.\r\n\r\nThese checks are similar to the ones found in the pyspark-streaming tests.\r\n\r\nThese required files will be missing if you follow the sbt build instructions. They are less likely to be missing if you follow the mvn build instructions (mvn compiles the test scala files, and there are mvn build instructions for running the pyspark tests).\r\n\r\n## How was this patch tested?\r\n\r\nFor sbt build:\r\n- run ./build/sbt package\r\n- run python/run-tests --modules \"pyspark-sql\" --python-executables python2.7\r\n- see failure, follow sbt instructions in exception message\r\n- run test again\r\n- see second failure (sbt only), follow sbt instructions in exception message\r\n- run test again, verify success\r\n- repeat for python3.4\r\n\r\nFor mvn build:\r\n- run ./build/mvn -DskipTests clean package\r\n- run python/run-tests --modules \"pyspark-sql\" --python-executables python2.7\r\n- see failure, follow mvn instructions in exception message\r\n- run test again, verify success\r\n- repeat for python3.4\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/395972051","html_url":"https://github.com/apache/spark/pull/20909#issuecomment-395972051","issue_url":"https://api.github.com/repos/apache/spark/issues/20909","id":395972051,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NTk3MjA1MQ==","user":{"login":"SparkQA","id":8108735,"node_id":"MDQ6VXNlcjgxMDg3MzU=","avatar_url":"https://avatars1.githubusercontent.com/u/8108735?v=4","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2018-06-09T14:08:15Z","updated_at":"2018-06-09T14:08:15Z","author_association":"NONE","body":"**[Test build #91612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91612/testReport)** for PR 20909 at commit [`db14acb`](https://github.com/apache/spark/commit/db14acbb3a90c9da184fc9c909640e07100c38fa).\n * This patch **fails from timeout after a configured wait of \\`300m\\`**.\n * This patch **does not merge cleanly**.\n * This patch adds no public classes."}}
{"actor":{"display_login":"SparkQA"}}