{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15725","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15725/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15725/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15725/events","html_url":"https://github.com/apache/spark/pull/15725","id":186692912,"number":15725,"title":"[SPARK-18167] [DO NOT MERGE] Print out spark confs, and hive confs when SQLQuerySuite fails","user":{"login":"ericl","id":14922,"avatar_url":"https://avatars.githubusercontent.com/u/14922?v=3","gravatar_id":"","url":"https://api.github.com/users/ericl","html_url":"https://github.com/ericl","followers_url":"https://api.github.com/users/ericl/followers","following_url":"https://api.github.com/users/ericl/following{/other_user}","gists_url":"https://api.github.com/users/ericl/gists{/gist_id}","starred_url":"https://api.github.com/users/ericl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ericl/subscriptions","organizations_url":"https://api.github.com/users/ericl/orgs","repos_url":"https://api.github.com/users/ericl/repos","events_url":"https://api.github.com/users/ericl/events{/privacy}","received_events_url":"https://api.github.com/users/ericl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":24,"created_at":"2016-11-02T01:49:07Z","updated_at":"2016-11-02T23:03:55Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15725","html_url":"https://github.com/apache/spark/pull/15725","diff_url":"https://github.com/apache/spark/pull/15725.diff","patch_url":"https://github.com/apache/spark/pull/15725.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nIt seems the proximate cause of the test failures is that `cast(str as decimal)` in derby will raise an exception instead of returning NULL. This is a problem since Hive sometimes inserts `__HIVE_DEFAULT_PARTITION__` entries into the partition table as documented here: https://github.com/apache/hive/blob/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1034\r\n\r\nBasically, when these special default partitions are present, partition pruning pushdown using the SQL-direct mode will fail due this cast exception. As commented on in `MetaStoreDirectSql.java` above, this is normally fine since Hive falls back to JDO pruning, however when the pruning predicate contains an unsupported operator such as `>`, that will fail as well.\r\n\r\nThe only remaining question is why this behavior is nondeterministic. We know that when the test flakes, retries do not help, therefore the cause must be environmental. The current best hypothesis is that some config is different between different jenkins runs, which is why this PR prints out the Spark SQL and Hive confs for the test. The hope is that by comparing the config state for failure vs success we can isolate the root cause of the flakiness.\r\n\r\n## How was this patch tested?\r\n\r\nI verified the confs are printed out in a readable form locally."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/258025872","html_url":"https://github.com/apache/spark/pull/15725#issuecomment-258025872","issue_url":"https://api.github.com/repos/apache/spark/issues/15725","id":258025872,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2016-11-02T23:03:55Z","updated_at":"2016-11-02T23:03:55Z","body":"Merged build finished. Test FAILed."}}
{"actor":{"display_login":"AmplabJenkins"}}