{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/16047","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/16047/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/16047/comments","events_url":"https://api.github.com/repos/apache/spark/issues/16047/events","html_url":"https://github.com/apache/spark/pull/16047","id":192146938,"number":16047,"title":"[SPARK-17783] [SQL] [BACKPORT-2.0] Hide Credentials in CREATE and DESC FORMATTED/EXTENDED a PERSISTENT/TEMP Table for JDBC ","user":{"login":"gatorsmile","id":11567269,"avatar_url":"https://avatars.githubusercontent.com/u/11567269?v=3","gravatar_id":"","url":"https://api.github.com/users/gatorsmile","html_url":"https://github.com/gatorsmile","followers_url":"https://api.github.com/users/gatorsmile/followers","following_url":"https://api.github.com/users/gatorsmile/following{/other_user}","gists_url":"https://api.github.com/users/gatorsmile/gists{/gist_id}","starred_url":"https://api.github.com/users/gatorsmile/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gatorsmile/subscriptions","organizations_url":"https://api.github.com/users/gatorsmile/orgs","repos_url":"https://api.github.com/users/gatorsmile/repos","events_url":"https://api.github.com/users/gatorsmile/events{/privacy}","received_events_url":"https://api.github.com/users/gatorsmile/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-11-28T23:31:54Z","updated_at":"2016-11-29T10:58:28Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/16047","html_url":"https://github.com/apache/spark/pull/16047","diff_url":"https://github.com/apache/spark/pull/16047.diff","patch_url":"https://github.com/apache/spark/pull/16047.patch"},"body":"### What changes were proposed in this pull request?\r\n\r\nThis PR is to backport https://github.com/apache/spark/pull/15358 to Spark 2.0. \r\n\r\n------ \r\n\r\nWe should never expose the Credentials in the EXPLAIN and DESC FORMATTED/EXTENDED command. However, below commands exposed the credentials.\r\n\r\nIn the related PR: https://github.com/apache/spark/pull/10452\r\n\r\n> URL patterns to specify credential seems to be vary between different databases. \r\n\r\nThus, we hide the whole `url` value if it contains the keyword `password`. We also hide the `password` property.\r\n\r\nBefore the fix, the command outputs look like:\r\n\r\n``` SQL\r\nCREATE TABLE tab1\r\nUSING org.apache.spark.sql.jdbc\r\nOPTIONS (\r\n url 'jdbc:h2:mem:testdb0;user=testUser;password=testPass',\r\n dbtable 'TEST.PEOPLE',\r\n user 'testUser',\r\n password '$password')\r\n\r\nDESC FORMATTED tab1\r\nDESC EXTENDED tab1\r\n```\r\n\r\n\r\nBefore the fix,\r\n- The output of SQL statement EXPLAIN \r\n```\r\n== Physical Plan ==\r\nExecutedCommand\r\n   +- CreateDataSourceTableCommand CatalogTable(\r\n\tTable: `tab1`\r\n\tCreated: Wed Nov 16 23:00:10 PST 2016\r\n\tLast Access: Wed Dec 31 15:59:59 PST 1969\r\n\tType: MANAGED\r\n\tProvider: org.apache.spark.sql.jdbc\r\n\tStorage(Properties: [url=jdbc:h2:mem:testdb0;user=testUser;password=testPass, dbtable=TEST.PEOPLE, user=testUser, password=testPass])), false\r\n```\r\n\r\n- The output of `DESC FORMATTED`\r\n```\r\n...\r\n|Storage Desc Parameters:    |                                                                  |       |\r\n|  url                       |jdbc:h2:mem:testdb0;user=testUser;password=testPass               |       |\r\n|  dbtable                   |TEST.PEOPLE                                                       |       |\r\n|  user                      |testUser                                                          |       |\r\n|  password                  |testPass                                                          |       |\r\n+----------------------------+------------------------------------------------------------------+-------+\r\n```\r\n\r\n- The output of `DESC EXTENDED`\r\n```\r\n|# Detailed Table Information|CatalogTable(\r\n\tTable: `default`.`tab1`\r\n\tCreated: Wed Nov 16 23:00:10 PST 2016\r\n\tLast Access: Wed Dec 31 15:59:59 PST 1969\r\n\tType: MANAGED\r\n\tSchema: [StructField(NAME,StringType,false), StructField(THEID,IntegerType,false)]\r\n\tProvider: org.apache.spark.sql.jdbc\r\n\tStorage(Location: file:/Users/xiaoli/IdeaProjects/sparkDelivery/spark-warehouse/tab1, Properties: [url=jdbc:h2:mem:testdb0;user=testUser;password=testPass, dbtable=TEST.PEOPLE, user=testUser, password=testPass]))|       |\r\n```\r\n\r\nAfter the fix, \r\n- The output of SQL statement EXPLAIN \r\n```\r\n== Physical Plan ==\r\nExecutedCommand\r\n   +- CreateDataSourceTableCommand CatalogTable(\r\n\tTable: `tab1`\r\n\tCreated: Wed Nov 16 22:43:49 PST 2016\r\n\tLast Access: Wed Dec 31 15:59:59 PST 1969\r\n\tType: MANAGED\r\n\tProvider: org.apache.spark.sql.jdbc\r\n\tStorage(Properties: [url=###, dbtable=TEST.PEOPLE, user=testUser, password=###])), false\r\n```\r\n- The output of `DESC FORMATTED`\r\n```\r\n...\r\n|Storage Desc Parameters:    |                                                                  |       |\r\n|  url                       |###                                                               |       |\r\n|  dbtable                   |TEST.PEOPLE                                                       |       |\r\n|  user                      |testUser                                                          |       |\r\n|  password                  |###                                                               |       |\r\n+----------------------------+------------------------------------------------------------------+-------+\r\n```\r\n\r\n- The output of `DESC EXTENDED`\r\n```\r\n|# Detailed Table Information|CatalogTable(\r\n\tTable: `default`.`tab1`\r\n\tCreated: Wed Nov 16 22:43:49 PST 2016\r\n\tLast Access: Wed Dec 31 15:59:59 PST 1969\r\n\tType: MANAGED\r\n\tSchema: [StructField(NAME,StringType,false), StructField(THEID,IntegerType,false)]\r\n\tProvider: org.apache.spark.sql.jdbc\r\n\tStorage(Location: file:/Users/xiaoli/IdeaProjects/sparkDelivery/spark-warehouse/tab1, Properties: [url=###, dbtable=TEST.PEOPLE, user=testUser, password=###]))|       |\r\n```\r\n\r\n### How was this patch tested?\r\n\r\nAdded test cases"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/263540188","html_url":"https://github.com/apache/spark/pull/16047#issuecomment-263540188","issue_url":"https://api.github.com/repos/apache/spark/issues/16047","id":263540188,"user":{"login":"hvanhovell","id":9616802,"avatar_url":"https://avatars.githubusercontent.com/u/9616802?v=3","gravatar_id":"","url":"https://api.github.com/users/hvanhovell","html_url":"https://github.com/hvanhovell","followers_url":"https://api.github.com/users/hvanhovell/followers","following_url":"https://api.github.com/users/hvanhovell/following{/other_user}","gists_url":"https://api.github.com/users/hvanhovell/gists{/gist_id}","starred_url":"https://api.github.com/users/hvanhovell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hvanhovell/subscriptions","organizations_url":"https://api.github.com/users/hvanhovell/orgs","repos_url":"https://api.github.com/users/hvanhovell/repos","events_url":"https://api.github.com/users/hvanhovell/events{/privacy}","received_events_url":"https://api.github.com/users/hvanhovell/received_events","type":"User","site_admin":false},"created_at":"2016-11-29T10:58:28Z","updated_at":"2016-11-29T10:58:28Z","body":"LGTM - merging to 2.0. Thanks!\r\n\r\nCan you close?"}}
{"actor":{"display_login":"hvanhovell"}}