{"push_id":1425837318,"size":1,"distinct_size":1,"ref":"refs/heads/branch-2.1","head":"a9d4febe900aa3eb9c595089e7283a64a24c8761","before":"e449f7546897c5f29075e6a0913a5a6106bcbb5f","commits":[{"sha":"a9d4febe900aa3eb9c595089e7283a64a24c8761","author":{"name":"gatorsmile","email":"ea6589371ad7a107e4268706d7d8403e3ba585bb@gmail.com"},"message":"[SPARK-17783][SQL] Hide Credentials in CREATE and DESC FORMATTED/EXTENDED a PERSISTENT/TEMP Table for JDBC\n\n### What changes were proposed in this pull request?\n\nWe should never expose the Credentials in the EXPLAIN and DESC FORMATTED/EXTENDED command. However, below commands exposed the credentials.\n\nIn the related PR: https://github.com/apache/spark/pull/10452\n\n> URL patterns to specify credential seems to be vary between different databases.\n\nThus, we hide the whole `url` value if it contains the keyword `password`. We also hide the `password` property.\n\nBefore the fix, the command outputs look like:\n\n``` SQL\nCREATE TABLE tab1\nUSING org.apache.spark.sql.jdbc\nOPTIONS (\n url 'jdbc:h2:mem:testdb0;user=testUser;password=testPass',\n dbtable 'TEST.PEOPLE',\n user 'testUser',\n password '$password')\n\nDESC FORMATTED tab1\nDESC EXTENDED tab1\n```\n\nBefore the fix,\n- The output of SQL statement EXPLAIN\n```\n== Physical Plan ==\nExecutedCommand\n   +- CreateDataSourceTableCommand CatalogTable(\n\tTable: `tab1`\n\tCreated: Wed Nov 16 23:00:10 PST 2016\n\tLast Access: Wed Dec 31 15:59:59 PST 1969\n\tType: MANAGED\n\tProvider: org.apache.spark.sql.jdbc\n\tStorage(Properties: [url=jdbc:h2:mem:testdb0;user=testUser;password=testPass, dbtable=TEST.PEOPLE, user=testUser, password=testPass])), false\n```\n\n- The output of `DESC FORMATTED`\n```\n...\n|Storage Desc Parameters:    |                                                                  |       |\n|  url                       |jdbc:h2:mem:testdb0;user=testUser;password=testPass               |       |\n|  dbtable                   |TEST.PEOPLE                                                       |       |\n|  user                      |testUser                                                          |       |\n|  password                  |testPass                                                          |       |\n+----------------------------+------------------------------------------------------------------+-------+\n```\n\n- The output of `DESC EXTENDED`\n```\n|# Detailed Table Information|CatalogTable(\n\tTable: `default`.`tab1`\n\tCreated: Wed Nov 16 23:00:10 PST 2016\n\tLast Access: Wed Dec 31 15:59:59 PST 1969\n\tType: MANAGED\n\tSchema: [StructField(NAME,StringType,false), StructField(THEID,IntegerType,false)]\n\tProvider: org.apache.spark.sql.jdbc\n\tStorage(Location: file:/Users/xiaoli/IdeaProjects/sparkDelivery/spark-warehouse/tab1, Properties: [url=jdbc:h2:mem:testdb0;user=testUser;password=testPass, dbtable=TEST.PEOPLE, user=testUser, password=testPass]))|       |\n```\n\nAfter the fix,\n- The output of SQL statement EXPLAIN\n```\n== Physical Plan ==\nExecutedCommand\n   +- CreateDataSourceTableCommand CatalogTable(\n\tTable: `tab1`\n\tCreated: Wed Nov 16 22:43:49 PST 2016\n\tLast Access: Wed Dec 31 15:59:59 PST 1969\n\tType: MANAGED\n\tProvider: org.apache.spark.sql.jdbc\n\tStorage(Properties: [url=###, dbtable=TEST.PEOPLE, user=testUser, password=###])), false\n```\n- The output of `DESC FORMATTED`\n```\n...\n|Storage Desc Parameters:    |                                                                  |       |\n|  url                       |###                                                               |       |\n|  dbtable                   |TEST.PEOPLE                                                       |       |\n|  user                      |testUser                                                          |       |\n|  password                  |###                                                               |       |\n+----------------------------+------------------------------------------------------------------+-------+\n```\n\n- The output of `DESC EXTENDED`\n```\n|# Detailed Table Information|CatalogTable(\n\tTable: `default`.`tab1`\n\tCreated: Wed Nov 16 22:43:49 PST 2016\n\tLast Access: Wed Dec 31 15:59:59 PST 1969\n\tType: MANAGED\n\tSchema: [StructField(NAME,StringType,false), StructField(THEID,IntegerType,false)]\n\tProvider: org.apache.spark.sql.jdbc\n\tStorage(Location: file:/Users/xiaoli/IdeaProjects/sparkDelivery/spark-warehouse/tab1, Properties: [url=###, dbtable=TEST.PEOPLE, user=testUser, password=###]))|       |\n```\n\n### How was this patch tested?\n\nAdded test cases\n\nAuthor: gatorsmile <gatorsmile@gmail.com>\n\nCloses #15358 from gatorsmile/maskCredentials.\n\n(cherry picked from commit 9f273c5173c05017c3009faaf3e10f2f70a842d0)\nSigned-off-by: Herman van Hovell <hvanhovell@databricks.com>","distinct":true,"url":"https://api.github.com/repos/apache/spark/commits/a9d4febe900aa3eb9c595089e7283a64a24c8761"}]}
{"actor":{"display_login":"asfgit"}}