{"push_id":1154675100,"size":1,"distinct_size":1,"ref":"refs/heads/branch-2.0","head":"7e2bfff20c7278a20dca857cfd452b96d4d97c1a","before":"beb75300455a4f92000b69e740256102d9f2d472","commits":[{"sha":"7e2bfff20c7278a20dca857cfd452b96d4d97c1a","author":{"name":"Dongjoon Hyun","email":"1bf3a3c57610939bd1558dffb564906c959abecb@apache.org"},"message":"[SPARK-15807][SQL] Support varargs for dropDuplicates in Dataset/DataFrame\n\n## What changes were proposed in this pull request?\nThis PR adds `varargs`-types `dropDuplicates` functions in `Dataset/DataFrame`. Currently, `dropDuplicates` supports only `Seq` or `Array`.\n\n**Before**\n```scala\nscala> val ds = spark.createDataFrame(Seq((\"a\", 1), (\"b\", 2), (\"a\", 2)))\nds: org.apache.spark.sql.DataFrame = [_1: string, _2: int]\n\nscala> ds.dropDuplicates(Seq(\"_1\", \"_2\"))\nres0: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [_1: string, _2: int]\n\nscala> ds.dropDuplicates(\"_1\", \"_2\")\n<console>:26: error: overloaded method value dropDuplicates with alternatives:\n  (colNames: Array[String])org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] <and>\n  (colNames: Seq[String])org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] <and>\n  ()org.apache.spark.sql.Dataset[org.apache.spark.sql.Row]\n cannot be applied to (String, String)\n       ds.dropDuplicates(\"_1\", \"_2\")\n          ^\n```\n\n**After**\n```scala\nscala> val ds = spark.createDataFrame(Seq((\"a\", 1), (\"b\", 2), (\"a\", 2)))\nds: org.apache.spark.sql.DataFrame = [_1: string, _2: int]\n\nscala> ds.dropDuplicates(\"_1\", \"_2\")\nres0: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [_1: string, _2: int]\n```\n\n## How was this patch tested?\n\nPass the Jenkins tests with new testcases.\n\nAuthor: Dongjoon Hyun <dongjoon@apache.org>\n\nCloses #13545 from dongjoon-hyun/SPARK-15807.\n\n(cherry picked from commit 3fd2ff4dd85633af49865456a52bf0c09c99708b)\nSigned-off-by: Reynold Xin <rxin@databricks.com>","distinct":true,"url":"https://api.github.com/repos/apache/spark/commits/7e2bfff20c7278a20dca857cfd452b96d4d97c1a"}]}
{"actor":{"display_login":"asfgit"}}