{"push_id":1266708597,"size":127,"distinct_size":127,"ref":"refs/heads/master","head":"6b8cb1fe52e2c8b4b87b0c7d820f3a1824287328","before":"ea0bf91b4a2ca3ef472906e50e31fd6268b6f53e","commits":[{"sha":"7a9e25c38380e6c62080d62ad38a4830e44fe753","author":{"name":"Jeff Zhang","email":"8540ccd5710ce8e6da3f6c1c8189a4ef2fa98c60@apache.org"},"message":"[SPARK-13081][PYSPARK][SPARK_SUBMIT] Allow set pythonExec of driver and executor through conf…\n\nBefore this PR, user have to export environment variable to specify the python of driver & executor which is not so convenient for users. This PR is trying to allow user to specify python through configuration \"--pyspark-driver-python\" & \"--pyspark-executor-python\"\n\nManually test in local & yarn mode for pyspark-shell and pyspark batch mode.\n\nAuthor: Jeff Zhang <zjffdu@apache.org>\n\nCloses #13146 from zjffdu/SPARK-13081.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/7a9e25c38380e6c62080d62ad38a4830e44fe753"},{"sha":"ac84fb64dd85257da06f93a48fed9bb188140423","author":{"name":"hyukjinkwon","email":"4c114bf3676ebcdb1616a11ca478d9181f4025da@gmail.com"},"message":"[SPARK-16434][SQL] Avoid per-record type dispatch in JSON when reading\n\n## What changes were proposed in this pull request?\n\nCurrently, `JacksonParser.parse` is doing type-based dispatch for each row to convert the tokens to appropriate values for Spark.\nIt might not have to be done like this because the schema is already kept.\n\nSo, appropriate converters can be created first according to the schema once, and then apply them to each row.\n\nThis PR corrects `JacksonParser` so that it creates all converters for the schema once and then applies them to each row rather than type dispatching for every row.\n\nBenchmark was proceeded with the codes below:\n\n#### Parser tests\n\n**Before**\n\n```scala\ntest(\"Benchmark for JSON converter\") {\n  val N = 500 << 8\n  val row =\n    \"\"\"{\"struct\":{\"field1\": true, \"field2\": 92233720368547758070},\n    \"structWithArrayFields\":{\"field1\":[4, 5, 6], \"field2\":[\"str1\", \"str2\"]},\n    \"arrayOfString\":[\"str1\", \"str2\"],\n    \"arrayOfInteger\":[1, 2147483647, -2147483648],\n    \"arrayOfLong\":[21474836470, 9223372036854775807, -9223372036854775808],\n    \"arrayOfBigInteger\":[922337203685477580700, -922337203685477580800],\n    \"arrayOfDouble\":[1.2, 1.7976931348623157E308, 4.9E-324, 2.2250738585072014E-308],\n    \"arrayOfBoolean\":[true, false, true],\n    \"arrayOfNull\":[null, null, null, null],\n    \"arrayOfStruct\":[{\"field1\": true, \"field2\": \"str1\"}, {\"field1\": false}, {\"field3\": null}],\n    \"arrayOfArray1\":[[1, 2, 3], [\"str1\", \"str2\"]],\n    \"arrayOfArray2\":[[1, 2, 3], [1.1, 2.1, 3.1]]\n   }\"\"\"\n  val data = List.fill(N)(row)\n  val dummyOption = new JSONOptions(Map.empty[String, String])\n  val schema =\n    InferSchema.infer(spark.sparkContext.parallelize(Seq(row)), \"\", dummyOption)\n  val factory = new JsonFactory()\n\n  val benchmark = new Benchmark(\"JSON converter\", N)\n  benchmark.addCase(\"convert JSON file\", 10) { _ =>\n    data.foreach { input =>\n      val parser = factory.createParser(input)\n      parser.nextToken()\n      JacksonParser.convertRootField(factory, parser, schema)\n    }\n  }\n  benchmark.run()\n}\n```\n\n```\nJSON converter:                          Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\n------------------------------------------------------------------------------------------------\nconvert JSON file                             1697 / 1807          0.1       13256.9       1.0X\n```\n\n**After**\n\n```scala\ntest(\"Benchmark for JSON converter\") {\n  val N = 500 << 8\n  val row =\n    \"\"\"{\"struct\":{\"field1\": true, \"field2\": 92233720368547758070},\n    \"structWithArrayFields\":{\"field1\":[4, 5, 6], \"field2\":[\"str1\", \"str2\"]},\n    \"arrayOfString\":[\"str1\", \"str2\"],\n    \"arrayOfInteger\":[1, 2147483647, -2147483648],\n    \"arrayOfLong\":[21474836470, 9223372036854775807, -9223372036854775808],\n    \"arrayOfBigInteger\":[922337203685477580700, -922337203685477580800],\n    \"arrayOfDouble\":[1.2, 1.7976931348623157E308, 4.9E-324, 2.2250738585072014E-308],\n    \"arrayOfBoolean\":[true, false, true],\n    \"arrayOfNull\":[null, null, null, null],\n    \"arrayOfStruct\":[{\"field1\": true, \"field2\": \"str1\"}, {\"field1\": false}, {\"field3\": null}],\n    \"arrayOfArray1\":[[1, 2, 3], [\"str1\", \"str2\"]],\n    \"arrayOfArray2\":[[1, 2, 3], [1.1, 2.1, 3.1]]\n   }\"\"\"\n  val data = List.fill(N)(row)\n  val dummyOption = new JSONOptions(Map.empty[String, String], new SQLConf())\n  val schema =\n    InferSchema.infer(spark.sparkContext.parallelize(Seq(row)), dummyOption)\n\n  val benchmark = new Benchmark(\"JSON converter\", N)\n  benchmark.addCase(\"convert JSON file\", 10) { _ =>\n    val parser = new JacksonParser(schema, dummyOption)\n    data.foreach { input =>\n      parser.parse(input)\n    }\n  }\n  benchmark.run()\n}\n```\n\n```\nJSON converter:                          Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\n------------------------------------------------------------------------------------------------\nconvert JSON file                             1401 / 1461          0.1       10947.4       1.0X\n```\n\nIt seems parsing time is improved by roughly ~20%\n\n#### End-to-End test\n\n```scala\ntest(\"Benchmark for JSON reader\") {\n  val N = 500 << 8\n  val row =\n    \"\"\"{\"struct\":{\"field1\": true, \"field2\": 92233720368547758070},\n    \"structWithArrayFields\":{\"field1\":[4, 5, 6], \"field2\":[\"str1\", \"str2\"]},\n    \"arrayOfString\":[\"str1\", \"str2\"],\n    \"arrayOfInteger\":[1, 2147483647, -2147483648],\n    \"arrayOfLong\":[21474836470, 9223372036854775807, -9223372036854775808],\n    \"arrayOfBigInteger\":[922337203685477580700, -922337203685477580800],\n    \"arrayOfDouble\":[1.2, 1.7976931348623157E308, 4.9E-324, 2.2250738585072014E-308],\n    \"arrayOfBoolean\":[true, false, true],\n    \"arrayOfNull\":[null, null, null, null],\n    \"arrayOfStruct\":[{\"field1\": true, \"field2\": \"str1\"}, {\"field1\": false}, {\"field3\": null}],\n    \"arrayOfArray1\":[[1, 2, 3], [\"str1\", \"str2\"]],\n    \"arrayOfArray2\":[[1, 2, 3], [1.1, 2.1, 3.1]]\n   }\"\"\"\n  val df = spark.sqlContext.read.json(spark.sparkContext.parallelize(List.fill(N)(row)))\n  withTempPath { path =>\n    df.write.format(\"json\").save(path.getCanonicalPath)\n\n    val benchmark = new Benchmark(\"JSON reader\", N)\n    benchmark.addCase(\"reading JSON file\", 10) { _ =>\n      spark.read.format(\"json\").load(path.getCanonicalPath).collect()\n    }\n    benchmark.run()\n  }\n}\n```\n\n**Before**\n\n```\nJSON reader:                             Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\n------------------------------------------------------------------------------------------------\nreading JSON file                             6485 / 6924          0.0       50665.0       1.0X\n```\n\n**After**\n\n```\nJSON reader:                             Best/Avg Time(ms)    Rate(M/s)   Per Row(ns)   Relative\n------------------------------------------------------------------------------------------------\nreading JSON file                             6350 / 6529          0.0       49609.3       1.0X\n```\n\n## How was this patch tested?\n\nExisting test cases should cover this.\n\nAuthor: hyukjinkwon <gurwls223@gmail.com>\n\nCloses #14102 from HyukjinKwon/SPARK-16434.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/ac84fb64dd85257da06f93a48fed9bb188140423"},{"sha":"ccc6dc0f4b62837c73fca0e3c8b9c14be798b062","author":{"name":"Yanbo Liang","email":"b6929798b4e4fba75014af8265904a7818902ec7@gmail.com"},"message":"[MINOR][ML] Rename TreeEnsembleModels to TreeEnsembleModel for PySpark\n\n## What changes were proposed in this pull request?\nFix the typo of ```TreeEnsembleModels``` for PySpark, it should ```TreeEnsembleModel``` which will be consistent with Scala. What's more, it represents a tree ensemble model, so  ```TreeEnsembleModel``` should be more reasonable. This should not be used public, so it will not involve  breaking change.\n\n## How was this patch tested?\nNo new tests, should pass existing ones.\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #14454 from yanboliang/TreeEnsembleModel.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/ccc6dc0f4b62837c73fca0e3c8b9c14be798b062"},{"sha":"abff92bfdc7d4c9d2308794f0350561fe0ceb4dd","author":{"name":"Dongjoon Hyun","email":"1bf3a3c57610939bd1558dffb564906c959abecb@apache.org"},"message":"[SPARK-16975][SQL] Column-partition path starting '_' should be handled correctly\n\n## What changes were proposed in this pull request?\n\nCurrently, Spark ignores path names starting with underscore `_` and `.`. This causes read-failures for the column-partitioned file data sources whose partition column names starts from '_', e.g. `_col`.\n\n**Before**\n```scala\nscala> spark.range(10).withColumn(\"_locality_code\", $\"id\").write.partitionBy(\"_locality_code\").save(\"/tmp/parquet\")\nscala> spark.read.parquet(\"/tmp/parquet\")\norg.apache.spark.sql.AnalysisException: Unable to infer schema for ParquetFormat at /tmp/parquet20. It must be specified manually;\n```\n\n**After**\n```scala\nscala> spark.range(10).withColumn(\"_locality_code\", $\"id\").write.partitionBy(\"_locality_code\").save(\"/tmp/parquet\")\nscala> spark.read.parquet(\"/tmp/parquet\")\nres2: org.apache.spark.sql.DataFrame = [id: bigint, _locality_code: int]\n```\n\n## How was this patch tested?\n\nPass the Jenkins with a new test case.\n\nAuthor: Dongjoon Hyun <dongjoon@apache.org>\n\nCloses #14585 from dongjoon-hyun/SPARK-16975-PARQUET.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/abff92bfdc7d4c9d2308794f0350561fe0ceb4dd"},{"sha":"00e103a6edd1a1f001a94d41dd1f7acc40a1e30f","author":{"name":"petermaxlee","email":"ac6c8937256e5ff211210b88da2643bdd0703390@gmail.com"},"message":"[SPARK-17013][SQL] Parse negative numeric literals\n\n## What changes were proposed in this pull request?\nThis patch updates the SQL parser to parse negative numeric literals as numeric literals, instead of unary minus of positive literals.\n\nThis allows the parser to parse the minimal value for each data type, e.g. \"-32768S\".\n\n## How was this patch tested?\nUpdated test cases.\n\nAuthor: petermaxlee <petermaxlee@gmail.com>\n\nCloses #14608 from petermaxlee/SPARK-17013.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/00e103a6edd1a1f001a94d41dd1f7acc40a1e30f"},{"sha":"993923c8f5ca719daf905285738b7fdcaf944d8c","author":{"name":"hongshen","email":"0ddf15eae228419f727fc3ecdc8f1b3f1a502622@126.com"},"message":"[SPARK-16985] Change dataFormat from yyyyMMddHHmm to yyyyMMddHHmmss\n\n## What changes were proposed in this pull request?\n\nIn our cluster, sometimes the sql output maybe overrided. When I submit some sql, all insert into the same table, and the sql will cost less one minute, here is the detail,\n1 sql1, 11:03 insert into table.\n2 sql2, 11:04:11 insert into table.\n3 sql3, 11:04:48 insert into table.\n4 sql4, 11:05 insert into table.\n5 sql5, 11:06 insert into table.\nThe sql3's output file will override the sql2's output file. here is the log:\n```\n16/05/04 11:04:11 INFO hive.SparkHiveHadoopWriter: XXfinalPath=hdfs://tl-sng-gdt-nn-tdw.tencent-distribute.com:54310/tmp/assorz/tdw-tdwadmin/20160504/04559505496526517_-1_1204544348/10000/_tmp.p_20160428/attempt_201605041104_0001_m_000000_1\n\n16/05/04 11:04:48 INFO hive.SparkHiveHadoopWriter: XXfinalPath=hdfs://tl-sng-gdt-nn-tdw.tencent-distribute.com:54310/tmp/assorz/tdw-tdwadmin/20160504/04559505496526517_-1_212180468/10000/_tmp.p_20160428/attempt_201605041104_0001_m_000000_1\n\n```\n\nThe reason is the output file use SimpleDateFormat(\"yyyyMMddHHmm\"), if two sql insert into the same table in the same minute, the output will be overrite. I think we should change dateFormat to \"yyyyMMddHHmmss\", in our cluster, we can't finished a sql in one second.\n\n## How was this patch tested?\n\n(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)\n\n(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)\n\nAuthor: hongshen <shenh062326@126.com>\n\nCloses #14574 from shenh062326/SPARK-16985.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/993923c8f5ca719daf905285738b7fdcaf944d8c"},{"sha":"f4482225c405b9cfe078deac74e4c28e2dcc97c3","author":{"name":"hyukjinkwon","email":"4c114bf3676ebcdb1616a11ca478d9181f4025da@gmail.com"},"message":"[MINOR][DOC] Fix style in examples across documentation\n\n## What changes were proposed in this pull request?\n\nThis PR fixes the documentation as below:\n\n  -  Python has 4 spaces and Java and Scala has 2 spaces (See https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide).\n\n  - Avoid excessive parentheses and curly braces for anonymous functions. (See https://github.com/databricks/scala-style-guide#anonymous)\n\n## How was this patch tested?\n\nN/A\n\nAuthor: hyukjinkwon <gurwls223@gmail.com>\n\nCloses #14593 from HyukjinKwon/minor-documentation.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/f4482225c405b9cfe078deac74e4c28e2dcc97c3"},{"sha":"79e2caa1328843457841d71642b60be919ebb1e0","author":{"name":"gatorsmile","email":"ea6589371ad7a107e4268706d7d8403e3ba585bb@gmail.com"},"message":"[SPARK-16598][SQL][TEST] Added a test case for verifying the table identifier parsing\n\n#### What changes were proposed in this pull request?\nSo far, the test cases of `TableIdentifierParserSuite` do not cover the quoted cases. We should add one for avoiding regression.\n\n#### How was this patch tested?\nN/A\n\nAuthor: gatorsmile <gatorsmile@gmail.com>\n\nCloses #14244 from gatorsmile/quotedIdentifiers.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/79e2caa1328843457841d71642b60be919ebb1e0"},{"sha":"bbae20ade14e50541e4403ca7b45bf6c11695d15","author":{"name":"Yanbo Liang","email":"b6929798b4e4fba75014af8265904a7818902ec7@gmail.com"},"message":"[SPARK-17033][ML][MLLIB] GaussianMixture should use treeAggregate to improve performance\n\n## What changes were proposed in this pull request?\n```GaussianMixture``` should use ```treeAggregate``` rather than ```aggregate``` to improve performance and scalability. In my test of dataset with 200 features and 1M instance, I found there is 20% increased performance.\nBTW, we should destroy broadcast variable ```compute``` at the end of each iteration.\n\n## How was this patch tested?\nExisting tests.\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #14621 from yanboliang/spark-17033.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/bbae20ade14e50541e4403ca7b45bf6c11695d15"},{"sha":"2a105134e9a3efd46b761fab5e563ddebb26575d","author":{"name":"Dongjoon Hyun","email":"1bf3a3c57610939bd1558dffb564906c959abecb@apache.org"},"message":"[SPARK-16771][SQL] WITH clause should not fall into infinite loop.\n\n## What changes were proposed in this pull request?\n\nThis PR changes the CTE resolving rule to use only **forward-declared** tables in order to prevent infinite loops. More specifically, new logic is like the following.\n\n* Resolve CTEs in `WITH` clauses first before replacing the main SQL body.\n* When resolving CTEs, only forward-declared CTEs or base tables are referenced.\n  - Self-referencing is not allowed any more.\n  - Cross-referencing is not allowed any more.\n\n**Reported Error Scenarios**\n```scala\nscala> sql(\"WITH t AS (SELECT 1 FROM t) SELECT * FROM t\")\njava.lang.StackOverflowError\n...\nscala> sql(\"WITH t1 AS (SELECT * FROM t2), t2 AS (SELECT 2 FROM t1) SELECT * FROM t1, t2\")\njava.lang.StackOverflowError\n...\n```\nNote that `t`, `t1`, and `t2` are not declared in database. Spark falls into infinite loops before resolving table names.\n\n## How was this patch tested?\n\nPass the Jenkins tests with new two testcases.\n\nAuthor: Dongjoon Hyun <dongjoon@apache.org>\n\nCloses #14397 from dongjoon-hyun/SPARK-16771-TREENODE.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/2a105134e9a3efd46b761fab5e563ddebb26575d"},{"sha":"91f2735a180f0af1f15303fd0a32633dfd1c1fe0","author":{"name":"WeichenXu","email":"7a440c883b23bc43b7a6fdb423cbfd8e75175a83@outlook.com"},"message":"[DOC] add config option spark.ui.enabled into document\n\n## What changes were proposed in this pull request?\n\nThe configuration doc lost the config option `spark.ui.enabled` (default value is `true`)\nI think this option is important because many cases we would like to turn it off.\nso I add it.\n\n## How was this patch tested?\n\nN/A\n\nAuthor: WeichenXu <WeichenXu123@outlook.com>\n\nCloses #14604 from WeichenXu123/add_doc_param_spark_ui_enabled.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/91f2735a180f0af1f15303fd0a32633dfd1c1fe0"},{"sha":"e46cb78b3b9fd04a50b5ae50f360db612d656a48","author":{"name":"Jagadeesan","email":"77eaebe535b25af84722e54783804ea15c5d230a@us.ibm.com"},"message":"[SPARK-12370][DOCUMENTATION] Documentation should link to examples …\n\n## What changes were proposed in this pull request?\n\nWhen documentation is built is should reference examples from the same build. There are times when the docs have links that point to files in the GitHub head which may not be valid on the current release. Changed that in URLs to make them point to the right tag in git using ```SPARK_VERSION_SHORT```\n\n…from its own release version] [Streaming programming guide]\n\nAuthor: Jagadeesan <as2@us.ibm.com>\n\nCloses #14596 from jagadeesanas2/SPARK-12370.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/e46cb78b3b9fd04a50b5ae50f360db612d656a48"},{"sha":"7f7133bdccecaccd6dfb52f13c18c1e320d65f86","author":{"name":"Xin Ren","email":"82e6f9780faa35cac922f524c8a765f69b3ff7ff@126.com"},"message":"[MINOR][CORE] fix warnings on depreciated methods in MesosClusterSchedulerSuite and DiskBlockObjectWriterSuite\n\n## What changes were proposed in this pull request?\n\nFixed warnings below after scanning through warnings during build:\n\n```\n[warn] /home/jenkins/workspace/SparkPullRequestBuilder/core/src/test/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterSchedulerSuite.scala:34: imported `Utils' is permanently hidden by definition of object Utils in package mesos\n[warn] import org.apache.spark.scheduler.cluster.mesos.Utils\n[warn]                                                 ^\n```\n\nand\n```\n[warn] /home/jenkins/workspace/SparkPullRequestBuilder/core/src/test/scala/org/apache/spark/storage/DiskBlockObjectWriterSuite.scala:113: method shuffleBytesWritten in class ShuffleWriteMetrics is deprecated: use bytesWritten instead\n[warn]     assert(writeMetrics.shuffleBytesWritten === file.length())\n[warn]                         ^\n[warn] /home/jenkins/workspace/SparkPullRequestBuilder/core/src/test/scala/org/apache/spark/storage/DiskBlockObjectWriterSuite.scala:119: method shuffleBytesWritten in class ShuffleWriteMetrics is deprecated: use bytesWritten instead\n[warn]     assert(writeMetrics.shuffleBytesWritten === file.length())\n[warn]                         ^\n[warn] /home/jenkins/workspace/SparkPullRequestBuilder/core/src/test/scala/org/apache/spark/storage/DiskBlockObjectWriterSuite.scala:131: method shuffleBytesWritten in class ShuffleWriteMetrics is deprecated: use bytesWritten instead\n[warn]     assert(writeMetrics.shuffleBytesWritten === file.length())\n[warn]                         ^\n[warn] /home/jenkins/workspace/SparkPullRequestBuilder/core/src/test/scala/org/apache/spark/storage/DiskBlockObjectWriterSuite.scala:135: method shuffleBytesWritten in class ShuffleWriteMetrics is deprecated: use bytesWritten instead\n[warn]     assert(writeMetrics.shuffleBytesWritten === file.length())\n[warn]                         ^\n```\n\n## How was this patch tested?\n\nTested manually on local laptop.\n\nAuthor: Xin Ren <iamshrek@126.com>\n\nCloses #14609 from keypointt/suiteWarnings.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/7f7133bdccecaccd6dfb52f13c18c1e320d65f86"},{"sha":"8c8acdec9365136cba13060ce36c22b28e29b59b","author":{"name":"GraceH","email":"e70f0e6a931ab7293132ea4f5c74d74fb0d68827@qq.com"},"message":"[SPARK-16968] Add additional options in jdbc when creating a new table\n\n## What changes were proposed in this pull request?\n\nIn the PR, we just allow the user to add additional options when create a new table in JDBC writer.\nThe options can be table_options or partition_options.\nE.g., \"CREATE TABLE t (name string) ENGINE=InnoDB DEFAULT CHARSET=utf8\"\n\nHere is the usage example:\n```\ndf.write.option(\"createTableOptions\", \"ENGINE=InnoDB DEFAULT CHARSET=utf8\").jdbc(...)\n```\n## How was this patch tested?\n\n(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)\nwill apply test result soon.\n\nAuthor: GraceH <93113783@qq.com>\n\nCloses #14559 from GraceH/jdbc_options.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/8c8acdec9365136cba13060ce36c22b28e29b59b"},{"sha":"67f025d90e6ba8c039ff45e26d34f20d24b92e6a","author":{"name":"Luciano Resende","email":"a964278a26a7baf3aa6d46d63840770571cd46e0@apache.org"},"message":"[SPARK-17023][BUILD] Upgrade to Kafka 0.10.0.1 release\n\n## What changes were proposed in this pull request?\nUpdate Kafka streaming connector to use Kafka 0.10.0.1 release\n\n## How was this patch tested?\nTested via Spark unit and integration tests\n\nAuthor: Luciano Resende <lresende@apache.org>\n\nCloses #14606 from lresende/kafka-upgrade.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/67f025d90e6ba8c039ff45e26d34f20d24b92e6a"},{"sha":"cdaa562c9a09e2e83e6df4e84d911ce1428a7a7c","author":{"name":"Sean Owen","email":"4c59f357315334980589bde1049040bf6de6532c@cloudera.com"},"message":"[SPARK-16966][SQL][CORE] App Name is a randomUUID even when \"spark.app.name\" exists\n\n## What changes were proposed in this pull request?\n\nDon't override app name specified in `SparkConf` with a random app name. Only set it if the conf has no app name even after options have been applied.\n\nSee also https://github.com/apache/spark/pull/14602\nThis is similar to Sherry302 's original proposal in https://github.com/apache/spark/pull/14556\n\n## How was this patch tested?\n\nJenkins test, with new case reproducing the bug\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #14630 from srowen/SPARK-16966.2.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/cdaa562c9a09e2e83e6df4e84d911ce1428a7a7c"},{"sha":"0ebf7c1bff736cf54ec47957d71394d5b75b47a7","author":{"name":"zero323","email":"173f3ab6399f1b748c2cb687d75e045296a62aa6@users.noreply.github.com"},"message":"[SPARK-17027][ML] Avoid integer overflow in PolynomialExpansion.getPolySize\n\n## What changes were proposed in this pull request?\n\nReplaces custom choose function with o.a.commons.math3.CombinatoricsUtils.binomialCoefficient\n\n## How was this patch tested?\n\nSpark unit tests\n\nAuthor: zero323 <zero323@users.noreply.github.com>\n\nCloses #14614 from zero323/SPARK-17027.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/0ebf7c1bff736cf54ec47957d71394d5b75b47a7"},{"sha":"2a3d286f3421f6836b71afcbda3084222752e6b1","author":{"name":"Zhenglai Zhang","email":"0522130a845b4b2e33ab68c8ccdc01da08ba68ad@hotmail.com"},"message":"[WIP][MINOR][TYPO] Fix several trivival typos\n\n## What changes were proposed in this pull request?\n\n* Fixed one typo `\"overriden\"` as `\"overridden\"`, also make sure no other same typo.\n* Fixed one typo `\"lowcase\"` as `\"lowercase\"`, also make sure no other same typo.\n\n## How was this patch tested?\n\nSince the change is very tiny, so I just make sure compilation is successful.\nI am new to the spark community,  please feel free to let me do other necessary steps.\n\nThanks in advance!\n\n----\nUpdated: Found another typo `lowcase` later and fixed then in the same patch\n\nAuthor: Zhenglai Zhang <zhenglaizhang@hotmail.com>\n\nCloses #14622 from zhenglaizhang/fixtypo.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/2a3d286f3421f6836b71afcbda3084222752e6b1"},{"sha":"1a028bdefa6312bf0eec46b89a1947da7e9d84af","author":{"name":"Stavros Kontopoulos","email":"42bafac589feb2200b51ee576d084d6a9af943fa@lightbend.com"},"message":"[SPARK-11714][MESOS] Make Spark on Mesos honor port restrictions on coarse grain mode\n\n- Make mesos coarse grained scheduler accept port offers and pre-assign ports\n\nPrevious attempt was for fine grained: https://github.com/apache/spark/pull/10808\n\nAuthor: Stavros Kontopoulos <stavros.kontopoulos@lightbend.com>\nAuthor: Stavros Kontopoulos <stavros.kontopoulos@typesafe.com>\n\nCloses #11157 from skonto/honour_ports_coarse.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/1a028bdefa6312bf0eec46b89a1947da7e9d84af"},{"sha":"ddf0d1e3fe18bcd01e1447feea1b76ce86087b3b","author":{"name":"Yanbo Liang","email":"b6929798b4e4fba75014af8265904a7818902ec7@gmail.com"},"message":"[TRIVIAL][ML] Fix LogisticRegression typo in error message.\n\n## What changes were proposed in this pull request?\nFix ```LogisticRegression``` typo in error message.\n\n## How was this patch tested?\nDocs change, no new tests.\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #14633 from yanboliang/lr-typo.","distinct":true,"url":"https://api.github.com/repos/JerryLead/spark/commits/ddf0d1e3fe18bcd01e1447feea1b76ce86087b3b"}]}
{"actor":{"display_login":"JerryLead"}}