{"push_id":994881509,"size":1000,"distinct_size":28,"ref":"refs/heads/master","head":"50e60e36f7775a10cf39338e7c5716578a24d89f","before":"3535b91ddc9fd05b613a121e09263b0f378bd5fa","commits":[{"sha":"f57e6c9effdb9e282fc8ae66dc30fe053fed5272","author":{"email":"b59cd5ce8cb5ae44d26fc6984be1f84cf74771a7@databricks.com","name":"Shixiong Zhu"},"message":"[SPARK-12021][STREAMING][TESTS] Fix the potential dead-lock in StreamingListenerSuite\n\nIn StreamingListenerSuite.\"don't call ssc.stop in listener\", after the main thread calls `ssc.stop()`,  `StreamingContextStoppingCollector` may call  `ssc.stop()` in the listener bus thread, which is a dead-lock. This PR updated `StreamingContextStoppingCollector` to only call `ssc.stop()` in the first batch to avoid the dead-lock.\n\nAuthor: Shixiong Zhu <shixiong@databricks.com>\n\nCloses #10011 from zsxwing/fix-test-deadlock.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/f57e6c9effdb9e282fc8ae66dc30fe053fed5272"},{"sha":"b9921524d970f9413039967c1f17ae2e736982f0","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-12020][TESTS][TEST-HADOOP2.0] PR builder cannot trigger hadoop 2.0 test\n\nhttps://issues.apache.org/jira/browse/SPARK-12020\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #10010 from yhuai/SPARK-12020.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/b9921524d970f9413039967c1f17ae2e736982f0"},{"sha":"149cd692ee2e127d79386fd8e584f4f70a2906ba","author":{"email":"905db6c57c9ba2de5402239d136139da9b5d0058@gmail.com","name":"gatorsmile"},"message":"[SPARK-12028] [SQL] get_json_object returns an incorrect result when the value is null literals\n\nWhen calling `get_json_object` for the following two cases, both results are `\"null\"`:\n\n```scala\n    val tuple: Seq[(String, String)] = (\"5\", \"\"\"{\"f1\": null}\"\"\") :: Nil\n    val df: DataFrame = tuple.toDF(\"key\", \"jstring\")\n    val res = df.select(functions.get_json_object($\"jstring\", \"$.f1\")).collect()\n```\n```scala\n    val tuple2: Seq[(String, String)] = (\"5\", \"\"\"{\"f1\": \"null\"}\"\"\") :: Nil\n    val df2: DataFrame = tuple2.toDF(\"key\", \"jstring\")\n    val res3 = df2.select(functions.get_json_object($\"jstring\", \"$.f1\")).collect()\n```\n\nFixed the problem and also added a test case.\n\nAuthor: gatorsmile <gatorsmile@gmail.com>\n\nCloses #10018 from gatorsmile/get_json_object.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/149cd692ee2e127d79386fd8e584f4f70a2906ba"},{"sha":"28e46ab46368ea3833c8e805163893bbb6f2a265","author":{"email":"fd68b5404b021a07e5ec5abdeb1b0ffe8f04e45b@hotmail.com","name":"felixcheung"},"message":"[SPARK-12029][SPARKR] Improve column functions signature, param check, tests, fix doc and add examples\n\nshivaram sun-rui\n\nAuthor: felixcheung <felixcheung_m@hotmail.com>\n\nCloses #10019 from felixcheung/rfunctionsdoc.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/28e46ab46368ea3833c8e805163893bbb6f2a265"},{"sha":"c793d2d9a1ccc203fc103eb0636958fe8d71f471","author":{"email":"fd68b5404b021a07e5ec5abdeb1b0ffe8f04e45b@hotmail.com","name":"felixcheung"},"message":"[SPARK-9319][SPARKR] Add support for setting column names, types\n\nAdd support for for colnames, colnames<-, coltypes<-\nAlso added tests for names, names<- which have no test previously.\n\nI merged with PR 8984 (coltypes). Clicked the wrong thing, crewed up the PR. Recreated it here. Was #9218\n\nshivaram sun-rui\n\nAuthor: felixcheung <felixcheung_m@hotmail.com>\n\nCloses #9654 from felixcheung/colnamescoltypes.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/c793d2d9a1ccc203fc103eb0636958fe8d71f471"},{"sha":"cc7a1bc9370b163f51230e5ca4be612d133a5086","author":{"email":"3f27eab2afd49386f0ac811afb0bc245aa92c0fa@intel.com","name":"Sun Rui"},"message":"[SPARK-11781][SPARKR] SparkR has problem in inferring type of raw type.\n\nAuthor: Sun Rui <rui.sun@intel.com>\n\nCloses #9769 from sun-rui/SPARK-11781.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/cc7a1bc9370b163f51230e5ca4be612d133a5086"},{"sha":"3d28081e53698ed77e93c04299957c02bcaba9bf","author":{"email":"679c4ded150c6b9a8052807f108ce836961fed8d@questtec.nl","name":"Herman van Hovell"},"message":"[SPARK-12024][SQL] More efficient multi-column counting.\n\nIn https://github.com/apache/spark/pull/9409 we enabled multi-column counting. The approach taken in that PR introduces a bit of overhead by first creating a row only to check if all of the columns are non-null.\n\nThis PR fixes that technical debt. Count now takes multiple columns as its input. In order to make this work I have also added support for multiple columns in the single distinct code path.\n\ncc yhuai\n\nAuthor: Herman van Hovell <hvanhovell@questtec.nl>\n\nCloses #10015 from hvanhovell/SPARK-12024.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/3d28081e53698ed77e93c04299957c02bcaba9bf"},{"sha":"0ddfe7868948e302858a2b03b50762eaefbeb53e","author":{"email":"8adc4758e0403b3bd961e4e414291eaa9ba64098@databricks.com","name":"Yin Huai"},"message":"[SPARK-12039] [SQL] Ignore HiveSparkSubmitSuite's \"SPARK-9757 Persist Parquet relation with decimal column\".\n\nhttps://issues.apache.org/jira/browse/SPARK-12039\n\nSince it is pretty flaky in hadoop 1 tests, we can disable it while we are investigating the cause.\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #10035 from yhuai/SPARK-12039-ignore.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/0ddfe7868948e302858a2b03b50762eaefbeb53e"},{"sha":"e0749442051d6e29dae4f4cdcb2937c0b015f98f","author":{"email":"907e88dd99290229902e1e2df1b993ada3ef63a3@outlook.com","name":"toddwan"},"message":"[SPARK-11859][MESOS] SparkContext accepts invalid Master URLs in the form zk://host:port for a multi-master Mesos cluster using ZooKeeper\n\n* According to below doc and validation logic in [SparkSubmit.scala](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L231), master URL for a mesos cluster should always start with `mesos://`\n\nhttp://spark.apache.org/docs/latest/running-on-mesos.html\n`The Master URLs for Mesos are in the form mesos://host:5050 for a single-master Mesos cluster, or mesos://zk://host:2181 for a multi-master Mesos cluster using ZooKeeper.`\n\n* However, [SparkContext.scala](https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SparkContext.scala#L2749) fails the validation and can receive master URL in the form `zk://host:port`\n\n* For the master URLs in the form `zk:host:port`, the valid form should be `mesos://zk://host:port`\n\n* This PR restrict the validation in `SparkContext.scala`, and now only mesos master URLs prefixed with `mesos://` can be accepted.\n\n* This PR also updated corresponding unit test.\n\nAuthor: toddwan <tawan0109@outlook.com>\n\nCloses #9886 from toddwan/S11859.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/e0749442051d6e29dae4f4cdcb2937c0b015f98f"},{"sha":"953e8e6dcb32cd88005834e9c3720740e201826c","author":{"email":"4c04de27f2e5731e300e66d2829258fd5fdc3661@gmail.com","name":"Prashant Sharma"},"message":"[MINOR][BUILD] Changed the comment to reflect the plugin project is there to support SBT pom reader only.\n\nAuthor: Prashant Sharma <scrapcodes@gmail.com>\n\nCloses #10012 from ScrapCodes/minor-build-comment.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/953e8e6dcb32cd88005834e9c3720740e201826c"},{"sha":"26c3581f17f475fab2f3b5301b8f253ff2fa6438","author":{"email":"e11edbb41334a578b9287de27715b1cd2772a54a@gmail.com","name":"Wieland Hoffmann"},"message":"[DOC] Explicitly state that top maintains the order of elements\n\nTop is implemented in terms of takeOrdered, which already maintains the\norder, so top should, too.\n\nAuthor: Wieland Hoffmann <themineo@gmail.com>\n\nCloses #10013 from mineo/top-order.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/26c3581f17f475fab2f3b5301b8f253ff2fa6438"},{"sha":"bf0e85a70a54a2d7fd6804b6bd00c63c20e2bb00","author":{"email":"4c04de27f2e5731e300e66d2829258fd5fdc3661@gmail.com","name":"Prashant Sharma"},"message":"[SPARK-12023][BUILD] Fix warnings while packaging spark with maven.\n\nthis is a trivial fix, discussed [here](http://stackoverflow.com/questions/28500401/maven-assembly-plugin-warning-the-assembly-descriptor-contains-a-filesystem-roo/).\n\nAuthor: Prashant Sharma <scrapcodes@gmail.com>\n\nCloses #10014 from ScrapCodes/assembly-warning.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/bf0e85a70a54a2d7fd6804b6bd00c63c20e2bb00"},{"sha":"2db4662fe2d72749c06ad5f11f641a388343f77c","author":{"email":"74bab60ee07bbc7c6e7ece9bc76c048745a2c533@oracle.com","name":"CK50"},"message":"[SPARK-11989][SQL] Only use commit in JDBC data source if the underlying database supports transactions\n\nFixes [SPARK-11989](https://issues.apache.org/jira/browse/SPARK-11989)\n\nAuthor: CK50 <christian.kurz@oracle.com>\nAuthor: Christian Kurz <christian.kurz@oracle.com>\n\nCloses #9973 from CK50/branch-1.6_non-transactional.\n\n(cherry picked from commit a589736a1b237ef2f3bd59fbaeefe143ddcc8f4e)\nSigned-off-by: Reynold Xin <rxin@databricks.com>","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/2db4662fe2d72749c06ad5f11f641a388343f77c"},{"sha":"17275fa99c670537c52843df405279a52b5c9594","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-11700] [SQL] Remove thread local SQLContext  in SparkPlan\n\nIn 1.6, we introduce a public API to have a SQLContext for current thread, SparkPlan should use that.\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #9990 from davies/leak_context.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/17275fa99c670537c52843df405279a52b5c9594"},{"sha":"8df584b0200402d8b2ce0a8de24f7a760ced8655","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-11982] [SQL] improve performance of cartesian product\n\nThis PR improve the performance of CartesianProduct by caching the result of right plan.\n\nAfter this patch, the query time of TPC-DS Q65 go down to 4 seconds from 28 minutes (420X faster).\n\ncc nongli\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #9969 from davies/improve_cartesian.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/8df584b0200402d8b2ce0a8de24f7a760ced8655"},{"sha":"f2fbfa444f6e8d27953ec2d1c0b3abd603c963f9","author":{"email":"07c870ffe79a3d57c30fc67475f42d361ec88a03@gmail.com","name":"BenFradet"},"message":"[MINOR][DOCS] fixed list display in ml-ensembles\n\nThe list in ml-ensembles.md wasn't properly formatted and, as a result, was looking like this:\n![old](http://i.imgur.com/2ZhELLR.png)\n\nThis PR aims to make it look like this:\n![new](http://i.imgur.com/0Xriwd2.png)\n\nAuthor: BenFradet <benjamin.fradet@gmail.com>\n\nCloses #10025 from BenFradet/ml-ensembles-doc.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/f2fbfa444f6e8d27953ec2d1c0b3abd603c963f9"},{"sha":"2c5dee0fb8e4d1734ea3a0f22e0b5bfd2f6dba46","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"Revert \"[SPARK-11206] Support SQL UI on the history server\"\n\nThis reverts commit cc243a079b1c039d6e7f0b410d1654d94a090e14 / PR #9297\n\nI'm reverting this because it broke SQLListenerMemoryLeakSuite in the master Maven builds.\n\nSee #9991 for a discussion of why this broke the tests.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/2c5dee0fb8e4d1734ea3a0f22e0b5bfd2f6dba46"},{"sha":"a8ceec5e8c1572dd3d74783c06c78b7ca0b8a7ce","author":{"email":"dfc272d1e78652e4de5b4e76a0545cef11a46a5c@gmail.com","name":"Teng Qiu"},"message":"[SPARK-12053][CORE] EventLoggingListener.getLogPath needs 4 parameters\n\n```EventLoggingListener.getLogPath``` needs 4 input arguments:\nhttps://github.com/apache/spark/blob/v1.6.0-preview2/core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala#L276-L280\n\nthe 3rd parameter should be appAttemptId, 4th parameter is codec...\n\nAuthor: Teng Qiu <teng.qiu@gmail.com>\n\nCloses #10044 from chutium/SPARK-12053.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/a8ceec5e8c1572dd3d74783c06c78b7ca0b8a7ce"},{"sha":"e232720a65dfb9ae6135cbb7674e35eddd88d625","author":{"email":"66332c93c294b372ab1bdd5acb9b89a538e2d2f2@gmail.com","name":"Yuhao Yang"},"message":"[SPARK-11689][ML] Add user guide and example code for LDA under spark.ml\n\njira: https://issues.apache.org/jira/browse/SPARK-11689\n\nAdd simple user guide for LDA under spark.ml and example code under examples/. Use include_example to include example code in the user guide markdown. Check SPARK-11606 for instructions.\n\nOriginal PR is reverted due to document build error. https://github.com/apache/spark/pull/9722\n\nmengxr feynmanliang yinxusen  Sorry for the troubling.\n\nAuthor: Yuhao Yang <hhbyyh@gmail.com>\n\nCloses #9974 from hhbyyh/ldaMLExample.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/e232720a65dfb9ae6135cbb7674e35eddd88d625"},{"sha":"de64b65f7cf2ac58c1abc310ba547637fdbb8557","author":{"email":"452ae006f36be6c70342f33f64bb6f6eb552689c@gmail.com","name":"Yanbo Liang"},"message":"[SPARK-11975][ML] Remove duplicate mllib example (DT/RF/GBT in Java/Python)\n\nRemove duplicate mllib example (DT/RF/GBT in Java/Python).\nSince we have tutorial code for DT/RF/GBT classification/regression in Scala/Java/Python and example applications for DT/RF/GBT in Scala, so we mark these as duplicated and remove them.\nmengxr\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #9954 from yanboliang/SPARK-11975.","distinct":false,"url":"https://api.github.com/repos/yu-iskw/spark/commits/de64b65f7cf2ac58c1abc310ba547637fdbb8557"}]}