{"push_id":1170391191,"size":13,"distinct_size":13,"ref":"refs/heads/lrcomp","head":"45409672050b965b3e9876614a4f8b2c2af7df8d","before":"4f1f0e034c517b7732819860e64c3dc847c8fbfa","commits":[{"sha":"c399c7f0e485dcfc6cbc343bc246b8adc3f0648c","author":{"name":"Shixiong Zhu","email":"eb2f6da57ec54996cebae8c2a82b9fbfd527e14c@databricks.com"},"message":"[SPARK-16002][SQL] Sleep when no new data arrives to avoid 100% CPU usage\n\n## What changes were proposed in this pull request?\n\nAdd a configuration to allow people to set a minimum polling delay when no new data arrives (default is 10ms). This PR also cleans up some INFO logs.\n\n## How was this patch tested?\n\nExisting unit tests.\n\nAuthor: Shixiong Zhu <shixiong@databricks.com>\n\nCloses #13718 from zsxwing/SPARK-16002.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/c399c7f0e485dcfc6cbc343bc246b8adc3f0648c"},{"sha":"918c91954fb46400ce2c5ab066d2ec0ae48dda4a","author":{"name":"Xiangrui Meng","email":"e4720f5dc69a5967cefd679a46054c60c7e8d310@databricks.com"},"message":"[MINOR][MLLIB] move setCheckpointInterval to non-expert setters\n\n## What changes were proposed in this pull request?\n\nThe `checkpointInterval` is a non-expert param. This PR moves its setter to non-expert group.\n\nAuthor: Xiangrui Meng <meng@databricks.com>\n\nCloses #13813 from mengxr/checkpoint-non-expert.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/918c91954fb46400ce2c5ab066d2ec0ae48dda4a"},{"sha":"dbfdae4e41a900de01b48639d6554d32edbb2e0b","author":{"name":"Felix Cheung","email":"00eb71019ad7acfd67cff51fce666aad32ce6eed@hotmail.com"},"message":"[SPARK-16096][SPARKR] add union and deprecate unionAll\n\n## What changes were proposed in this pull request?\n\nadd union and deprecate unionAll, separate roxygen2 doc for rbind (since their usage and parameter lists are quite different)\n\n`explode` is also deprecated - but seems like replacement is a combination of calls; not sure if we should deprecate it in SparkR, yet.\n\n## How was this patch tested?\n\nunit tests, manual checks for r doc\n\nAuthor: Felix Cheung <felixcheung_m@hotmail.com>\n\nCloses #13805 from felixcheung/runion.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/dbfdae4e41a900de01b48639d6554d32edbb2e0b"},{"sha":"f4e8c31adf45af05751e0d77aefb5cacc58375ee","author":{"name":"Xiangrui Meng","email":"e4720f5dc69a5967cefd679a46054c60c7e8d310@databricks.com"},"message":"[SPARK-16117][MLLIB] hide LibSVMFileFormat and move its doc to LibSVMDataSource\n\n## What changes were proposed in this pull request?\n\nLibSVMFileFormat implements data source for LIBSVM format. However, users do not really need to call its APIs to use it. So we should hide it in the public API docs. The main issue is that we still need to put the documentation and example code somewhere. The proposal it to have a dummy class to hold the documentation, as a workaround to https://issues.scala-lang.org/browse/SI-8124.\n\n## How was this patch tested?\n\nManually checked the generated API doc and tested loading LIBSVM data.\n\nAuthor: Xiangrui Meng <meng@databricks.com>\n\nCloses #13819 from mengxr/SPARK-16117.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/f4e8c31adf45af05751e0d77aefb5cacc58375ee"},{"sha":"9493b079a0050f0a6f4936c17622b96fb185b67f","author":{"name":"Xiangrui Meng","email":"e4720f5dc69a5967cefd679a46054c60c7e8d310@databricks.com"},"message":"[SPARK-16118][MLLIB] add getDropLast to OneHotEncoder\n\n## What changes were proposed in this pull request?\n\nWe forgot the getter of `dropLast` in `OneHotEncoder`\n\n## How was this patch tested?\n\nunit test\n\nAuthor: Xiangrui Meng <meng@databricks.com>\n\nCloses #13821 from mengxr/SPARK-16118.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/9493b079a0050f0a6f4936c17622b96fb185b67f"},{"sha":"79aa1d82ca56eb847cbf4ff81de0564b339988f6","author":{"name":"Felix Cheung","email":"00eb71019ad7acfd67cff51fce666aad32ce6eed@hotmail.com"},"message":"[SQL][DOC] SQL programming guide add deprecated methods in 2.0.0\n\n## What changes were proposed in this pull request?\n\nDoc changes\n\n## How was this patch tested?\n\nmanual\n\nliancheng\n\nAuthor: Felix Cheung <felixcheung_m@hotmail.com>\n\nCloses #13827 from felixcheung/sqldocdeprecate.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/79aa1d82ca56eb847cbf4ff81de0564b339988f6"},{"sha":"d77c4e6e2eef24f4276c38b3add8c29bb885f4db","author":{"name":"Xiangrui Meng","email":"e4720f5dc69a5967cefd679a46054c60c7e8d310@databricks.com"},"message":"[MINOR][MLLIB] deprecate setLabelCol in ChiSqSelectorModel\n\n## What changes were proposed in this pull request?\n\nDeprecate `labelCol`, which is not used by ChiSqSelectorModel.\n\nAuthor: Xiangrui Meng <meng@databricks.com>\n\nCloses #13823 from mengxr/deprecate-setLabelCol-in-ChiSqSelectorModel.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/d77c4e6e2eef24f4276c38b3add8c29bb885f4db"},{"sha":"7580f3041a1a3757a0b14b9d8afeb720f261fff6","author":{"name":"hyukjinkwon","email":"4c114bf3676ebcdb1616a11ca478d9181f4025da@gmail.com"},"message":"[SPARK-16104] [SQL] Do not creaate CSV writer object for every flush when writing\n\n## What changes were proposed in this pull request?\n\nThis PR let `CsvWriter` object is not created for each time but able to be reused. This way was taken after from JSON data source.\n\nOriginal `CsvWriter` was being created for each row but it was enhanced in https://github.com/apache/spark/pull/13229. However, it still creates `CsvWriter` object for each `flush()` in `LineCsvWriter`. It seems it does not have to close the object and re-create this for every flush.\n\nIt follows the original logic as it is but `CsvWriter` is reused by reseting `CharArrayWriter`.\n\n## How was this patch tested?\n\nExisting tests should cover this.\n\nAuthor: hyukjinkwon <gurwls223@gmail.com>\n\nCloses #13809 from HyukjinKwon/write-perf.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/7580f3041a1a3757a0b14b9d8afeb720f261fff6"},{"sha":"0e3ce75332dd536c0db8467d456ad46e4bf228f4","author":{"name":"gatorsmile","email":"ea6589371ad7a107e4268706d7d8403e3ba585bb@gmail.com"},"message":"[SPARK-15644][MLLIB][SQL] Replace SQLContext with SparkSession in MLlib\n\n#### What changes were proposed in this pull request?\nThis PR is to use the latest `SparkSession` to replace the existing `SQLContext` in `MLlib`. `SQLContext` is removed from `MLlib`.\n\nAlso fix a test case issue in `BroadcastJoinSuite`.\n\nBTW, `SQLContext` is not being used in the `MLlib` test suites.\n#### How was this patch tested?\nExisting test cases.\n\nAuthor: gatorsmile <gatorsmile@gmail.com>\nAuthor: xiaoli <lixiao1983@gmail.com>\nAuthor: Xiao Li <xiaoli@Xiaos-MacBook-Pro.local>\n\nCloses #13380 from gatorsmile/sqlContextML.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/0e3ce75332dd536c0db8467d456ad46e4bf228f4"},{"sha":"d281b0bafe6aa23085d4d2b68f0ce321f1978b50","author":{"name":"Holden Karau","email":"0dd5affe763abbc7f4de3fb3797050edd8c481d3@us.ibm.com"},"message":"[SPARK-15162][SPARK-15164][PYSPARK][DOCS][ML] update some pydocs\n\n## What changes were proposed in this pull request?\n\nMark ml.classification algorithms as experimental to match Scala algorithms, update PyDoc for for thresholds on `LogisticRegression` to have same level of info as Scala, and enable mathjax for PyDoc.\n\n## How was this patch tested?\n\nBuilt docs locally & PySpark SQL tests\n\nAuthor: Holden Karau <holden@us.ibm.com>\n\nCloses #12938 from holdenk/SPARK-15162-SPARK-15164-update-some-pydocs.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/d281b0bafe6aa23085d4d2b68f0ce321f1978b50"},{"sha":"39ad53f7ffddae5ba0ff0a76089ba671b14c44c8","author":{"name":"Yin Huai","email":"7e8362bd012819bee68b9375d9cefa24afbef722@databricks.com"},"message":"[SPARK-16121] ListingFileCatalog does not list in parallel anymore\n\n## What changes were proposed in this pull request?\nSeems the fix of SPARK-14959 breaks the parallel partitioning discovery. This PR fixes the problem\n\n## How was this patch tested?\nTested manually. (This PR also adds a proper test for SPARK-14959)\n\nAuthor: Yin Huai <yhuai@databricks.com>\n\nCloses #13830 from yhuai/SPARK-16121.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/39ad53f7ffddae5ba0ff0a76089ba671b14c44c8"},{"sha":"01277d4b259dcf9cad25eece1377162b7a8c946d","author":{"name":"Wenchen Fan","email":"51df79715bab6f96b68cf2b8813ef1aaa5547bdb@databricks.com"},"message":"[SPARK-16097][SQL] Encoders.tuple should handle null object correctly\n\n## What changes were proposed in this pull request?\n\nAlthough the top level input object can not be null, but when we use `Encoders.tuple` to combine 2 encoders, their input objects are not top level anymore and can be null. We should handle this case.\n\n## How was this patch tested?\n\nnew test in DatasetSuite\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #13807 from cloud-fan/bug.","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/01277d4b259dcf9cad25eece1377162b7a8c946d"},{"sha":"45409672050b965b3e9876614a4f8b2c2af7df8d","author":{"name":"Yuhao Yang","email":"718d4d5b3fde31fb7d3c3b83ad143ab39ee820cb@gmail.com"},"message":"merge conflict","distinct":true,"url":"https://api.github.com/repos/hhbyyh/spark/commits/45409672050b965b3e9876614a4f8b2c2af7df8d"}]}
{"actor":{"display_login":"hhbyyh"}}