{"push_id":1140434547,"size":132,"distinct_size":17,"ref":"refs/heads/SPARK-14480","head":"7abdfc111166f2bf275fc4318c0ffe8836dcbb70","before":"66b17570a8d1ad53b5073bbfa439eb01b05413c1","commits":[{"sha":"d207716451f722c899b3845ee454f1e16c094125","author":{"name":"gatorsmile","email":"ea6589371ad7a107e4268706d7d8403e3ba585bb@gmail.com"},"message":"[SPARK-15485][SQL][DOCS] Spark SQL Configuration\n\n#### What changes were proposed in this pull request?\nSo far, the page Configuration in the official documentation does not have a section for Spark SQL.\nhttp://spark.apache.org/docs/latest/configuration.html\n\nFor Spark users, the information and default values of these public configuration parameters are very useful. This PR is to add this missing section to the configuration.html.\n\nrxin yhuai marmbrus\n\n#### How was this patch tested?\nBelow is the generated webpage.\n<img width=\"924\" alt=\"screenshot 2016-05-23 11 35 57\" src=\"https://cloud.githubusercontent.com/assets/11567269/15480492/b08fefc4-20da-11e6-9fa2-7cd5b699ed35.png\">\n<img width=\"914\" alt=\"screenshot 2016-05-23 11 37 38\" src=\"https://cloud.githubusercontent.com/assets/11567269/15480499/c5f9482e-20da-11e6-95ff-10821add1af4.png\">\n<img width=\"923\" alt=\"screenshot 2016-05-23 11 36 11\" src=\"https://cloud.githubusercontent.com/assets/11567269/15480506/cbd81644-20da-11e6-9d27-effb716b2fac.png\">\n<img width=\"920\" alt=\"screenshot 2016-05-23 11 36 18\" src=\"https://cloud.githubusercontent.com/assets/11567269/15480511/d013e332-20da-11e6-854a-cf8813c46f36.png\">\n\nAuthor: gatorsmile <gatorsmile@gmail.com>\n\nCloses #13263 from gatorsmile/configurationSQL.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/d207716451f722c899b3845ee454f1e16c094125"},{"sha":"fa244e5a90690d6a31be50f2aa203ae1a2e9a1cf","author":{"name":"Kazuaki Ishizaki","email":"64981c6bd46971e76f308b8a384367244e8169bf@jp.ibm.com"},"message":"[SPARK-15285][SQL] Generated SpecificSafeProjection.apply method grows beyond 64 KB\n\n## What changes were proposed in this pull request?\n\nThis PR splits the generated code for ```SafeProjection.apply``` by using ```ctx.splitExpressions()```. This is because the large code body for ```NewInstance``` may grow beyond 64KB bytecode size for ```apply()``` method.\n\n## How was this patch tested?\n\nAdded new tests\n\nAuthor: Kazuaki Ishizaki <ishizaki@jp.ibm.com>\n\nCloses #13243 from kiszk/SPARK-15285.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/fa244e5a90690d6a31be50f2aa203ae1a2e9a1cf"},{"sha":"de726b0d533158d3ca08841bd6976bcfa26ca79d","author":{"name":"Andrew Or","email":"6b7c097d961f68685fc978cf91eeae17be6cbb45@databricks.com"},"message":"Revert \"[SPARK-15285][SQL] Generated SpecificSafeProjection.apply method grows beyond 64 KB\"\n\nThis reverts commit fa244e5a90690d6a31be50f2aa203ae1a2e9a1cf.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/de726b0d533158d3ca08841bd6976bcfa26ca79d"},{"sha":"d642b273544bb77ef7f584326aa2d214649ac61b","author":{"name":"Daoyuan Wang","email":"ca9858c9dc0e8e1b4a4756052ed9da2bddb5bfdf@intel.com"},"message":"[SPARK-15397][SQL] fix string udf locate as hive\n\n## What changes were proposed in this pull request?\n\nin hive, `locate(\"aa\", \"aaa\", 0)` would yield 0, `locate(\"aa\", \"aaa\", 1)` would yield 1 and `locate(\"aa\", \"aaa\", 2)` would yield 2, while in Spark, `locate(\"aa\", \"aaa\", 0)` would yield 1,  `locate(\"aa\", \"aaa\", 1)` would yield 2 and  `locate(\"aa\", \"aaa\", 2)` would yield 0. This results from the different understanding of the third parameter in udf `locate`. It means the starting index and starts from 1, so when we use 0, the return would always be 0.\n\n## How was this patch tested?\n\ntested with modified `StringExpressionsSuite` and `StringFunctionsSuite`\n\nAuthor: Daoyuan Wang <daoyuan.wang@intel.com>\n\nCloses #13186 from adrian-wang/locate.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/d642b273544bb77ef7f584326aa2d214649ac61b"},{"sha":"6075f5b4d8e98483d26c31576f58e2229024b4f4","author":{"name":"Nick Pentreath","email":"01c7185e211fff3925ce587674cd3ea63724775a@za.ibm.com"},"message":"[SPARK-15442][ML][PYSPARK] Add 'relativeError' param to PySpark QuantileDiscretizer\n\nThis PR adds the `relativeError` param to PySpark's `QuantileDiscretizer` to match Scala.\n\nAlso cleaned up a duplication of `numBuckets` where the param is both a class and instance attribute (I removed the instance attr to match the style of params throughout `ml`).\n\nFinally, cleaned up the docs for `QuantileDiscretizer` to reflect that it now uses `approxQuantile`.\n\n## How was this patch tested?\n\nA little doctest and built API docs locally to check HTML doc generation.\n\nAuthor: Nick Pentreath <nickp@za.ibm.com>\n\nCloses #13228 from MLnick/SPARK-15442-py-relerror-param.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/6075f5b4d8e98483d26c31576f58e2229024b4f4"},{"sha":"c24b6b679c3efa053f7de19be73eb36dc70d9930","author":{"name":"Liang-Chi Hsieh","email":"36eec8bb19b3166395ab9bc07614f8325d08f990@tw.ibm.com"},"message":"[SPARK-11753][SQL][TEST-HADOOP2.2] Make allowNonNumericNumbers option work\n\n## What changes were proposed in this pull request?\n\nJackson suppprts `allowNonNumericNumbers` option to parse non-standard non-numeric numbers such as \"NaN\", \"Infinity\", \"INF\".  Currently used Jackson version (2.5.3) doesn't support it all. This patch upgrades the library and make the two ignored tests in `JsonParsingOptionsSuite` passed.\n\n## How was this patch tested?\n\n`JsonParsingOptionsSuite`.\n\nAuthor: Liang-Chi Hsieh <simonh@tw.ibm.com>\nAuthor: Liang-Chi Hsieh <viirya@appier.com>\n\nCloses #9759 from viirya/fix-json-nonnumric.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/c24b6b679c3efa053f7de19be73eb36dc70d9930"},{"sha":"f8763b80ecd9968566018396c8cdc1851e7f8a46","author":{"name":"Dongjoon Hyun","email":"1bf3a3c57610939bd1558dffb564906c959abecb@apache.org"},"message":"[SPARK-13135] [SQL] Don't print expressions recursively in generated code\n\n## What changes were proposed in this pull request?\n\nThis PR is an up-to-date and a little bit improved version of #11019 of rxin for\n- (1) preventing recursive printing of expressions in generated code.\n\nSince the major function of this PR is indeed the above,  he should be credited for the work he did. In addition to #11019, this PR improves the followings in code generation.\n- (2) Improve multiline comment indentation.\n- (3) Reduce the number of empty lines (mainly consecutive empty lines).\n- (4) Remove all space characters on empty lines.\n\n**Example**\n```scala\nspark.range(1, 1000).select('id+1+2+3, 'id+4+5+6)\n```\n\n**Before**\n```\nGenerated code:\n/* 001 */ public Object generate(Object[] references) {\n...\n/* 005 */ /**\n/* 006 */ * Codegend pipeline for\n/* 007 */ * Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 008 */ * +- Range 1, 1, 8, 999, [id#0L]\n/* 009 */ */\n...\n/* 075 */     // PRODUCE: Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 076 */\n/* 077 */     // PRODUCE: Range 1, 1, 8, 999, [id#0L]\n/* 078 */\n/* 079 */     // initialize Range\n...\n/* 092 */       // CONSUME: Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 093 */\n/* 094 */       // CONSUME: WholeStageCodegen\n/* 095 */\n/* 096 */       // (((input[0, bigint, false] + 1) + 2) + 3)\n/* 097 */       // ((input[0, bigint, false] + 1) + 2)\n/* 098 */       // (input[0, bigint, false] + 1)\n...\n/* 107 */       // (((input[0, bigint, false] + 4) + 5) + 6)\n/* 108 */       // ((input[0, bigint, false] + 4) + 5)\n/* 109 */       // (input[0, bigint, false] + 4)\n...\n/* 126 */ }\n```\n\n**After**\n```\nGenerated code:\n/* 001 */ public Object generate(Object[] references) {\n...\n/* 005 */ /**\n/* 006 */  * Codegend pipeline for\n/* 007 */  * Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 008 */  * +- Range 1, 1, 8, 999, [id#0L]\n/* 009 */  */\n...\n/* 075 */     // PRODUCE: Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 076 */     // PRODUCE: Range 1, 1, 8, 999, [id#0L]\n/* 077 */     // initialize Range\n...\n/* 090 */       // CONSUME: Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 091 */       // CONSUME: WholeStageCodegen\n/* 092 */       // (((input[0, bigint, false] + 1) + 2) + 3)\n...\n/* 101 */       // (((input[0, bigint, false] + 4) + 5) + 6)\n...\n/* 118 */ }\n```\n\n## How was this patch tested?\n\nPass the Jenkins tests and see the result of the following command manually.\n```scala\nscala> spark.range(1, 1000).select('id+1+2+3, 'id+4+5+6).queryExecution.debug.codegen()\n```\n\nAuthor: Dongjoon Hyun <dongjoonapache.org>\nAuthor: Reynold Xin <rxindatabricks.com>\n\nAuthor: Dongjoon Hyun <dongjoon@apache.org>\n\nCloses #13192 from dongjoon-hyun/SPARK-13135.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/f8763b80ecd9968566018396c8cdc1851e7f8a46"},{"sha":"695d9a0fd461070ee2684b2210fb69d0b6ed1a95","author":{"name":"Liang-Chi Hsieh","email":"36eec8bb19b3166395ab9bc07614f8325d08f990@tw.ibm.com"},"message":"[SPARK-15433] [PYSPARK] PySpark core test should not use SerDe from PythonMLLibAPI\n\n## What changes were proposed in this pull request?\n\nCurrently PySpark core test uses the `SerDe` from `PythonMLLibAPI` which includes many MLlib things. It should use `SerDeUtil` instead.\n\n## How was this patch tested?\nExisting tests.\n\nAuthor: Liang-Chi Hsieh <simonh@tw.ibm.com>\n\nCloses #13214 from viirya/pycore-use-serdeutil.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/695d9a0fd461070ee2684b2210fb69d0b6ed1a95"},{"sha":"a313a5ae74ae4e7686283657ba56076222317595","author":{"name":"Marcelo Vanzin","email":"31a02c4d82e3479e776883a3423bc52579c6a1cf@cloudera.com"},"message":"[SPARK-15405][YARN] Remove unnecessary upload of config archive.\n\nWe only need one copy of it. The client code that was uploading the\nsecond copy just needs to be modified to update the metadata in the\ncache, so that the AM knows where to find the configuration.\n\nTested by running app on YARN and verifying in the logs only one archive\nis uploaded.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #13232 from vanzin/SPARK-15405.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/a313a5ae74ae4e7686283657ba56076222317595"},{"sha":"784cc07d1675eb9e0a387673cf86874e1bfc10f9","author":{"name":"wangyang","email":"eba22d259fe8b5d85428661d5efb47a9dcdcf825@haizhi.com"},"message":"[SPARK-15388][SQL] Fix spark sql CREATE FUNCTION with hive 1.2.1\n\n## What changes were proposed in this pull request?\n\nspark.sql(\"CREATE FUNCTION myfunc AS 'com.haizhi.bdp.udf.UDFGetGeoCode'\") throws \"org.apache.hadoop.hive.ql.metadata.HiveException:MetaException(message:NoSuchObjectException(message:Function default.myfunc does not exist))\" with hive 1.2.1.\n\nI think it is introduced by pr #12853. Fixing it by catching Exception (not NoSuchObjectException) and string matching.\n\n## How was this patch tested?\n\nadded a unit test and also tested it manually\n\nAuthor: wangyang <wangyang@haizhi.com>\n\nCloses #13177 from wangyang1992/fixCreateFunc2.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/784cc07d1675eb9e0a387673cf86874e1bfc10f9"},{"sha":"be99a99fe7976419d727c0cc92e872aa4af58bf1","author":{"name":"Dongjoon Hyun","email":"1bf3a3c57610939bd1558dffb564906c959abecb@apache.org"},"message":"[MINOR][CORE][TEST] Update obsolete `takeSample` test case.\n\n## What changes were proposed in this pull request?\n\nThis PR fixes some obsolete comments and assertion in `takeSample` testcase of `RDDSuite.scala`.\n\n## How was this patch tested?\n\nThis fixes the testcase only.\n\nAuthor: Dongjoon Hyun <dongjoon@apache.org>\n\nCloses #13260 from dongjoon-hyun/SPARK-15481.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/be99a99fe7976419d727c0cc92e872aa4af58bf1"},{"sha":"20900e5feced76e87f0a12823d0e3f07e082105f","author":{"name":"Nick Pentreath","email":"01c7185e211fff3925ce587674cd3ea63724775a@za.ibm.com"},"message":"[SPARK-15502][DOC][ML][PYSPARK] add guide note that ALS only supports integer ids\n\nThis PR adds a note to clarify that the ML API for ALS only supports integers for user/item ids, and that other types for these columns can be used but the ids must fall within integer range.\n\n(Refer [SPARK-14891](https://issues.apache.org/jira/browse/SPARK-14891)).\n\nAlso cleaned up a reference to `mllib` in the ML doc.\n\n## How was this patch tested?\nBuilt and viewed User Guide doc locally.\n\nAuthor: Nick Pentreath <nickp@za.ibm.com>\n\nCloses #13278 from MLnick/SPARK-15502-als-int-id-doc-note.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/20900e5feced76e87f0a12823d0e3f07e082105f"},{"sha":"e631b819fe348729aab062207a452b8f1d1511bd","author":{"name":"Tathagata Das","email":"58b3b6773dfe925407947ae07f6b206d48277ab4@gmail.com"},"message":"[SPARK-15458][SQL][STREAMING] Disable schema inference for streaming datasets on file streams\n\n## What changes were proposed in this pull request?\n\nIf the user relies on the schema to be inferred in file streams can break easily for multiple reasons\n- accidentally running on a directory which has no data\n- schema changing underneath\n- on restart, the query will infer schema again, and may unexpectedly infer incorrect schema, as the file in the directory may be different at the time of the restart.\n\nTo avoid these complicated scenarios, for Spark 2.0, we are going to disable schema inferencing by default with a config, so that user is forced to consider explicitly what is the schema it wants, rather than the system trying to infer it and run into weird corner cases.\n\nIn this PR, I introduce a SQLConf that determines whether schema inference for file streams is allowed or not. It is disabled by default.\n\n## How was this patch tested?\nUpdated unit tests that test error behavior with and without schema inference enabled.\n\nAuthor: Tathagata Das <tathagata.das1565@gmail.com>\n\nCloses #13238 from tdas/SPARK-15458.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/e631b819fe348729aab062207a452b8f1d1511bd"},{"sha":"f08bf587b1913c6cc8ecb34c45331cf4750961c9","author":{"name":"Dongjoon Hyun","email":"1bf3a3c57610939bd1558dffb564906c959abecb@apache.org"},"message":"[SPARK-15512][CORE] repartition(0) should raise IllegalArgumentException\n\n## What changes were proposed in this pull request?\n\nPreviously, SPARK-8893 added the constraints on positive number of partitions for repartition/coalesce operations in general. This PR adds one missing part for that and adds explicit two testcases.\n\n**Before**\n```scala\nscala> sc.parallelize(1 to 5).coalesce(0)\njava.lang.IllegalArgumentException: requirement failed: Number of partitions (0) must be positive.\n...\nscala> sc.parallelize(1 to 5).repartition(0).collect()\nres1: Array[Int] = Array()   // empty\nscala> spark.sql(\"select 1\").coalesce(0)\nres2: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [1: int]\nscala> spark.sql(\"select 1\").coalesce(0).collect()\njava.lang.IllegalArgumentException: requirement failed: Number of partitions (0) must be positive.\nscala> spark.sql(\"select 1\").repartition(0)\nres3: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row] = [1: int]\nscala> spark.sql(\"select 1\").repartition(0).collect()\nres4: Array[org.apache.spark.sql.Row] = Array()  // empty\n```\n\n**After**\n```scala\nscala> sc.parallelize(1 to 5).coalesce(0)\njava.lang.IllegalArgumentException: requirement failed: Number of partitions (0) must be positive.\n...\nscala> sc.parallelize(1 to 5).repartition(0)\njava.lang.IllegalArgumentException: requirement failed: Number of partitions (0) must be positive.\n...\nscala> spark.sql(\"select 1\").coalesce(0)\njava.lang.IllegalArgumentException: requirement failed: Number of partitions (0) must be positive.\n...\nscala> spark.sql(\"select 1\").repartition(0)\njava.lang.IllegalArgumentException: requirement failed: Number of partitions (0) must be positive.\n...\n```\n\n## How was this patch tested?\n\nPass the Jenkins tests with new testcases.\n\nAuthor: Dongjoon Hyun <dongjoon@apache.org>\n\nCloses #13282 from dongjoon-hyun/SPARK-15512.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/f08bf587b1913c6cc8ecb34c45331cf4750961c9"},{"sha":"14494da87bdf057d2d2f796b962a4d8bc4747d31","author":{"name":"Reynold Xin","email":"9a371236ace08d82aa86864cade97762c12d4503@databricks.com"},"message":"[SPARK-15518] Rename various scheduler backend for consistency\n\n## What changes were proposed in this pull request?\nThis patch renames various scheduler backends to make them consistent:\n\n- LocalScheduler -> LocalSchedulerBackend\n- AppClient -> StandaloneAppClient\n- AppClientListener -> StandaloneAppClientListener\n- SparkDeploySchedulerBackend -> StandaloneSchedulerBackend\n- CoarseMesosSchedulerBackend -> MesosCoarseGrainedSchedulerBackend\n- MesosSchedulerBackend -> MesosFineGrainedSchedulerBackend\n\n## How was this patch tested?\nUpdated test cases to reflect the name change.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #13288 from rxin/SPARK-15518.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/14494da87bdf057d2d2f796b962a4d8bc4747d31"},{"sha":"4acababcaba567c85f3be0d5e939d99119b82d1d","author":{"name":"Parth Brahmbhatt","email":"e727429cd656096f2bcabde94f07eaa4f20221a7@netflix.com"},"message":"[SPARK-15365][SQL] When table size statistics are not available from metastore, we should fallback to HDFS\n\n## What changes were proposed in this pull request?\nCurrently if a table is used in join operation we rely on Metastore returned size to calculate if we can convert the operation to Broadcast join. This optimization only kicks in for table's that have the statistics available in metastore. Hive generally rolls over to HDFS if the statistics are not available directly from metastore and this seems like a reasonable choice to adopt given the optimization benefit of using broadcast joins.\n\n## How was this patch tested?\nI have executed queries locally to test.\n\nAuthor: Parth Brahmbhatt <pbrahmbhatt@netflix.com>\n\nCloses #13150 from Parth-Brahmbhatt/SPARK-15365.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/4acababcaba567c85f3be0d5e939d99119b82d1d"},{"sha":"50b660d725269dc0c11e0d350ddd7fc8b19539a0","author":{"name":"Wenchen Fan","email":"51df79715bab6f96b68cf2b8813ef1aaa5547bdb@databricks.com"},"message":"[SPARK-15498][TESTS] fix slow tests\n\n## What changes were proposed in this pull request?\n\nThis PR fixes 3 slow tests:\n\n1. `ParquetQuerySuite.read/write wide table`: This is not a good unit test as it runs more than 5 minutes. This PR removes it and add a new regression test in `CodeGenerationSuite`, which is more \"unit\".\n2. `ParquetQuerySuite.returning batch for wide table`: reduce the threshold and use smaller data size.\n3. `DatasetSuite.SPARK-14554: Dataset.map may generate wrong java code for wide table`: Improve `CodeFormatter.format`(introduced at https://github.com/apache/spark/pull/12979) can dramatically speed this it up.\n\n## How was this patch tested?\n\nN/A\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #13273 from cloud-fan/test.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/50b660d725269dc0c11e0d350ddd7fc8b19539a0"},{"sha":"c9c1c0e54d34773ac2cf5457fe5925559ece36c7","author":{"name":"Shixiong Zhu","email":"eb2f6da57ec54996cebae8c2a82b9fbfd527e14c@databricks.com"},"message":"[SPARK-15508][STREAMING][TESTS] Fix flaky test: JavaKafkaStreamSuite.testKafkaStream\n\n## What changes were proposed in this pull request?\n\n`JavaKafkaStreamSuite.testKafkaStream` assumes when `sent.size == result.size`, the contents of `sent` and `result` should be same. However, that's not true. The content of `result` may not be the final content.\n\nThis PR modified the test to always retry the assertions even if the contents of `sent` and `result` are not same.\n\nHere is the failure in Jenkins: http://spark-tests.appspot.com/tests/org.apache.spark.streaming.kafka.JavaKafkaStreamSuite/testKafkaStream\n\n## How was this patch tested?\n\nJenkins unit tests.\n\nAuthor: Shixiong Zhu <shixiong@databricks.com>\n\nCloses #13281 from zsxwing/flaky-kafka-test.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/c9c1c0e54d34773ac2cf5457fe5925559ece36c7"},{"sha":"cd9f16906cabd012b7676eb0f524e68a9cbe4db1","author":{"name":"Holden Karau","email":"0dd5affe763abbc7f4de3fb3797050edd8c481d3@us.ibm.com"},"message":"[SPARK-15412][PYSPARK][SPARKR][DOCS] Improve linear isotonic regression pydoc & doc build insturctions\n\n## What changes were proposed in this pull request?\n\nPySpark: Add links to the predictors from the models in regression.py, improve linear and isotonic pydoc in minor ways.\nUser guide / R: Switch the installed package list to be enough to build the R docs on a \"fresh\" install on ubuntu and add sudo to match the rest of the commands.\nUser Guide: Add a note about using gem2.0 for systems with both 1.9 and 2.0 (e.g. some ubuntu but maybe more).\n\n## How was this patch tested?\n\nbuilt pydocs locally, tested new user build instructions\n\nAuthor: Holden Karau <holden@us.ibm.com>\n\nCloses #13199 from holdenk/SPARK-15412-improve-linear-isotonic-regression-pydoc.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/cd9f16906cabd012b7676eb0f524e68a9cbe4db1"},{"sha":"9082b7968ad952e05fc6f4feb499febef6aa45a7","author":{"name":"Krishna Kalyan","email":"914fd6fe19f18e9d18d9b734c46138271e4880d2@gmail.com"},"message":"[SPARK-12071][DOC] Document the behaviour of NA in R\n\n## What changes were proposed in this pull request?\n\nUnder Upgrading From SparkR 1.5.x to 1.6.x section added the information, SparkSQL converts `NA` in R to `null`.\n\n## How was this patch tested?\n\nDocument update, no tests.\n\nAuthor: Krishna Kalyan <krishnakalyan3@gmail.com>\n\nCloses #13268 from krishnakalyan3/spark-12071-1.","distinct":false,"url":"https://api.github.com/repos/HyukjinKwon/spark/commits/9082b7968ad952e05fc6f4feb499febef6aa45a7"}]}
{"actor":{"display_login":"HyukjinKwon"}}