{"push_id":1127230184,"size":6,"distinct_size":6,"ref":"refs/heads/branch-2.0","head":"31fb5fa4042eb46c541e5726a3f14da6f9f6bc2d","before":"6ee1583c34a29f3e0c6e99b1e7ba4fd59f4cdfd9","commits":[{"sha":"9f55951a134baf5916226b1889db45b33a59e5b4","author":{"email":"b226223d7355fe9fd300b8643a53cd1893f12f1f@tw.ibm.com","name":"Liang-Chi Hsieh"},"message":"[SPARK-11753][SQL][TEST-HADOOP2.2] Make allowNonNumericNumbers option work\n\n## What changes were proposed in this pull request?\n\nJackson suppprts `allowNonNumericNumbers` option to parse non-standard non-numeric numbers such as \"NaN\", \"Infinity\", \"INF\".  Currently used Jackson version (2.5.3) doesn't support it all. This patch upgrades the library and make the two ignored tests in `JsonParsingOptionsSuite` passed.\n\n## How was this patch tested?\n\n`JsonParsingOptionsSuite`.\n\nAuthor: Liang-Chi Hsieh <simonh@tw.ibm.com>\nAuthor: Liang-Chi Hsieh <viirya@appier.com>\n\nCloses #9759 from viirya/fix-json-nonnumric.\n\n(cherry picked from commit c24b6b679c3efa053f7de19be73eb36dc70d9930)\nSigned-off-by: Wenchen Fan <wenchen@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/9f55951a134baf5916226b1889db45b33a59e5b4"},{"sha":"4e2a53ba4f0fcd1b5aff0b6428fe8babfd9bb842","author":{"email":"6ff48b33cbb828151035af58a92f18a39f1d1036@apache.org","name":"Dongjoon Hyun"},"message":"[SPARK-13135] [SQL] Don't print expressions recursively in generated code\n\n## What changes were proposed in this pull request?\n\nThis PR is an up-to-date and a little bit improved version of #11019 of rxin for\n- (1) preventing recursive printing of expressions in generated code.\n\nSince the major function of this PR is indeed the above,  he should be credited for the work he did. In addition to #11019, this PR improves the followings in code generation.\n- (2) Improve multiline comment indentation.\n- (3) Reduce the number of empty lines (mainly consecutive empty lines).\n- (4) Remove all space characters on empty lines.\n\n**Example**\n```scala\nspark.range(1, 1000).select('id+1+2+3, 'id+4+5+6)\n```\n\n**Before**\n```\nGenerated code:\n/* 001 */ public Object generate(Object[] references) {\n...\n/* 005 */ /**\n/* 006 */ * Codegend pipeline for\n/* 007 */ * Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 008 */ * +- Range 1, 1, 8, 999, [id#0L]\n/* 009 */ */\n...\n/* 075 */     // PRODUCE: Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 076 */\n/* 077 */     // PRODUCE: Range 1, 1, 8, 999, [id#0L]\n/* 078 */\n/* 079 */     // initialize Range\n...\n/* 092 */       // CONSUME: Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 093 */\n/* 094 */       // CONSUME: WholeStageCodegen\n/* 095 */\n/* 096 */       // (((input[0, bigint, false] + 1) + 2) + 3)\n/* 097 */       // ((input[0, bigint, false] + 1) + 2)\n/* 098 */       // (input[0, bigint, false] + 1)\n...\n/* 107 */       // (((input[0, bigint, false] + 4) + 5) + 6)\n/* 108 */       // ((input[0, bigint, false] + 4) + 5)\n/* 109 */       // (input[0, bigint, false] + 4)\n...\n/* 126 */ }\n```\n\n**After**\n```\nGenerated code:\n/* 001 */ public Object generate(Object[] references) {\n...\n/* 005 */ /**\n/* 006 */  * Codegend pipeline for\n/* 007 */  * Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 008 */  * +- Range 1, 1, 8, 999, [id#0L]\n/* 009 */  */\n...\n/* 075 */     // PRODUCE: Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 076 */     // PRODUCE: Range 1, 1, 8, 999, [id#0L]\n/* 077 */     // initialize Range\n...\n/* 090 */       // CONSUME: Project [(((id#0L + 1) + 2) + 3) AS (((id + 1) + 2) + 3)#3L,(((id#0L + 4) + 5) + 6) AS (((id + 4) + 5) + 6)#4L]\n/* 091 */       // CONSUME: WholeStageCodegen\n/* 092 */       // (((input[0, bigint, false] + 1) + 2) + 3)\n...\n/* 101 */       // (((input[0, bigint, false] + 4) + 5) + 6)\n...\n/* 118 */ }\n```\n\n## How was this patch tested?\n\nPass the Jenkins tests and see the result of the following command manually.\n```scala\nscala> spark.range(1, 1000).select('id+1+2+3, 'id+4+5+6).queryExecution.debug.codegen()\n```\n\nAuthor: Dongjoon Hyun <dongjoonapache.org>\nAuthor: Reynold Xin <rxindatabricks.com>\n\nAuthor: Dongjoon Hyun <dongjoon@apache.org>\n\nCloses #13192 from dongjoon-hyun/SPARK-13135.","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/4e2a53ba4f0fcd1b5aff0b6428fe8babfd9bb842"},{"sha":"988d4dbf427a18b69a7eba25a9b786bc48cea637","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-15405][YARN] Remove unnecessary upload of config archive.\n\nWe only need one copy of it. The client code that was uploading the\nsecond copy just needs to be modified to update the metadata in the\ncache, so that the AM knows where to find the configuration.\n\nTested by running app on YARN and verifying in the logs only one archive\nis uploaded.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #13232 from vanzin/SPARK-15405.\n\n(cherry picked from commit a313a5ae74ae4e7686283657ba56076222317595)\nSigned-off-by: Marcelo Vanzin <vanzin@cloudera.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/988d4dbf427a18b69a7eba25a9b786bc48cea637"},{"sha":"1bb0aa4b0a5b680e535cac6305b9eac856606ef4","author":{"email":"e28ad19e4e56395ff72e0397107366a06912a175@haizhi.com","name":"wangyang"},"message":"[SPARK-15388][SQL] Fix spark sql CREATE FUNCTION with hive 1.2.1\n\n## What changes were proposed in this pull request?\n\nspark.sql(\"CREATE FUNCTION myfunc AS 'com.haizhi.bdp.udf.UDFGetGeoCode'\") throws \"org.apache.hadoop.hive.ql.metadata.HiveException:MetaException(message:NoSuchObjectException(message:Function default.myfunc does not exist))\" with hive 1.2.1.\n\nI think it is introduced by pr #12853. Fixing it by catching Exception (not NoSuchObjectException) and string matching.\n\n## How was this patch tested?\n\nadded a unit test and also tested it manually\n\nAuthor: wangyang <wangyang@haizhi.com>\n\nCloses #13177 from wangyang1992/fixCreateFunc2.\n\n(cherry picked from commit 784cc07d1675eb9e0a387673cf86874e1bfc10f9)\nSigned-off-by: Andrew Or <andrew@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/1bb0aa4b0a5b680e535cac6305b9eac856606ef4"},{"sha":"2574abea088be725b6ecf2c473819535cd1cf0a5","author":{"email":"6ff48b33cbb828151035af58a92f18a39f1d1036@apache.org","name":"Dongjoon Hyun"},"message":"[MINOR][CORE][TEST] Update obsolete `takeSample` test case.\n\n## What changes were proposed in this pull request?\n\nThis PR fixes some obsolete comments and assertion in `takeSample` testcase of `RDDSuite.scala`.\n\n## How was this patch tested?\n\nThis fixes the testcase only.\n\nAuthor: Dongjoon Hyun <dongjoon@apache.org>\n\nCloses #13260 from dongjoon-hyun/SPARK-15481.\n\n(cherry picked from commit be99a99fe7976419d727c0cc92e872aa4af58bf1)\nSigned-off-by: Andrew Or <andrew@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/2574abea088be725b6ecf2c473819535cd1cf0a5"},{"sha":"31fb5fa4042eb46c541e5726a3f14da6f9f6bc2d","author":{"email":"6fe12bfdafd6d3c41da5ad6593c53ed449c85282@za.ibm.com","name":"Nick Pentreath"},"message":"[SPARK-15502][DOC][ML][PYSPARK] add guide note that ALS only supports integer ids\n\nThis PR adds a note to clarify that the ML API for ALS only supports integers for user/item ids, and that other types for these columns can be used but the ids must fall within integer range.\n\n(Refer [SPARK-14891](https://issues.apache.org/jira/browse/SPARK-14891)).\n\nAlso cleaned up a reference to `mllib` in the ML doc.\n\n## How was this patch tested?\nBuilt and viewed User Guide doc locally.\n\nAuthor: Nick Pentreath <nickp@za.ibm.com>\n\nCloses #13278 from MLnick/SPARK-15502-als-int-id-doc-note.\n\n(cherry picked from commit 20900e5feced76e87f0a12823d0e3f07e082105f)\nSigned-off-by: Joseph K. Bradley <joseph@databricks.com>","distinct":true,"url":"https://api.github.com/repos/ueshin/apache-spark/commits/31fb5fa4042eb46c541e5726a3f14da6f9f6bc2d"}]}
{"actor":{"display_login":"ueshin"}}