{"push_id":2528878725,"size":1,"distinct_size":1,"ref":"refs/heads/csd-2.2","head":"7ab2930e002c047d8e4c32d0f51ba0eafe5c997f","before":"2af28fc60eb6e9c099e46aed9811f7ce18087ce5","commits":[{"sha":"7ab2930e002c047d8e4c32d0f51ba0eafe5c997f","author":{"name":"Mark Hamstra","email":"47f947204c0a430d1f3d77ee35599c230fda5709@gmail.com"},"message":"Branch 2.2 merge (#230)\n\n* [SPARK-23816][CORE] Killed tasks should ignore FetchFailures.\n\nSPARK-19276 ensured that FetchFailures do not get swallowed by other\nlayers of exception handling, but it also meant that a killed task could\nlook like a fetch failure.  This is particularly a problem with\nspeculative execution, where we expect to kill tasks as they are reading\nshuffle data.  The fix is to ensure that we always check for killed\ntasks first.\n\nAdded a new unit test which fails before the fix, ran it 1k times to\ncheck for flakiness.  Full suite of tests on jenkins.\n\nAuthor: Imran Rashid <irashid@cloudera.com>\n\nCloses #20987 from squito/SPARK-23816.\n\n(cherry picked from commit 10f45bb8233e6ac838dd4f053052c8556f5b54bd)\nSigned-off-by: Marcelo Vanzin <vanzin@cloudera.com>\n\n* [SPARK-24007][SQL] EqualNullSafe for FloatType and DoubleType might generate a wrong result by codegen.\n\n`EqualNullSafe` for `FloatType` and `DoubleType` might generate a wrong result by codegen.\n\n```scala\nscala> val df = Seq((Some(-1.0d), None), (None, Some(-1.0d))).toDF()\ndf: org.apache.spark.sql.DataFrame = [_1: double, _2: double]\n\nscala> df.show()\n+----+----+\n|  _1|  _2|\n+----+----+\n|-1.0|null|\n|null|-1.0|\n+----+----+\n\nscala> df.filter(\"_1 <=> _2\").show()\n+----+----+\n|  _1|  _2|\n+----+----+\n|-1.0|null|\n|null|-1.0|\n+----+----+\n```\n\nThe result should be empty but the result remains two rows.\n\nAdded a test.\n\nAuthor: Takuya UESHIN <ueshin@databricks.com>\n\nCloses #21094 from ueshin/issues/SPARK-24007/equalnullsafe.\n\n(cherry picked from commit f09a9e9418c1697d198de18f340b1288f5eb025c)\nSigned-off-by: gatorsmile <gatorsmile@gmail.com>\n\n* [SPARK-23963][SQL] Properly handle large number of columns in query on text-based Hive table\n\n## What changes were proposed in this pull request?\n\nTableReader would get disproportionately slower as the number of columns in the query increased.\n\nI fixed the way TableReader was looking up metadata for each column in the row. Previously, it had been looking up this data in linked lists, accessing each linked list by an index (column number). Now it looks up this data in arrays, where indexing by column number works better.\n\n## How was this patch tested?\n\nManual testing\nAll sbt unit tests\npython sql tests\n\nAuthor: Bruce Robbins <bersprockets@gmail.com>\n\nCloses #21043 from bersprockets/tabreadfix.\n\n* [MINOR][DOCS] Fix comments of SQLExecution#withExecutionId\n\n## What changes were proposed in this pull request?\nFix comment. Change `BroadcastHashJoin.broadcastFuture` to `BroadcastExchangeExec.relationFuture`: https://github.com/apache/spark/blob/d28d5732ae205771f1f443b15b10e64dcffb5ff0/sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala#L66\n\n## How was this patch tested?\nN/A\n\nAuthor: seancxmao <seancxmao@gmail.com>\n\nCloses #21113 from seancxmao/SPARK-13136.\n\n(cherry picked from commit c303b1b6766a3dc5961713f98f62cd7d7ac7972a)\nSigned-off-by: hyukjinkwon <gurwls223@apache.org>","distinct":true,"url":"https://api.github.com/repos/clearstorydata/spark/commits/7ab2930e002c047d8e4c32d0f51ba0eafe5c997f"}]}
{"actor":{"display_login":"csd-jenkins"}}