{"push_id":2543026331,"size":1000,"distinct_size":1,"ref":"refs/heads/revise-udt","head":"9fd6c2f21353b41e11748a3f9edaf17647c8bd7f","before":"e6b01f07947da06be2fc3114793d7793a0f7406a","commits":[{"sha":"70221903f54eaa0514d5d189dfb6f175a62228a8","author":{"name":"Wenchen Fan","email":"563f5923eae9a92c8a3f34a26b67fa348a303ed1@databricks.com"},"message":"[SPARK-22596][SQL] set ctx.currentVars in CodegenSupport.consume\n\n## What changes were proposed in this pull request?\n\n`ctx.currentVars` means the input variables for the current operator, which is already decided in `CodegenSupport`, we can set it there instead of `doConsume`.\n\nalso add more comments to help people understand the codegen framework.\n\nAfter this PR, we now have a principle about setting `ctx.currentVars` and `ctx.INPUT_ROW`:\n1. for non-whole-stage-codegen path, never set them. (permit some special cases like generating ordering)\n2. for whole-stage-codegen `produce` path, mostly we don't need to set them, but blocking operators may need to set them for expressions that produce data from data source, sort buffer, aggregate buffer, etc.\n3. for whole-stage-codegen `consume` path, mostly we don't need to set them because `currentVars` is automatically set to child input variables and `INPUT_ROW` is mostly not used. A few plans need to tweak them as they may have different inputs, or they use the input row.\n\n## How was this patch tested?\n\nexisting tests.\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #19803 from cloud-fan/codegen.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/70221903f54eaa0514d5d189dfb6f175a62228a8"},{"sha":"e3fd93f149ff0ff1caff28a5191215e2a29749a9","author":{"name":"Wenchen Fan","email":"563f5923eae9a92c8a3f34a26b67fa348a303ed1@databricks.com"},"message":"[SPARK-22604][SQL] remove the get address methods from ColumnVector\n\n## What changes were proposed in this pull request?\n\n`nullsNativeAddress` and `valuesNativeAddress` are only used in tests and benchmark, no need to be top class API.\n\n## How was this patch tested?\n\nexisting tests\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #19818 from cloud-fan/minor.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/e3fd93f149ff0ff1caff28a5191215e2a29749a9"},{"sha":"4d8ace48698c9a5e45cfc896b170914f1c21dc7b","author":{"name":"Kalvin Chau","email":"1a6237d0a1c35a017a1e1fb43a3ce5e0944f7f8a@viasat.com"},"message":"[SPARK-22583] First delegation token renewal time is not 75% of renewal time in Mesos\n\nThe first scheduled renewal time is is set to the exact expiration time,\nand all subsequent renewal times are 75% of the renewal time. This makes\nit so that the inital renewal time is also 75%.\n\n## What changes were proposed in this pull request?\n\nSet the initial renewal time to be 75% of renewal time.\n\n## How was this patch tested?\n\nTested locally in a test HDFS cluster, checking various renewal times.\n\nAuthor: Kalvin Chau <kalvin.chau@viasat.com>\n\nCloses #19798 from kalvinnchau/fix-inital-renewal-time.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/4d8ace48698c9a5e45cfc896b170914f1c21dc7b"},{"sha":"fba63c1a7bc5c907b909bfd9247b85b36efba469","author":{"name":"Sean Owen","email":"291c18f3fb7528c712d9098b0e50a515ea0b91d5@cloudera.com"},"message":"[SPARK-22607][BUILD] Set large stack size consistently for tests to avoid StackOverflowError\n\n## What changes were proposed in this pull request?\n\nSet `-ea` and `-Xss4m` consistently for tests, to fix in particular:\n\n```\nOrderingSuite:\n...\n- GenerateOrdering with ShortType\n*** RUN ABORTED ***\njava.lang.StackOverflowError:\nat org.codehaus.janino.CodeContext.flowAnalysis(CodeContext.java:370)\nat org.codehaus.janino.CodeContext.flowAnalysis(CodeContext.java:541)\nat org.codehaus.janino.CodeContext.flowAnalysis(CodeContext.java:541)\nat org.codehaus.janino.CodeContext.flowAnalysis(CodeContext.java:541)\nat org.codehaus.janino.CodeContext.flowAnalysis(CodeContext.java:541)\nat org.codehaus.janino.CodeContext.flowAnalysis(CodeContext.java:541)\nat org.codehaus.janino.CodeContext.flowAnalysis(CodeContext.java:541)\nat org.codehaus.janino.CodeContext.flowAnalysis(CodeContext.java:541)\n...\n```\n\n## How was this patch tested?\n\nExisting tests. Manually verified it resolves the StackOverflowError this intends to resolve.\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #19820 from srowen/SPARK-22607.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/fba63c1a7bc5c907b909bfd9247b85b36efba469"},{"sha":"d49d9e40383209eed9584a4ef2c3964f27f4a08f","author":{"name":"hyukjinkwon","email":"2680965ad5aae29275c8fb10150cb10f5554e78b@gmail.com"},"message":"[SPARK-21693][R][FOLLOWUP] Reduce shuffle partitions running R worker in few tests to speed up\n\n## What changes were proposed in this pull request?\n\nThis is a followup to reduce AppVeyor test time. This PR proposes to reduce the number of shuffle partitions to reduce the tasks running R workers in few particular tests.\n\nThe symptom is similar as described in `https://github.com/apache/spark/pull/19722`. There are many R processes newly launched on Windows without forking and it makes the differences of elapsed time between Linux and Windows.\n\nHere is the simple comparison for before/after of this change. I manually tested this by disabling `spark.sparkr.use.daemon`. Disabling it resembles the tests on Windows:\n\n**Before**\n\n<img width=\"672\" alt=\"2017-11-25 12 22 13\" src=\"https://user-images.githubusercontent.com/6477701/33217949-b5528dfa-d17d-11e7-8050-75675c39eb20.png\">\n\n**After**\n\n<img width=\"682\" alt=\"2017-11-25 12 32 00\" src=\"https://user-images.githubusercontent.com/6477701/33217958-c6518052-d17d-11e7-9f8e-1be21a784559.png\">\n\nSo, this probably will reduce roughly more than 10 minutes.\n\n## How was this patch tested?\n\nAppVeyor tests\n\nAuthor: hyukjinkwon <gurwls223@gmail.com>\n\nCloses #19816 from HyukjinKwon/SPARK-21693-followup.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/d49d9e40383209eed9584a4ef2c3964f27f4a08f"},{"sha":"5a02e3a2ac8a25d92d98d3b3b0d1173dddb9cc91","author":{"name":"Wenchen Fan","email":"563f5923eae9a92c8a3f34a26b67fa348a303ed1@databricks.com"},"message":"[SPARK-22602][SQL] remove ColumnVector#loadBytes\n\n## What changes were proposed in this pull request?\n\n`ColumnVector#loadBytes` is only used as an optimization for reading UTF8String in `WritableColumnVector`, this PR moves this optimization to `WritableColumnVector` and simplified it.\n\n## How was this patch tested?\n\nexisting test\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #19815 from cloud-fan/load-bytes.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/5a02e3a2ac8a25d92d98d3b3b0d1173dddb9cc91"},{"sha":"2dbe275b2d26035b610ed8385d88e3c9562eaf19","author":{"name":"Kazuaki Ishizaki","email":"409019bab0e07036ad6abfa289bcfe098eb16317@jp.ibm.com"},"message":"[SPARK-22603][SQL] Fix 64KB JVM bytecode limit problem with FormatString\n\n## What changes were proposed in this pull request?\n\nThis PR changes `FormatString` code generation to place generated code for expressions for arguments into separated methods if these size could be large.\nThis PR passes variable arguments by using an `Object` array.\n\n## How was this patch tested?\n\nAdded new test cases into `StringExpressionSuite`\n\nAuthor: Kazuaki Ishizaki <ishizaki@jp.ibm.com>\n\nCloses #19817 from kiszk/SPARK-22603.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/2dbe275b2d26035b610ed8385d88e3c9562eaf19"},{"sha":"1ff4a77be498615ee7216fd9cc2d510ecbd43b27","author":{"name":"Zhenhua Wang","email":"f82454bf38f3ce64ebf8037d45d28db4dce5c9db@huawei.com"},"message":"[SPARK-22529][SQL] Relation stats should be consistent with other plans based on cbo config\n\n## What changes were proposed in this pull request?\n\nCurrently, relation stats is the same whether cbo is enabled or not. While relation (`LogicalRelation` or `HiveTableRelation`) is a `LogicalPlan`, its behavior is inconsistent with other plans. This can cause confusion when user runs EXPLAIN COST commands. Besides, when CBO is disabled, we apply the size-only estimation strategy, so there's no need to propagate other catalog statistics to relation.\n\n## How was this patch tested?\n\nEnhanced existing tests case and added a test case.\n\nAuthor: Zhenhua Wang <wangzhenhua@huawei.com>\n\nCloses #19757 from wzhfy/catalog_stats_conversion.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/1ff4a77be498615ee7216fd9cc2d510ecbd43b27"},{"sha":"087879a77acb37b790c36f8da67355b90719c2dc","author":{"name":"Marco Gaido","email":"035f2fe0944149c518e8b1dfd76528ec1d89d784@hortonworks.com"},"message":"[SPARK-22520][SQL] Support code generation for large CaseWhen\n\n## What changes were proposed in this pull request?\n\nCode generation is disabled for CaseWhen when the number of branches is higher than `spark.sql.codegen.maxCaseBranches` (which defaults to 20). This was done to prevent the well known 64KB method limit exception.\nThis PR proposes to support code generation also in those cases (without causing exceptions of course). As a side effect, we could get rid of the `spark.sql.codegen.maxCaseBranches` configuration.\n\n## How was this patch tested?\n\nexisting UTs\n\nAuthor: Marco Gaido <mgaido@hortonworks.com>\nAuthor: Marco Gaido <marcogaido91@gmail.com>\n\nCloses #19752 from mgaido91/SPARK-22520.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/087879a77acb37b790c36f8da67355b90719c2dc"},{"sha":"33d43bf1b6f55594187066f0e38ba3985fa2542b","author":{"name":"gaborgsomogyi","email":"40e6b04f18fce378745bb3a5827ff2f7df5c5829@gmail.com"},"message":"[SPARK-22484][DOC] Document PySpark DataFrame csv writer behavior whe…\n\n## What changes were proposed in this pull request?\n\nIn PySpark API Document, DataFrame.write.csv() says that setting the quote parameter to an empty string should turn off quoting. Instead, it uses the [null character](https://en.wikipedia.org/wiki/Null_character) as the quote.\n\nThis PR fixes the doc.\n\n## How was this patch tested?\n\nManual.\n\n```\ncd python/docs\nmake html\nopen _build/html/pyspark.sql.html\n```\n\nAuthor: gaborgsomogyi <gabor.g.somogyi@gmail.com>\n\nCloses #19814 from gaborgsomogyi/SPARK-22484.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/33d43bf1b6f55594187066f0e38ba3985fa2542b"},{"sha":"64817c423c0d82a805abd69a3e166e5bfd79c739","author":{"name":"Takuya UESHIN","email":"f070f8f0c50caeda5841963fc7b57e73e59dad56@databricks.com"},"message":"[SPARK-22395][SQL][PYTHON] Fix the behavior of timestamp values for Pandas to respect session timezone\n\n## What changes were proposed in this pull request?\n\nWhen converting Pandas DataFrame/Series from/to Spark DataFrame using `toPandas()` or pandas udfs, timestamp values behave to respect Python system timezone instead of session timezone.\n\nFor example, let's say we use `\"America/Los_Angeles\"` as session timezone and have a timestamp value `\"1970-01-01 00:00:01\"` in the timezone. Btw, I'm in Japan so Python timezone would be `\"Asia/Tokyo\"`.\n\nThe timestamp value from current `toPandas()` will be the following:\n\n```\n>>> spark.conf.set(\"spark.sql.session.timeZone\", \"America/Los_Angeles\")\n>>> df = spark.createDataFrame([28801], \"long\").selectExpr(\"timestamp(value) as ts\")\n>>> df.show()\n+-------------------+\n|                 ts|\n+-------------------+\n|1970-01-01 00:00:01|\n+-------------------+\n\n>>> df.toPandas()\n                   ts\n0 1970-01-01 17:00:01\n```\n\nAs you can see, the value becomes `\"1970-01-01 17:00:01\"` because it respects Python timezone.\nAs we discussed in #18664, we consider this behavior is a bug and the value should be `\"1970-01-01 00:00:01\"`.\n\n## How was this patch tested?\n\nAdded tests and existing tests.\n\nAuthor: Takuya UESHIN <ueshin@databricks.com>\n\nCloses #19607 from ueshin/issues/SPARK-22395.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/64817c423c0d82a805abd69a3e166e5bfd79c739"},{"sha":"1e07fff248088ef4854e430928149ad3809f799f","author":{"name":"Wenchen Fan","email":"563f5923eae9a92c8a3f34a26b67fa348a303ed1@databricks.com"},"message":"[SPARK-22520][SQL][FOLLOWUP] remove outer if for case when codegen\n\n## What changes were proposed in this pull request?\n\na minor cleanup for https://github.com/apache/spark/pull/19752 . Remove the outer if as the code is inside `do while`\n\n## How was this patch tested?\n\nexisting tests\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #19830 from cloud-fan/minor.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/1e07fff248088ef4854e430928149ad3809f799f"},{"sha":"b70e483cb32d07eaab80739cd0cfcd8fe922547c","author":{"name":"Wenchen Fan","email":"563f5923eae9a92c8a3f34a26b67fa348a303ed1@databricks.com"},"message":"[SPARK-22617][SQL] make splitExpressions extract current input of the context\n\n## What changes were proposed in this pull request?\n\nMostly when we call `CodegenContext.splitExpressions`, we want to split the code into methods and pass the current inputs of the codegen context to these methods so that the code in these methods can still be evaluated.\n\nThis PR makes the expectation clear, while still keep the advanced version of `splitExpressions` to customize the inputs to pass to generated methods.\n\n## How was this patch tested?\n\nexisting test\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #19827 from cloud-fan/codegen.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/b70e483cb32d07eaab80739cd0cfcd8fe922547c"},{"sha":"da35574297d96eb750bdfee755a48defc36e284a","author":{"name":"Zhenhua Wang","email":"2c44cc71c11288549d137eb566464c5958003a70@163.com"},"message":"[SPARK-22515][SQL] Estimation relation size based on numRows * rowSize\n\n## What changes were proposed in this pull request?\n\nCurrently, relation size is computed as the sum of file size, which is error-prone because storage format like parquet may have a much smaller file size compared to in-memory size. When we choose broadcast join based on file size, there's a risk of OOM. But if the number of rows is available in statistics, we can get a better estimation by `numRows * rowSize`, which helps to alleviate this problem.\n\n## How was this patch tested?\n\nAdded a new test case for data source table and hive table.\n\nAuthor: Zhenhua Wang <wzh_zju@163.com>\nAuthor: Zhenhua Wang <wangzhenhua@huawei.com>\n\nCloses #19743 from wzhfy/better_leaf_size.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/da35574297d96eb750bdfee755a48defc36e284a"},{"sha":"a10b328dbc056aafaa696579f9a6e2b0cb8eb25f","author":{"name":"Sunitha Kambhampati","email":"ad75ecec1e870ba3fe77d1ae27a83ffb47a52c06@us.ibm.com"},"message":"[SPARK-22431][SQL] Ensure that the datatype in the schema for the table/view metadata is parseable by Spark before persisting it\n\n## What changes were proposed in this pull request?\n* JIRA:  [SPARK-22431](https://issues.apache.org/jira/browse/SPARK-22431)  : Creating Permanent view with illegal type\n\n**Description:**\n- It is possible in Spark SQL to create a permanent view that uses an nested field with an illegal name.\n- For example if we create the following view:\n```create view x as select struct('a' as `$q`, 1 as b) q```\n- A simple select fails with the following exception:\n\n```\nselect * from x;\n\norg.apache.spark.SparkException: Cannot recognize hive type string: struct<$q:string,b:int>\n  at org.apache.spark.sql.hive.client.HiveClientImpl$.fromHiveColumn(HiveClientImpl.scala:812)\n  at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$getTableOption$1$$anonfun$apply$11$$anonfun$7.apply(HiveClientImpl.scala:378)\n  at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$getTableOption$1$$anonfun$apply$11$$anonfun$7.apply(HiveClientImpl.scala:378)\n...\n```\n**Issue/Analysis**: Right now, we can create a view with a schema that cannot be read back by Spark from the Hive metastore.  For more details, please see the discussion about the analysis and proposed fix options in comment 1 and comment 2 in the [SPARK-22431](https://issues.apache.org/jira/browse/SPARK-22431)\n\n**Proposed changes**:\n - Fix the hive table/view codepath to check whether the schema datatype is parseable by Spark before persisting it in the metastore. This change is localized to HiveClientImpl to do the check similar to the check in FromHiveColumn. This is fail-fast and we will avoid the scenario where we write something to the metastore that we are unable to read it back.\n- Added new unit tests\n- Ran the sql related unit test suites ( hive/test, sql/test, catalyst/test) OK\n\nWith the fix:\n```\ncreate view x as select struct('a' as `$q`, 1 as b) q;\n17/11/28 10:44:55 ERROR SparkSQLDriver: Failed in [create view x as select struct('a' as `$q`, 1 as b) q]\norg.apache.spark.SparkException: Cannot recognize hive type string: struct<$q:string,b:int>\n\tat org.apache.spark.sql.hive.client.HiveClientImpl$.org$apache$spark$sql$hive$client$HiveClientImpl$$getSparkSQLDataType(HiveClientImpl.scala:884)\n\tat org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$org$apache$spark$sql$hive$client$HiveClientImpl$$verifyColumnDataType$1.apply(HiveClientImpl.scala:906)\n\tat org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$org$apache$spark$sql$hive$client$HiveClientImpl$$verifyColumnDataType$1.apply(HiveClientImpl.scala:906)\n\tat scala.collection.Iterator$class.foreach(Iterator.scala:893)\n...\n```\n## How was this patch tested?\n- New unit tests have been added.\n\nhvanhovell, Please review and share your thoughts/comments.  Thank you so much.\n\nAuthor: Sunitha Kambhampati <skambha@us.ibm.com>\n\nCloses #19747 from skambha/spark22431.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/a10b328dbc056aafaa696579f9a6e2b0cb8eb25f"},{"sha":"475a29f11ef488e7cb19bf7e0696d9d099d77c92","author":{"name":"Herman van Hovell","email":"679c4ded150c6b9a8052807f108ce836961fed8d@databricks.com"},"message":"[SPARK-22637][SQL] Only refresh a logical plan once.\n\n## What changes were proposed in this pull request?\n`CatalogImpl.refreshTable` uses `foreach(..)` to refresh all tables in a view. This traverses all nodes in the subtree and calls `LogicalPlan.refresh()` on these nodes. However `LogicalPlan.refresh()` is also refreshing its children, as a result refreshing a large view can be quite expensive.\n\nThis PR just calls `LogicalPlan.refresh()` on the top node.\n\n## How was this patch tested?\nExisting tests.\n\nAuthor: Herman van Hovell <hvanhovell@databricks.com>\n\nCloses #19837 from hvanhovell/SPARK-22637.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/475a29f11ef488e7cb19bf7e0696d9d099d77c92"},{"sha":"e9b2070ab2d04993b1c0c1d6c6aba249e6664c8d","author":{"name":"Yinan Li","email":"827d966f2f9efc670bdbbcc2690277ac2fdde07b@gmail.com"},"message":"[SPARK-18278][SCHEDULER] Spark on Kubernetes - Basic Scheduler Backend\n\n## What changes were proposed in this pull request?\n\nThis is a stripped down version of the `KubernetesClusterSchedulerBackend` for Spark with the following components:\n- Static Allocation of Executors\n- Executor Pod Factory\n- Executor Recovery Semantics\n\nIt's step 1 from the step-wise plan documented [here](https://github.com/apache-spark-on-k8s/spark/issues/441#issuecomment-330802935).\nThis addition is covered by the [SPIP vote](http://apache-spark-developers-list.1001551.n3.nabble.com/SPIP-Spark-on-Kubernetes-td22147.html) which passed on Aug 31 .\n\n## How was this patch tested?\n\n- The patch contains unit tests which are passing.\n- Manual testing: `./build/mvn -Pkubernetes clean package` succeeded.\n- It is a **subset** of the entire changelist hosted in http://github.com/apache-spark-on-k8s/spark which is in active use in several organizations.\n- There is integration testing enabled in the fork currently [hosted by PepperData](spark-k8s-jenkins.pepperdata.org:8080) which is being moved over to RiseLAB CI.\n- Detailed documentation on trying out the patch in its entirety is in: https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html\n\ncc rxin felixcheung mateiz (shepherd)\nk8s-big-data SIG members & contributors: mccheah ash211 ssuchter varunkatta kimoonkim erikerlandson liyinan926 tnachen ifilonenko\n\nAuthor: Yinan Li <liyinan926@gmail.com>\nAuthor: foxish <ramanathana@google.com>\nAuthor: mcheah <mcheah@palantir.com>\n\nCloses #19468 from foxish/spark-kubernetes-3.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/e9b2070ab2d04993b1c0c1d6c6aba249e6664c8d"},{"sha":"20b239845b695fe6a893ebfe97b49ef05fae773d","author":{"name":"Wenchen Fan","email":"563f5923eae9a92c8a3f34a26b67fa348a303ed1@databricks.com"},"message":"[SPARK-22605][SQL] SQL write job should also set Spark task output metrics\n\n## What changes were proposed in this pull request?\n\nFor SQL write jobs, we only set metrics for the SQL listener and display them in the SQL plan UI. We should also set metrics for Spark task output metrics, which will be shown in spark job UI.\n\n## How was this patch tested?\n\ntest it manually. For a simple write job\n```\nspark.range(1000).write.parquet(\"/tmp/p1\")\n```\nnow the spark job UI looks like\n![ui](https://user-images.githubusercontent.com/3182036/33326478-05a25b7c-d490-11e7-96ef-806117774356.jpg)\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #19833 from cloud-fan/ui.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/20b239845b695fe6a893ebfe97b49ef05fae773d"},{"sha":"57687280d4171db98d4d9404c7bd3374f51deac0","author":{"name":"Wang Gengliang","email":"59f23ce21e834878dfd6cdde790f2fdea84f2c38@gmail.com"},"message":"[SPARK-22615][SQL] Handle more cases in PropagateEmptyRelation\n\n## What changes were proposed in this pull request?\n\nCurrently, in the optimize rule `PropagateEmptyRelation`, the following cases is not handled:\n1.  empty relation as right child in left outer join\n2. empty relation as left child in right outer join\n3. empty relation as right child  in left semi join\n4. empty relation as right child  in left anti join\n5. only one empty relation in full outer join\n\ncase 1 / 2 / 5 can be treated as **Cartesian product** and cause exception. See the new test cases.\n\n## How was this patch tested?\nUnit test\n\nAuthor: Wang Gengliang <ltnwgl@gmail.com>\n\nCloses #19825 from gengliangwang/SPARK-22615.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/57687280d4171db98d4d9404c7bd3374f51deac0"},{"sha":"284836862b2312aea5d7555c8e3c9d3c4dbc8eaf","author":{"name":"Kazuaki Ishizaki","email":"409019bab0e07036ad6abfa289bcfe098eb16317@jp.ibm.com"},"message":"[SPARK-22608][SQL] add new API to CodeGeneration.splitExpressions()\n\n## What changes were proposed in this pull request?\n\nThis PR adds a new API to ` CodeGenenerator.splitExpression` since since several ` CodeGenenerator.splitExpression` are used with `ctx.INPUT_ROW` to avoid code duplication.\n\n## How was this patch tested?\n\nUsed existing test suits\n\nAuthor: Kazuaki Ishizaki <ishizaki@jp.ibm.com>\n\nCloses #19821 from kiszk/SPARK-22608.","distinct":false,"url":"https://api.github.com/repos/viirya/spark-1/commits/284836862b2312aea5d7555c8e3c9d3c4dbc8eaf"}]}
{"actor":{"display_login":"viirya"}}