{"push_id":1150523410,"size":11,"distinct_size":6,"ref":"refs/heads/SPARK-15214","head":"1d2d595c0f5ac52fc896c966c3258f67ec07aabc","before":"60da24eef70c2b60a98140ebd9555cc8b3179b02","commits":[{"sha":"890baaca5078df0b50c0054f55a2c33023f7fd67","author":{"name":"Sean Zhong","email":"5320fbfa67c9b6c0e01431b7f024db2a7ff8bdaf@databricks.com"},"message":"[SPARK-15674][SQL] Deprecates \"CREATE TEMPORARY TABLE USING...\", uses \"CREAT TEMPORARY VIEW USING...\" instead\n\n## What changes were proposed in this pull request?\n\nThe current implementation of \"CREATE TEMPORARY TABLE USING datasource...\" is NOT creating any intermediate temporary data directory like temporary HDFS folder, instead, it only stores a SQL string in memory. Probably we should use \"TEMPORARY VIEW\" instead.\n\nThis PR assumes a temporary table has to link with some temporary intermediate data. It follows the definition of temporary table like this (from [hortonworks doc](https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_dataintegration/content/temp-tables.html)):\n> A temporary table is a convenient way for an application to automatically manage intermediate data generated during a complex query\n\n**Example**:\n\n```\nscala> spark.sql(\"CREATE temporary view  my_tab7 (c1: String, c2: String)  USING org.apache.spark.sql.execution.datasources.csv.CSVFileFormat  OPTIONS (PATH '/Users/seanzhong/csv/cars.csv')\")\nscala> spark.sql(\"select c1, c2 from my_tab7\").show()\n+----+-----+\n|  c1|   c2|\n+----+-----+\n|year| make|\n|2012|Tesla|\n...\n```\n\nIt NOW prints a **deprecation warning** if \"CREATE TEMPORARY TABLE USING...\" is used.\n\n```\nscala> spark.sql(\"CREATE temporary table  my_tab7 (c1: String, c2: String)  USING org.apache.spark.sql.execution.datasources.csv.CSVFileFormat  OPTIONS (PATH '/Users/seanzhong/csv/cars.csv')\")\n16/05/31 10:39:27 WARN SparkStrategies$DDLStrategy: CREATE TEMPORARY TABLE tableName USING... is deprecated, please use CREATE TEMPORARY VIEW viewName USING... instead\n```\n\n## How was this patch tested?\n\nUnit test.\n\nAuthor: Sean Zhong <seanzhong@databricks.com>\n\nCloses #13414 from clockfly/create_temp_view_using.","distinct":false,"url":"https://api.github.com/repos/hvanhovell/spark/commits/890baaca5078df0b50c0054f55a2c33023f7fd67"},{"sha":"6ecedf39b44c9acd58cdddf1a31cf11e8e24428c","author":{"name":"Yanbo Liang","email":"b6929798b4e4fba75014af8265904a7818902ec7@gmail.com"},"message":"[SPARK-13590][ML][DOC] Document spark.ml LiR, LoR and AFTSurvivalRegression behavior difference\n\n## What changes were proposed in this pull request?\nWhen fitting ```LinearRegressionModel```(by \"l-bfgs\" solver) and ```LogisticRegressionModel``` w/o intercept on dataset with constant nonzero column, spark.ml produce same model as R glmnet but different from LIBSVM.\n\nWhen fitting ```AFTSurvivalRegressionModel``` w/o intercept on dataset with constant nonzero column, spark.ml produce different model compared with R survival::survreg.\n\nWe should output a warning message and clarify in document for this condition.\n\n## How was this patch tested?\nDocument change, no unit test.\n\ncc mengxr\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #12731 from yanboliang/spark-13590.","distinct":false,"url":"https://api.github.com/repos/hvanhovell/spark/commits/6ecedf39b44c9acd58cdddf1a31cf11e8e24428c"},{"sha":"695dbc816a6d70289abeb145cb62ff4e62b3f49b","author":{"name":"zhonghaihua","email":"1b2a5aba34069dbc1947257902c49ae587ad2bf0@qq.com"},"message":"[SPARK-14485][CORE] ignore task finished for executor lost and removed by driver\n\nNow, when executor is removed by driver with heartbeats timeout, driver will re-queue the task on this executor and send a kill command to cluster to kill this executor.\nBut, in a situation, the running task of this executor is finished and return result to driver before this executor killed by kill command sent by driver. At this situation, driver will accept the task finished event and ignore speculative task and re-queued task.\nBut, as we know, this executor has removed by driver, the result of this finished task can not save in driver because the BlockManagerId has also removed from BlockManagerMaster by driver. So, the result data of this stage is not complete, and then, it will cause fetch failure. For more details, [link to jira issues SPARK-14485](https://issues.apache.org/jira/browse/SPARK-14485)\nThis PR introduce a mechanism to ignore this kind of task finished.\n\nN/A\n\nAuthor: zhonghaihua <793507405@qq.com>\n\nCloses #12258 from zhonghaihua/ignoreTaskFinishForExecutorLostAndRemovedByDriver.","distinct":false,"url":"https://api.github.com/repos/hvanhovell/spark/commits/695dbc816a6d70289abeb145cb62ff4e62b3f49b"},{"sha":"0cfd6192f38932a26195a6a8dbbc637d67f5ec55","author":{"name":"Shixiong Zhu","email":"eb2f6da57ec54996cebae8c2a82b9fbfd527e14c@databricks.com"},"message":"[SPARK-15580][SQL] Add ContinuousQueryInfo to make ContinuousQueryListener events serializable\n\n## What changes were proposed in this pull request?\n\nThis PR adds ContinuousQueryInfo to make ContinuousQueryListener events serializable in order to support writing events into the event log.\n\n## How was this patch tested?\n\nJenkins unit tests.\n\nAuthor: Shixiong Zhu <shixiong@databricks.com>\n\nCloses #13335 from zsxwing/query-info.","distinct":false,"url":"https://api.github.com/repos/hvanhovell/spark/commits/0cfd6192f38932a26195a6a8dbbc637d67f5ec55"},{"sha":"91fbc880b69bddcf5310afecc49df1102408e1f3","author":{"name":"Herman van Hovell","email":"38f32192f78c1df35671ebb666a8ae224cdccefb@databricks.com"},"message":"[SPARK-15789][SQL] Allow reserved keywords in most places\n\n## What changes were proposed in this pull request?\nThe parser currently does not allow the use of some SQL keywords as table or field names. This PR adds supports for all keywords as identifier. The exception to this are table aliases, in this case most keywords are allowed except for join keywords (```anti, full, inner, left, semi, right, natural, on, join, cross```) and set-operator keywords (```union, intersect, except```).\n\n## How was this patch tested?\nI have added/move/renamed test in the catalyst `*ParserSuite`s.\n\nAuthor: Herman van Hovell <hvanhovell@databricks.com>\n\nCloses #13534 from hvanhovell/SPARK-15789.","distinct":false,"url":"https://api.github.com/repos/hvanhovell/spark/commits/91fbc880b69bddcf5310afecc49df1102408e1f3"},{"sha":"87706eb66cd1370862a1f8ea447484c80969e45f","author":{"name":"yinxusen","email":"b3af76889e6bb255e66b77722a2ef0a4409760be@gmail.com"},"message":"[SPARK-15793][ML] Add maxSentenceLength for ml.Word2Vec\n\n## What changes were proposed in this pull request?\n\nhttps://issues.apache.org/jira/browse/SPARK-15793\n\nWord2vec in ML package should have maxSentenceLength method for feature parity.\n\n## How was this patch tested?\n\nTested with Spark unit test.\n\nAuthor: yinxusen <yinxusen@gmail.com>\n\nCloses #13536 from yinxusen/SPARK-15793.","distinct":true,"url":"https://api.github.com/repos/hvanhovell/spark/commits/87706eb66cd1370862a1f8ea447484c80969e45f"},{"sha":"f958c1c3e292aba98d283637606890f353a9836c","author":{"name":"Sandeep Singh","email":"69043e619d79bcf4ae068e5b131d66c33e73727d@techaddict.me"},"message":"[MINOR] Fix Java Lint errors introduced by #13286 and #13280\n\n## What changes were proposed in this pull request?\n\nrevived #13464\n\nFix Java Lint errors introduced by #13286 and #13280\nBefore:\n```\nUsing `mvn` from path: /Users/pichu/Project/spark/build/apache-maven-3.3.9/bin/mvn\nJava HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0\nCheckstyle checks failed at following occurrences:\n[ERROR] src/main/java/org/apache/spark/launcher/LauncherServer.java:[340,5] (whitespace) FileTabCharacter: Line contains a tab character.\n[ERROR] src/main/java/org/apache/spark/launcher/LauncherServer.java:[341,5] (whitespace) FileTabCharacter: Line contains a tab character.\n[ERROR] src/main/java/org/apache/spark/launcher/LauncherServer.java:[342,5] (whitespace) FileTabCharacter: Line contains a tab character.\n[ERROR] src/main/java/org/apache/spark/launcher/LauncherServer.java:[343,5] (whitespace) FileTabCharacter: Line contains a tab character.\n[ERROR] src/main/java/org/apache/spark/sql/streaming/OutputMode.java:[41,28] (naming) MethodName: Method name 'Append' must match pattern '^[a-z][a-z0-9][a-zA-Z0-9_]*$'.\n[ERROR] src/main/java/org/apache/spark/sql/streaming/OutputMode.java:[52,28] (naming) MethodName: Method name 'Complete' must match pattern '^[a-z][a-z0-9][a-zA-Z0-9_]*$'.\n[ERROR] src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java:[61,8] (imports) UnusedImports: Unused import - org.apache.parquet.schema.PrimitiveType.\n[ERROR] src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java:[62,8] (imports) UnusedImports: Unused import - org.apache.parquet.schema.Type.\n```\n\n## How was this patch tested?\nran `dev/lint-java` locally\n\nAuthor: Sandeep Singh <sandeep@techaddict.me>\n\nCloses #13559 from techaddict/minor-3.","distinct":true,"url":"https://api.github.com/repos/hvanhovell/spark/commits/f958c1c3e292aba98d283637606890f353a9836c"},{"sha":"ca70ab27cc73f6ea7fce5d179ca8f13459c8ba95","author":{"name":"prabs","email":"ea4ef3a8fe863f215f746899f6485bb2de91a666@gmail.com"},"message":"[DOCUMENTATION] Fixed target JAR path\n\n## What changes were proposed in this pull request?\n\nMentioned Scala version in the sbt configuration file is 2.11, so the path of the target JAR should be `/target/scala-2.11/simple-project_2.11-1.0.jar`\n\n## How was this patch tested?\n\nn/a\n\nAuthor: prabs <prabsmails@gmail.com>\nAuthor: Prabeesh K <prabsmails@gmail.com>\n\nCloses #13554 from prabeesh/master.","distinct":true,"url":"https://api.github.com/repos/hvanhovell/spark/commits/ca70ab27cc73f6ea7fce5d179ca8f13459c8ba95"},{"sha":"4e8ac6edd5808ca8245b39d804c6d4f5ea9d0d36","author":{"name":"Eric Liang","email":"8825d93f7d4ea34a2c506e18a7ac7514d6509e7d@databricks.com"},"message":"[SPARK-15735] Allow specifying min time to run in microbenchmarks\n\n## What changes were proposed in this pull request?\n\nThis makes microbenchmarks run for at least 2 seconds by default, to allow some time for jit compilation to kick in.\n\n## How was this patch tested?\n\nTested manually with existing microbenchmarks. This change is backwards compatible in that existing microbenchmarks which specified numIters per-case will still run exactly that number of iterations. Microbenchmarks which previously overrode defaultNumIters now override minNumIters.\n\ncc hvanhovell\n\nAuthor: Eric Liang <ekl@databricks.com>\nAuthor: Eric Liang <ekhliang@gmail.com>\n\nCloses #13472 from ericl/spark-15735.","distinct":true,"url":"https://api.github.com/repos/hvanhovell/spark/commits/4e8ac6edd5808ca8245b39d804c6d4f5ea9d0d36"},{"sha":"87688b122a985d4fac47aed54bb71e0c9b2c8cbd","author":{"name":"Herman van Hovell","email":"38f32192f78c1df35671ebb666a8ae224cdccefb@databricks.com"},"message":"Merge remote-tracking branch 'apache-github/master' into SPARK-15214","distinct":true,"url":"https://api.github.com/repos/hvanhovell/spark/commits/87688b122a985d4fac47aed54bb71e0c9b2c8cbd"},{"sha":"1d2d595c0f5ac52fc896c966c3258f67ec07aabc","author":{"name":"Herman van Hovell","email":"38f32192f78c1df35671ebb666a8ae224cdccefb@databricks.com"},"message":"Add benchmarks for explode map & json_tuple","distinct":true,"url":"https://api.github.com/repos/hvanhovell/spark/commits/1d2d595c0f5ac52fc896c966c3258f67ec07aabc"}]}
{"actor":{"display_login":"hvanhovell"}}