{"push_id":1266304867,"size":15,"distinct_size":15,"ref":"refs/heads/branch-2.0","head":"aa57083af4cecb595bac09e437607d7142b54913","before":"eaea1c86b897d302107a9b6833a27a2b24ca31a0","commits":[{"sha":"d16f9a0b7c464728d7b11899740908e23820a797","author":{"name":"Felix Cheung","email":"00eb71019ad7acfd67cff51fce666aad32ce6eed@hotmail.com"},"message":"[SPARKR][MINOR] Update R DESCRIPTION file\n\n## What changes were proposed in this pull request?\n\nUpdate DESCRIPTION\n\n## How was this patch tested?\n\nRun install and CRAN tests\n\nAuthor: Felix Cheung <felixcheung_m@hotmail.com>\n\nCloses #14764 from felixcheung/rpackagedescription.\n\n(cherry picked from commit d2b3d3e63e1a9217de6ef507c350308017664a62)\nSigned-off-by: Xiangrui Meng <meng@databricks.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/d16f9a0b7c464728d7b11899740908e23820a797"},{"sha":"811a2cef03647c5be29fef522c423921c79b1bc3","author":{"name":"Davies Liu","email":"589979bd19a058b75730100a5a88539708000f52@databricks.com"},"message":"[SPARK-13286] [SQL] add the next expression of SQLException as cause\n\nSome JDBC driver (for example PostgreSQL) does not use the underlying exception as cause, but have another APIs (getNextException) to access that, so it it's included in the error logging, making us hard to find the root cause, especially in batch mode.\n\nThis PR will pull out the next exception and add it as cause (if it's different) or suppressed (if there is another different cause).\n\nCan't reproduce this on the default JDBC driver, so did not add a regression test.\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #14722 from davies/keep_cause.\n\n(cherry picked from commit 9afdfc94f49395e69a7959e881c19d787ce00c3e)\nSigned-off-by: Davies Liu <davies.liu@gmail.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/811a2cef03647c5be29fef522c423921c79b1bc3"},{"sha":"cc4018996740b3a68d4a557615c59c67b8996ebb","author":{"name":"Junyang Qian","email":"d5766da94a207df74ee6bb6bd7d2e5654c64d4d7@databricks.com"},"message":"[SPARKR][MINOR] Remove reference link for common Windows environment variables\n\n## What changes were proposed in this pull request?\n\nThe PR removes reference link in the doc for environment variables for common Windows folders. The cran check gave code 503: service unavailable on the original link.\n\n## How was this patch tested?\n\nManual check.\n\nAuthor: Junyang Qian <junyangq@databricks.com>\n\nCloses #14767 from junyangq/SPARKR-RemoveLink.\n\n(cherry picked from commit 8fd63e808e15c8a7e78fef847183c86f332daa91)\nSigned-off-by: Felix Cheung <felixcheung@apache.org>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/cc4018996740b3a68d4a557615c59c67b8996ebb"},{"sha":"a2a7506d06fe9d878d55cf5498f5bfef9a69171c","author":{"name":"hyukjinkwon","email":"4c114bf3676ebcdb1616a11ca478d9181f4025da@gmail.com"},"message":"[MINOR][DOC] Use standard quotes instead of \"curly quote\" marks from Mac in structured streaming programming guides\n\nThis PR fixes curly quotes (`“` and `”` ) to standard quotes (`\"`).\n\nThis will be a actual problem when users copy and paste the examples. This would not work.\n\nThis seems only happening in `structured-streaming-programming-guide.md`.\n\nManually built.\n\nThis will change some examples to be correctly marked down as below:\n\n![2016-08-23 3 24 13](https://cloud.githubusercontent.com/assets/6477701/17882878/2a38332e-694a-11e6-8e84-76bdb89151e0.png)\n\nto\n\n![2016-08-23 3 26 06](https://cloud.githubusercontent.com/assets/6477701/17882888/376eaa28-694a-11e6-8b88-32ea83997037.png)\n\nAuthor: hyukjinkwon <gurwls223@gmail.com>\n\nCloses #14770 from HyukjinKwon/minor-quotes.\n\n(cherry picked from commit 588559911de94bbe0932526ee1e1dd36a581a423)\nSigned-off-by: Sean Owen <sowen@cloudera.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/a2a7506d06fe9d878d55cf5498f5bfef9a69171c"},{"sha":"a772b4b5dea46cda1204a50a4909d40f8933ad77","author":{"name":"Josh Rosen","email":"1fa93d1db1fb7c1c51f2467ef55c960ec1b64919@databricks.com"},"message":"[SPARK-17194] Use single quotes when generating SQL for string literals\n\nWhen Spark emits SQL for a string literal, it should wrap the string in single quotes, not double quotes. Databases which adhere more strictly to the ANSI SQL standards, such as Postgres, allow only single-quotes to be used for denoting string literals (see http://stackoverflow.com/a/1992331/590203).\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #14763 from JoshRosen/SPARK-17194.\n\n(cherry picked from commit bf8ff833e30b39e5e5e35ba8dcac31b79323838c)\nSigned-off-by: Herman van Hovell <hvanhovell@databricks.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/a772b4b5dea46cda1204a50a4909d40f8933ad77"},{"sha":"a6e6a047bb9215df55b009957d4c560624d886fc","author":{"name":"Weiqing Yang","email":"c0c88f1153e01312d8f77a3074d15074c5d139c9@gmail.com"},"message":"[MINOR][SQL] Remove implemented functions from comments of 'HiveSessionCatalog.scala'\n\n## What changes were proposed in this pull request?\nThis PR removes implemented functions from comments of `HiveSessionCatalog.scala`: `java_method`, `posexplode`, `str_to_map`.\n\n## How was this patch tested?\nManual.\n\nAuthor: Weiqing Yang <yangweiqing001@gmail.com>\n\nCloses #14769 from Sherry302/cleanComment.\n\n(cherry picked from commit b9994ad05628077016331e6b411fbc09017b1e63)\nSigned-off-by: Reynold Xin <rxin@databricks.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/a6e6a047bb9215df55b009957d4c560624d886fc"},{"sha":"df87f161c9e40a49235ea722f6a662a488b41c4c","author":{"name":"Wenchen Fan","email":"51df79715bab6f96b68cf2b8813ef1aaa5547bdb@databricks.com"},"message":"[SPARK-17186][SQL] remove catalog table type INDEX\n\n## What changes were proposed in this pull request?\n\nActually Spark SQL doesn't support index, the catalog table type `INDEX` is from Hive. However, most operations in Spark SQL can't handle index table, e.g. create table, alter table, etc.\n\nLogically index table should be invisible to end users, and Hive also generates special table name for index table to avoid users accessing it directly. Hive has special SQL syntax to create/show/drop index tables.\n\nAt Spark SQL side, although we can describe index table directly, but the result is unreadable, we should use the dedicated SQL syntax to do it(e.g. `SHOW INDEX ON tbl`). Spark SQL can also read index table directly, but the result is always empty.(Can hive read index table directly?)\n\nThis PR remove the table type `INDEX`, to make it clear that Spark SQL doesn't support index currently.\n\n## How was this patch tested?\n\nexisting tests.\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nCloses #14752 from cloud-fan/minor2.\n\n(cherry picked from commit 52fa45d62a5a0bc832442f38f9e634c5d8e29e08)\nSigned-off-by: Reynold Xin <rxin@databricks.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/df87f161c9e40a49235ea722f6a662a488b41c4c"},{"sha":"ce7dce1755a8d36ec7346adc3de26d8fdc4f05e9","author":{"name":"Weiqing Yang","email":"c0c88f1153e01312d8f77a3074d15074c5d139c9@gmail.com"},"message":"[MINOR][BUILD] Fix Java CheckStyle Error\n\nAs Spark 2.0.1 will be released soon (mentioned in the spark dev mailing list), besides the critical bugs, it's better to fix the code style errors before the release.\n\nBefore:\n```\n./dev/lint-java\nCheckstyle checks failed at following occurrences:\n[ERROR] src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java:[525] (sizes) LineLength: Line is longer than 100 characters (found 119).\n[ERROR] src/main/java/org/apache/spark/examples/sql/streaming/JavaStructuredNetworkWordCount.java:[64] (sizes) LineLength: Line is longer than 100 characters (found 103).\n```\nAfter:\n```\n./dev/lint-java\nUsing `mvn` from path: /usr/local/bin/mvn\nCheckstyle checks passed.\n```\nManual.\n\nAuthor: Weiqing Yang <yangweiqing001@gmail.com>\n\nCloses #14768 from Sherry302/fixjavastyle.\n\n(cherry picked from commit 673a80d2230602c9e6573a23e35fb0f6b832bfca)\nSigned-off-by: Sean Owen <sowen@cloudera.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/ce7dce1755a8d36ec7346adc3de26d8fdc4f05e9"},{"sha":"33d79b58735770ac613540c21095a1e404f065b0","author":{"name":"VinceShieh","email":"2445c9be3c24c11954fcdbf8bdec5f86733bb70d@intel.com"},"message":"[SPARK-17086][ML] Fix InvalidArgumentException issue in QuantileDiscretizer when some quantiles are duplicated\n\n## What changes were proposed in this pull request?\n\nIn cases when QuantileDiscretizerSuite is called upon a numeric array with duplicated elements,  we will  take the unique elements generated from approxQuantiles as input for Bucketizer.\n\n## How was this patch tested?\n\nAn unit test is added in QuantileDiscretizerSuite\n\nQuantileDiscretizer.fit will throw an illegal exception when calling setSplits on a list of splits\nwith duplicated elements. Bucketizer.setSplits should only accept either a numeric vector of two\nor more unique cut points, although that may produce less number of buckets than requested.\n\nSigned-off-by: VinceShieh <vincent.xieintel.com>\n\nAuthor: VinceShieh <vincent.xie@intel.com>\n\nCloses #14747 from VinceShieh/SPARK-17086.\n\n(cherry picked from commit 92c0eaf348b42b3479610da0be761013f9d81c54)\nSigned-off-by: Sean Owen <sowen@cloudera.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/33d79b58735770ac613540c21095a1e404f065b0"},{"sha":"29091d7cd60c20bf019dc9c1625a22e80ea50928","author":{"name":"Junyang Qian","email":"d5766da94a207df74ee6bb6bd7d2e5654c64d4d7@databricks.com"},"message":"[SPARKR][MINOR] Fix doc for show method\n\n## What changes were proposed in this pull request?\n\nThe original doc of `show` put methods for multiple classes together but the text only talks about `SparkDataFrame`. This PR tries to fix this problem.\n\n## How was this patch tested?\n\nManual test.\n\nAuthor: Junyang Qian <junyangq@databricks.com>\n\nCloses #14776 from junyangq/SPARK-FixShowDoc.\n\n(cherry picked from commit d2932a0e987132c694ed59515b7c77adaad052e6)\nSigned-off-by: Felix Cheung <felixcheung@apache.org>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/29091d7cd60c20bf019dc9c1625a22e80ea50928"},{"sha":"9f924a01b27ebba56080c9ad01b84fff026d5dcd","author":{"name":"Sean Owen","email":"4c59f357315334980589bde1049040bf6de6532c@cloudera.com"},"message":"[SPARK-16781][PYSPARK] java launched by PySpark as gateway may not be the same java used in the spark environment\n\n## What changes were proposed in this pull request?\n\nUpdate to py4j 0.10.3 to enable JAVA_HOME support\n\n## How was this patch tested?\n\nPyspark tests\n\nAuthor: Sean Owen <sowen@cloudera.com>\n\nCloses #14748 from srowen/SPARK-16781.\n\n(cherry picked from commit 0b3a4be92ca6b38eef32ea5ca240d9f91f68aa65)\nSigned-off-by: Sean Owen <sowen@cloudera.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/9f924a01b27ebba56080c9ad01b84fff026d5dcd"},{"sha":"43273377a38a9136ff5e56929630930f076af5af","author":{"name":"Junyang Qian","email":"d5766da94a207df74ee6bb6bd7d2e5654c64d4d7@databricks.com"},"message":"[SPARKR][MINOR] Add more examples to window function docs\n\n## What changes were proposed in this pull request?\n\nThis PR adds more examples to window function docs to make them more accessible to the users.\n\nIt also fixes default value issues for `lag` and `lead`.\n\n## How was this patch tested?\n\nManual test, R unit test.\n\nAuthor: Junyang Qian <junyangq@databricks.com>\n\nCloses #14779 from junyangq/SPARKR-FixWindowFunctionDocs.\n\n(cherry picked from commit 18708f76c366c6e01b5865981666e40d8642ac20)\nSigned-off-by: Felix Cheung <felixcheung@apache.org>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/43273377a38a9136ff5e56929630930f076af5af"},{"sha":"9f363a690102f04a2a486853c1b89134455518bc","author":{"name":"Junyang Qian","email":"d5766da94a207df74ee6bb6bd7d2e5654c64d4d7@databricks.com"},"message":"[SPARKR][MINOR] Add installation message for remote master mode and improve other messages\n\n## What changes were proposed in this pull request?\n\nThis PR gives informative message to users when they try to connect to a remote master but don't have Spark package in their local machine.\n\nAs a clarification, for now, automatic installation will only happen if they start SparkR in R console (rather than from sparkr-shell) and connect to local master. In the remote master mode, local Spark package is still needed, but we will not trigger the install.spark function because the versions have to match those on the cluster, which involves more user input. Instead, we here try to provide detailed message that may help the users.\n\nSome of the other messages have also been slightly changed.\n\n## How was this patch tested?\n\nManual test.\n\nAuthor: Junyang Qian <junyangq@databricks.com>\n\nCloses #14761 from junyangq/SPARK-16579-V1.\n\n(cherry picked from commit 3a60be4b15a5ab9b6e0c4839df99dac7738aa7fe)\nSigned-off-by: Felix Cheung <felixcheung@apache.org>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/9f363a690102f04a2a486853c1b89134455518bc"},{"sha":"3258f27a881dfeb5ab8bae90c338603fa4b6f9d8","author":{"name":"hyukjinkwon","email":"4c114bf3676ebcdb1616a11ca478d9181f4025da@gmail.com"},"message":"[SPARK-16216][SQL][BRANCH-2.0] Backport Read/write dateFormat/timestampFormat options for CSV and JSON\n\n## What changes were proposed in this pull request?\n\nThis PR backports https://github.com/apache/spark/pull/14279 to 2.0.\n\n## How was this patch tested?\n\nUnit tests were added in `CSVSuite` and `JsonSuite`. For JSON, existing tests cover the default cases.\n\nAuthor: hyukjinkwon <gurwls223@gmail.com>\n\nCloses #14799 from HyukjinKwon/SPARK-16216-json-csv-backport.","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/3258f27a881dfeb5ab8bae90c338603fa4b6f9d8"},{"sha":"aa57083af4cecb595bac09e437607d7142b54913","author":{"name":"Sameer Agarwal","email":"a70383dff9f522db5c95b6d04d9bfe45099f5e1d@cs.berkeley.edu"},"message":"[SPARK-17228][SQL] Not infer/propagate non-deterministic constraints\n\n## What changes were proposed in this pull request?\n\nGiven that filters based on non-deterministic constraints shouldn't be pushed down in the query plan, unnecessarily inferring them is confusing and a source of potential bugs. This patch simplifies the inferring logic by simply ignoring them.\n\n## How was this patch tested?\n\nAdded a new test in `ConstraintPropagationSuite`.\n\nAuthor: Sameer Agarwal <sameerag@cs.berkeley.edu>\n\nCloses #14795 from sameeragarwal/deterministic-constraints.\n\n(cherry picked from commit ac27557eb622a257abeb3e8551f06ebc72f87133)\nSigned-off-by: Reynold Xin <rxin@databricks.com>","distinct":true,"url":"https://api.github.com/repos/jasonmoore2k/spark/commits/aa57083af4cecb595bac09e437607d7142b54913"}]}
{"actor":{"display_login":"jasonmoore2k"}}