{"push_id":947011602,"size":165,"distinct_size":28,"ref":"refs/heads/IntersectBySemiJoin","head":"6a7979d35d5cf89d9cd14eb6d6a2a4418a44a669","before":"4372170f600eb25996c3aa4f09d569312c263686","commits":[{"sha":"1fdf9bbd67b884f23150b651f0fefdab6ccf008a","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-12730][TESTS] De-duplicate some test code in BlockManagerSuite\n\nThis patch deduplicates some test code in BlockManagerSuite. I'm splitting this change off from a larger PR in order to make things easier to review.\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #10667 from JoshRosen/block-mgr-tests-cleanup.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/1fdf9bbd67b884f23150b651f0fefdab6ccf008a"},{"sha":"090d691323063c436601943506baac3ec5255dd9","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-4628][BUILD] Remove all non-Maven-Central repositories from build\n\nThis patch removes all non-Maven-central repositories from Spark's build, thereby avoiding any risk of future build-breaks due to us accidentally depending on an artifact which is not present in an immutable public Maven repository.\n\nI tested this by running\n\n```\nbuild/mvn \\\n        -Phive \\\n        -Phive-thriftserver \\\n        -Pkinesis-asl \\\n        -Pspark-ganglia-lgpl \\\n        -Pyarn \\\n        dependency:go-offline\n```\n\ninside of a fresh Ubuntu Docker container with no Ivy or Maven caches (I did a similar test for SBT).\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #10659 from JoshRosen/SPARK-4628.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/090d691323063c436601943506baac3ec5255dd9"},{"sha":"1debdfa8d14a6330ab46ed4b62c71e4fb35ca286","author":{"email":"905db6c57c9ba2de5402239d136139da9b5d0058@gmail.com","name":"gatorsmile"},"message":"Merge remote-tracking branch 'upstream/master'","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/1debdfa8d14a6330ab46ed4b62c71e4fb35ca286"},{"sha":"95cd5d95ce8aec8b2462204c791ba927326305ba","author":{"email":"d24118b4297e1ecf0c990d6935f5f9e6e4fd7701@gmail.com","name":"Liang-Chi Hsieh"},"message":"[SPARK-12577] [SQL] Better support of parentheses in partition by and order by clause of window function's over clause\n\nJIRA: https://issues.apache.org/jira/browse/SPARK-12577\n\nAuthor: Liang-Chi Hsieh <viirya@gmail.com>\n\nCloses #10620 from viirya/fix-parentheses.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/95cd5d95ce8aec8b2462204c791ba927326305ba"},{"sha":"3d77cffec093bed4d330969f1a996f3358b9a772","author":{"email":"452ae006f36be6c70342f33f64bb6f6eb552689c@gmail.com","name":"Yanbo Liang"},"message":"[SPARK-12645][SPARKR] SparkR support hash function\n\nAdd ```hash``` function for SparkR ```DataFrame```.\n\nAuthor: Yanbo Liang <ybliang8@gmail.com>\n\nCloses #10597 from yanboliang/spark-12645.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/3d77cffec093bed4d330969f1a996f3358b9a772"},{"sha":"b23c4521f5df905e4fe4d79dd5b670286e2697f7","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-12340] Fix overflow in various take functions.\n\nThis is a follow-up for the original patch #10562.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #10670 from rxin/SPARK-12340.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/b23c4521f5df905e4fe4d79dd5b670286e2697f7"},{"sha":"3efd106e5cc1312bfba693a694ed33a3609a6741","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"Close #10665","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/3efd106e5cc1312bfba693a694ed33a3609a6741"},{"sha":"5b0d544339ef02fc25c816b6d6841031ef3902c2","author":{"email":"c9d35a384bc579d1c6c8c055f2a2b5c04e11b521@databricks.com","name":"Reynold Xin"},"message":"[SPARK-12735] Consolidate & move spark-ec2 to AMPLab managed repository.\n\nAuthor: Reynold Xin <rxin@databricks.com>\n\nCloses #10673 from rxin/SPARK-12735.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/5b0d544339ef02fc25c816b6d6841031ef3902c2"},{"sha":"b78e028e37193a4e27b012f0b3c8343d850c5674","author":{"email":"9749d1492af3d43f9c09e04c5c43f27bb909af51@japila.pl","name":"Jacek Laskowski"},"message":"[SPARK-12736][CORE][DEPLOY] Standalone Master cannot be started due t…\n\n…o NoClassDefFoundError: org/spark-project/guava/collect/Maps\n\n/cc srowen rxin\n\nAuthor: Jacek Laskowski <jacek@japila.pl>\n\nCloses #10674 from jaceklaskowski/SPARK-12736.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/b78e028e37193a4e27b012f0b3c8343d850c5674"},{"sha":"e5904bb5e7d83b3731b312c40f7904c0511019f5","author":{"email":"7d45abe39fd242ee2f6575da28f13d66568418b3@oss.nttdata.co.jp","name":"Kousuke Saruta"},"message":"[SPARK-12692][BUILD][MLLIB] Scala style: Fix the style violation (Space before \",\" or \":\")\n\nFix the style violation (space before , and :).\nThis PR is a followup for #10643.\n\nAuthor: Kousuke Saruta <sarutak@oss.nttdata.co.jp>\n\nCloses #10684 from sarutak/SPARK-12692-followup-mllib.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/e5904bb5e7d83b3731b312c40f7904c0511019f5"},{"sha":"3119206b7188c23055621dfeaf6874f21c711a82","author":{"email":"7d45abe39fd242ee2f6575da28f13d66568418b3@oss.nttdata.co.jp","name":"Kousuke Saruta"},"message":"[SPARK-12692][BUILD][GRAPHX] Scala style: Fix the style violation (Space before \",\" or \":\")\n\nFix the style violation (space before `,` and `:`).\nThis PR is a followup for #10643.\n\nAuthor: Kousuke Saruta <sarutak@oss.nttdata.co.jp>\n\nCloses #10683 from sarutak/SPARK-12692-followup-graphx.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/3119206b7188c23055621dfeaf6874f21c711a82"},{"sha":"3ab0138b0fe0f9208b4b476855294a7c729583b7","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-12734][BUILD] Fix Netty exclusion and use Maven Enforcer to prevent future bugs\n\nNetty classes are published under multiple artifacts with different names, so our build needs to exclude the `io.netty:netty` and `org.jboss.netty:netty` versions of the Netty artifact. However, our existing exclusions were incomplete, leading to situations where duplicate Netty classes would wind up on the classpath and cause compile errors (or worse).\n\nThis patch fixes the exclusion issue by adding more exclusions and uses Maven Enforcer's [banned dependencies](https://maven.apache.org/enforcer/enforcer-rules/bannedDependencies.html) rule to prevent these classes from accidentally being reintroduced. I also updated `dev/test-dependencies.sh` to run `mvn validate` so that the enforcer rules can run as part of pull request builds.\n\n/cc rxin srowen pwendell. I'd like to backport at least the exclusion portion of this fix to `branch-1.5` in order to fix the documentation publishing job, which fails nondeterministically due to incompatible versions of Netty classes taking precedence on the compile-time classpath.\n\nAuthor: Josh Rosen <rosenville@gmail.com>\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #10672 from JoshRosen/enforce-netty-exclusions.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/3ab0138b0fe0f9208b4b476855294a7c729583b7"},{"sha":"6439a82503e900ae2e5c3cda5d10ac20dfd6e53f","author":{"email":"8283fd24ce58e0bb0d6bf349e9064fef42c4d84e@cloudera.com","name":"Marcelo Vanzin"},"message":"[SPARK-3873][BUILD] Enable import ordering error checking.\n\nTurn import ordering violations into build errors, plus a few adjustments\nto account for how the checker behaves. I'm a little on the fence about\nwhether the existing code is right, but it's easier to appease the checker\nthan to discuss what's the more correct order here.\n\nPlus a few fixes to imports that cropped in since my recent cleanups.\n\nAuthor: Marcelo Vanzin <vanzin@cloudera.com>\n\nCloses #10612 from vanzin/SPARK-3873-enable.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/6439a82503e900ae2e5c3cda5d10ac20dfd6e53f"},{"sha":"008a55828512056313da2626fd378e8aa1534790","author":{"email":"7d45abe39fd242ee2f6575da28f13d66568418b3@oss.nttdata.co.jp","name":"Kousuke Saruta"},"message":"[SPARK-4628][BUILD] Add a resolver to MiMaBuild.scala for mqttv3(1.0.1).\n\n#10659 removed the repository `https://repo.eclipse.org/content/repositories/paho-releases` but it's needed by MiMa because `spark-streaming-mqtt(1.6.0)` depends on `mqttv3(1.0.1)` and it is provided by the removed repository and maven-central provide only `mqttv3(1.0.2)` for now.\nOtherwise, if `mqttv3(1.0.1)` is absent from the local repository, dev/mima should fail.\n\nJoshRosen Do you have any other better idea?\n\nAuthor: Kousuke Saruta <sarutak@oss.nttdata.co.jp>\n\nCloses #10688 from sarutak/SPARK-4628-followup.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/008a55828512056313da2626fd378e8aa1534790"},{"sha":"f13c7f8f7dc8766b0a42406b5c3639d6be55cf33","author":{"email":"e5dd8a3899598b2e85930c5ff06b11c053ef654f@databricks.com","name":"Josh Rosen"},"message":"[SPARK-12734][HOTFIX][TEST-MAVEN] Fix bug in Netty exclusions\n\nThis is a hotfix for a build bug introduced by the Netty exclusion changes in #10672. We can't exclude `io.netty:netty` because Akka depends on it. There's not a direct conflict between `io.netty:netty` and `io.netty:netty-all`, because the former puts classes in the `org.jboss.netty` namespace while the latter uses the `io.netty` namespace. However, there still is a conflict between `org.jboss.netty:netty` and `io.netty:netty`, so we need to continue to exclude the JBoss version of that artifact.\n\nWhile the diff here looks somewhat large, note that this is only a revert of a some of the changes from #10672. You can see the net changes in pom.xml at https://github.com/apache/spark/compare/3119206b7188c23055621dfeaf6874f21c711a82...5211ab8#diff-600376dffeb79835ede4a0b285078036\n\nAuthor: Josh Rosen <joshrosen@databricks.com>\n\nCloses #10693 from JoshRosen/netty-hotfix.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/f13c7f8f7dc8766b0a42406b5c3639d6be55cf33"},{"sha":"f253feff62f3eb3cce22bbec0874f317a61b0092","author":{"email":"563f5923eae9a92c8a3f34a26b67fa348a303ed1@databricks.com","name":"Wenchen Fan"},"message":"[SPARK-12539][FOLLOW-UP] always sort in partitioning writer\n\naddress comments in #10498 , especially https://github.com/apache/spark/pull/10498#discussion_r49021259\n\nAuthor: Wenchen Fan <wenchen@databricks.com>\n\nThis patch had conflicts when merged, resolved by\nCommitter: Reynold Xin <rxin@databricks.com>\n\nCloses #10638 from cloud-fan/bucket-write.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/f253feff62f3eb3cce22bbec0874f317a61b0092"},{"sha":"bd723bd53d9a28239b60939a248a4ea13340aad8","author":{"email":"46f1a0bd5592a2f9244ca321b129902a06b53e03@blinkenlight.net","name":"Udo Klein"},"message":"removed lambda from sortByKey()\n\nAccording to the documentation the sortByKey method does not take a lambda as an argument, thus the example is flawed. Removed the argument completely as this will default to ascending sort.\n\nAuthor: Udo Klein <git@blinkenlight.net>\n\nCloses #10640 from udoklein/patch-1.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/bd723bd53d9a28239b60939a248a4ea13340aad8"},{"sha":"8fe928b4fe380ba527164bd413402abfed13c0e1","author":{"email":"760e7dab2836853c63805033e514668301fa9c47@seatgeek.com","name":"BrianLondon"},"message":"[SPARK-12269][STREAMING][KINESIS] Update aws-java-sdk version\n\nThe current Spark Streaming kinesis connector references a quite old version 1.9.40 of the AWS Java SDK (1.10.40 is current). Numerous AWS features including Kinesis Firehose are unavailable in 1.9. Those two versions of the AWS SDK in turn require conflicting versions of Jackson (2.4.4 and 2.5.3 respectively) such that one cannot include the current AWS SDK in a project that also uses the Spark Streaming Kinesis ASL.\n\nAuthor: BrianLondon <brian@seatgeek.com>\n\nCloses #10256 from BrianLondon/master.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/8fe928b4fe380ba527164bd413402abfed13c0e1"},{"sha":"9559ac5f74434cf4bf611bdcde9a216d39799826","author":{"email":"ee40c38804112ee33047651f87c281141386e6ad@gmail.com","name":"Anatoliy Plastinin"},"message":"[SPARK-12744][SQL] Change parsing JSON integers to timestamps to treat integers as number of seconds\n\nJIRA: https://issues.apache.org/jira/browse/SPARK-12744\n\nThis PR makes parsing JSON integers to timestamps consistent with casting behavior.\n\nAuthor: Anatoliy Plastinin <anatoliy.plastinin@gmail.com>\n\nCloses #10687 from antlypls/fix-json-timestamp-parsing.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/9559ac5f74434cf4bf611bdcde9a216d39799826"},{"sha":"b313badaa049f847f33663c61cd70ee2f2cbebac","author":{"email":"9749d1492af3d43f9c09e04c5c43f27bb909af51@japila.pl","name":"Jacek Laskowski"},"message":"[STREAMING][MINOR] Typo fixes\n\nAuthor: Jacek Laskowski <jacek@japila.pl>\n\nCloses #10698 from jaceklaskowski/streaming-kafka-typo-fixes.","distinct":false,"url":"https://api.github.com/repos/gatorsmile/spark/commits/b313badaa049f847f33663c61cd70ee2f2cbebac"}]}