{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/10974","labels_url":"https://api.github.com/repos/apache/spark/issues/10974/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/10974/comments","events_url":"https://api.github.com/repos/apache/spark/issues/10974/events","html_url":"https://github.com/apache/spark/pull/10974","id":129665997,"number":10974,"title":"[SPARK-13072][SQL] simplify and improve murmur3 hash expression codegen","user":{"login":"cloud-fan","id":3182036,"avatar_url":"https://avatars.githubusercontent.com/u/3182036?v=3","gravatar_id":"","url":"https://api.github.com/users/cloud-fan","html_url":"https://github.com/cloud-fan","followers_url":"https://api.github.com/users/cloud-fan/followers","following_url":"https://api.github.com/users/cloud-fan/following{/other_user}","gists_url":"https://api.github.com/users/cloud-fan/gists{/gist_id}","starred_url":"https://api.github.com/users/cloud-fan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cloud-fan/subscriptions","organizations_url":"https://api.github.com/users/cloud-fan/orgs","repos_url":"https://api.github.com/users/cloud-fan/repos","events_url":"https://api.github.com/users/cloud-fan/events{/privacy}","received_events_url":"https://api.github.com/users/cloud-fan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-01-29T04:52:17Z","updated_at":"2016-01-29T06:19:37Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/10974","html_url":"https://github.com/apache/spark/pull/10974","diff_url":"https://github.com/apache/spark/pull/10974.diff","patch_url":"https://github.com/apache/spark/pull/10974.patch"},"body":"simplify(remove several unnecessary local variables) the generated code of hash expression, and avoid null check if possible.\r\n\r\ngenerated code comparison for `hash(int, double, string, array<string>)`:\r\n**before:**\r\n```\r\n  public UnsafeRow apply(InternalRow i) {\r\n    /* hash(input[0, int],input[1, double],input[2, string],input[3, array<int>],42) */\r\n    int value1 = 42;\r\n    /* input[0, int] */\r\n    int value3 = i.getInt(0);\r\n    if (!false) {\r\n      value1 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(value3, value1);\r\n    }\r\n    /* input[1, double] */\r\n    double value5 = i.getDouble(1);\r\n    if (!false) {\r\n      value1 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashLong(Double.doubleToLongBits(value5), value1);\r\n    }\r\n    /* input[2, string] */\r\n    boolean isNull6 = i.isNullAt(2);\r\n    UTF8String value7 = isNull6 ? null : (i.getUTF8String(2));\r\n    if (!isNull6) {\r\n      value1 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashUnsafeBytes(value7.getBaseObject(), value7.getBaseOffset(), value7.numBytes(), value1);\r\n    }\r\n    /* input[3, array<int>] */\r\n    boolean isNull8 = i.isNullAt(3);\r\n    ArrayData value9 = isNull8 ? null : (i.getArray(3));\r\n    if (!isNull8) {\r\n      int result10 = value1;\r\n      for (int index11 = 0; index11 < value9.numElements(); index11++) {\r\n        if (!value9.isNullAt(index11)) {\r\n          final int element12 = value9.getInt(index11);\r\n          result10 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(element12, result10);\r\n        }\r\n      }\r\n      value1 = result10;\r\n    }\r\n  }\r\n```\r\n**after:**\r\n```\r\n  public UnsafeRow apply(InternalRow i) {\r\n    /* hash(input[0, int],input[1, double],input[2, string],input[3, array<int>],42) */\r\n    int value1 = 42;\r\n    /* input[0, int] */\r\n    int value3 = i.getInt(0);\r\n    value1 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(value3, value1);\r\n    /* input[1, double] */\r\n    double value5 = i.getDouble(1);\r\n    value1 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashLong(Double.doubleToLongBits(value5), value1);\r\n    /* input[2, string] */\r\n    boolean isNull6 = i.isNullAt(2);\r\n    UTF8String value7 = isNull6 ? null : (i.getUTF8String(2));\r\n\r\n    if (!isNull6) {\r\n      value1 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashUnsafeBytes(value7.getBaseObject(), value7.getBaseOffset(), value7.numBytes(), value1);\r\n    }\r\n\r\n    /* input[3, array<int>] */\r\n    boolean isNull8 = i.isNullAt(3);\r\n    ArrayData value9 = isNull8 ? null : (i.getArray(3));\r\n    if (!isNull8) {\r\n      for (int index10 = 0; index10 < value9.numElements(); index10++) {\r\n        final int element11 = value9.getInt(index10);\r\n        value1 = org.apache.spark.unsafe.hash.Murmur3_x86_32.hashInt(element11, value1);\r\n      }\r\n    }\r\n\r\n    rowWriter14.write(0, value1);\r\n    return result12;\r\n  }\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/176598205","html_url":"https://github.com/apache/spark/pull/10974#issuecomment-176598205","issue_url":"https://api.github.com/repos/apache/spark/issues/10974","id":176598205,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-01-29T06:19:37Z","updated_at":"2016-01-29T06:19:37Z","body":"**[Test build #50341 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50341/consoleFull)** for PR 10974 at commit [`f6a3ccc`](https://github.com/apache/spark/commit/f6a3cccab507c66f29f12132c9656bb409263a13).\n * This patch passes all tests.\n * This patch merges cleanly.\n * This patch adds no public classes."}}