{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/10917","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/10917/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/10917/comments","events_url":"https://api.github.com/repos/apache/spark/issues/10917/events","html_url":"https://github.com/apache/spark/pull/10917","id":128732849,"number":10917,"title":"[SPARK-12888][SQL][follow-up] benchmark the new hash expression","user":{"login":"cloud-fan","id":3182036,"avatar_url":"https://avatars.githubusercontent.com/u/3182036?v=3","gravatar_id":"","url":"https://api.github.com/users/cloud-fan","html_url":"https://github.com/cloud-fan","followers_url":"https://api.github.com/users/cloud-fan/followers","following_url":"https://api.github.com/users/cloud-fan/following{/other_user}","gists_url":"https://api.github.com/users/cloud-fan/gists{/gist_id}","starred_url":"https://api.github.com/users/cloud-fan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cloud-fan/subscriptions","organizations_url":"https://api.github.com/users/cloud-fan/orgs","repos_url":"https://api.github.com/users/cloud-fan/repos","events_url":"https://api.github.com/users/cloud-fan/events{/privacy}","received_events_url":"https://api.github.com/users/cloud-fan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":14,"created_at":"2016-01-26T05:48:16Z","updated_at":"2016-02-03T05:32:56Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/10917","html_url":"https://github.com/apache/spark/pull/10917","diff_url":"https://github.com/apache/spark/pull/10917.diff","patch_url":"https://github.com/apache/spark/pull/10917.patch"},"body":"Adds the benchmark results as comments.\r\n\r\nThe codegen version is slower than the interpreted version for `simple` case becasue of 3 reasons:\r\n\r\n1. codegen version use a more complex hash algorithm than interpreted version, i.e. `Murmur3_x86_32.hashInt` vs [simple multiplication and addition](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/rows.scala#L153).\r\n2. codegen version will write the hash value to a row first and then read it out. I tried to create a `GenerateHasher` that can generate code to return hash value directly and got about 60% speed up for the `simple` case, does it worth?\r\n3. the row in `simple` case only has one int field, so the runtime reflection may be removed because of branch prediction, which makes the interpreted version faster.\r\n\r\nThe `array` case is also slow for similar reasons, e.g. array elements are of same type, so interpreted version can probably get rid of runtime reflection by branch prediction."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/179021452","html_url":"https://github.com/apache/spark/pull/10917#issuecomment-179021452","issue_url":"https://api.github.com/repos/apache/spark/issues/10917","id":179021452,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2016-02-03T05:32:56Z","updated_at":"2016-02-03T05:32:56Z","body":"Test FAILed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/50640/\nTest FAILed."}}