{"action":"created","issue":{"url":"https://api.github.com/repos/apache/arrow/issues/2220","repository_url":"https://api.github.com/repos/apache/arrow","labels_url":"https://api.github.com/repos/apache/arrow/issues/2220/labels{/name}","comments_url":"https://api.github.com/repos/apache/arrow/issues/2220/comments","events_url":"https://api.github.com/repos/apache/arrow/issues/2220/events","html_url":"https://github.com/apache/arrow/pull/2220","id":338799406,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk5NjQwOTYx","number":2220,"title":"[Plasma] Use hashing function that takes into account all UniqueID bytes","user":{"login":"songqing","id":9260628,"node_id":"MDQ6VXNlcjkyNjA2Mjg=","avatar_url":"https://avatars1.githubusercontent.com/u/9260628?v=4","gravatar_id":"","url":"https://api.github.com/users/songqing","html_url":"https://github.com/songqing","followers_url":"https://api.github.com/users/songqing/followers","following_url":"https://api.github.com/users/songqing/following{/other_user}","gists_url":"https://api.github.com/users/songqing/gists{/gist_id}","starred_url":"https://api.github.com/users/songqing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/songqing/subscriptions","organizations_url":"https://api.github.com/users/songqing/orgs","repos_url":"https://api.github.com/users/songqing/repos","events_url":"https://api.github.com/users/songqing/events{/privacy}","received_events_url":"https://api.github.com/users/songqing/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-07-06T04:14:14Z","updated_at":"2018-07-06T06:04:28Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/arrow/pulls/2220","html_url":"https://github.com/apache/arrow/pull/2220","diff_url":"https://github.com/apache/arrow/pull/2220.diff","patch_url":"https://github.com/apache/arrow/pull/2220.patch"},"body":"Now, the hashing of UniqueID in plasma is too simple which has caused a problem.  In some cases(for example, in github/ray, UniqueID is composed of a taskID and a index),  the UniqueID may be like \"ffffffffffffffffffff00\", \"ffffffffffffffffff01\", \"fffffffffffffffffff02\" ...  . The current hashing method is only to copy the first few bytes of a UniqueID and the result is that most of the hashed ids  are same, so when the hashed ids  put to plasma store, it will become very slow when searching(plasma store uses unordered_map to store the ids, and when the keys are same, it will become slow just like list).\r\n\r\nIn fact, the same PR has been merged into ray, see https://github.com/ray-project/ray/pull/2174.\r\n\r\nand I have tested the perf between the new hashing method and the original one with putting lots of objects continuously, it seems the new hashing method doesn't cost more time."},"comment":{"url":"https://api.github.com/repos/apache/arrow/issues/comments/402935304","html_url":"https://github.com/apache/arrow/pull/2220#issuecomment-402935304","issue_url":"https://api.github.com/repos/apache/arrow/issues/2220","id":402935304,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMjkzNTMwNA==","user":{"login":"songqing","id":9260628,"node_id":"MDQ6VXNlcjkyNjA2Mjg=","avatar_url":"https://avatars1.githubusercontent.com/u/9260628?v=4","gravatar_id":"","url":"https://api.github.com/users/songqing","html_url":"https://github.com/songqing","followers_url":"https://api.github.com/users/songqing/followers","following_url":"https://api.github.com/users/songqing/following{/other_user}","gists_url":"https://api.github.com/users/songqing/gists{/gist_id}","starred_url":"https://api.github.com/users/songqing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/songqing/subscriptions","organizations_url":"https://api.github.com/users/songqing/orgs","repos_url":"https://api.github.com/users/songqing/repos","events_url":"https://api.github.com/users/songqing/events{/privacy}","received_events_url":"https://api.github.com/users/songqing/received_events","type":"User","site_admin":false},"created_at":"2018-07-06T06:04:28Z","updated_at":"2018-07-06T06:04:28Z","author_association":"CONTRIBUTOR","body":"@pcmoritz Thanks for your suggestion, I have added the murmurhash license."}}
{"actor":{"display_login":"songqing"}}