{"action":"created","issue":{"url":"https://api.github.com/repos/vnmakarov/mum-hash/issues/3","repository_url":"https://api.github.com/repos/vnmakarov/mum-hash","labels_url":"https://api.github.com/repos/vnmakarov/mum-hash/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/vnmakarov/mum-hash/issues/3/comments","events_url":"https://api.github.com/repos/vnmakarov/mum-hash/issues/3/events","html_url":"https://github.com/vnmakarov/mum-hash/issues/3","id":154303952,"number":3,"title":"Performance benchmarks with strings that are not multiple of 8 bytes","user":{"login":"vstakhov","id":487344,"avatar_url":"https://avatars.githubusercontent.com/u/487344?v=3","gravatar_id":"","url":"https://api.github.com/users/vstakhov","html_url":"https://github.com/vstakhov","followers_url":"https://api.github.com/users/vstakhov/followers","following_url":"https://api.github.com/users/vstakhov/following{/other_user}","gists_url":"https://api.github.com/users/vstakhov/gists{/gist_id}","starred_url":"https://api.github.com/users/vstakhov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vstakhov/subscriptions","organizations_url":"https://api.github.com/users/vstakhov/orgs","repos_url":"https://api.github.com/users/vstakhov/repos","events_url":"https://api.github.com/users/vstakhov/events{/privacy}","received_events_url":"https://api.github.com/users/vstakhov/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":13,"created_at":"2016-05-11T17:54:47Z","updated_at":"2016-06-06T12:54:44Z","closed_at":null,"body":"Hello, I'd like to thank you for your hard work - I really like your algorithm. \r\n\r\nHowever, I have a sort of question: I tried mum-hash to calculate shingles which implies a lot of hash calculations for individual words in some arbitrary text. It turns out, that mum-hash performs quite bad in these conditions: it is slower than xxhash64:\r\n\r\nmum-hash:\r\n```\r\n(50000 words of 5 len, 0.02 perm factor): percentage of common shingles: 0.968, generate time: 0.0533 sec\r\n(50000 words of 16 len, 0.02 perm factor): percentage of common shingles: 0.937, generate time: 0.0351 sec\r\n```\r\n\r\nxxhash64:\r\n```\r\n(50000 words of 5 len, 0.02 perm factor): percentage of common shingles: 0.781, generate time: 0.0330 sec\r\n(50000 words of 16 len, 0.02 perm factor): percentage of common shingles: 0.906, generate time: 0.0361 sec\r\n```\r\n\r\nAs you can see, for inputs that are not 'good' for hash the performance is reduced significantly. I think this is worth to mention in the documentation."},"comment":{"url":"https://api.github.com/repos/vnmakarov/mum-hash/issues/comments/223950187","html_url":"https://github.com/vnmakarov/mum-hash/issues/3#issuecomment-223950187","issue_url":"https://api.github.com/repos/vnmakarov/mum-hash/issues/3","id":223950187,"user":{"login":"HashTang","id":19777827,"avatar_url":"https://avatars.githubusercontent.com/u/19777827?v=3","gravatar_id":"","url":"https://api.github.com/users/HashTang","html_url":"https://github.com/HashTang","followers_url":"https://api.github.com/users/HashTang/followers","following_url":"https://api.github.com/users/HashTang/following{/other_user}","gists_url":"https://api.github.com/users/HashTang/gists{/gist_id}","starred_url":"https://api.github.com/users/HashTang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HashTang/subscriptions","organizations_url":"https://api.github.com/users/HashTang/orgs","repos_url":"https://api.github.com/users/HashTang/repos","events_url":"https://api.github.com/users/HashTang/events{/privacy}","received_events_url":"https://api.github.com/users/HashTang/received_events","type":"User","site_admin":false},"created_at":"2016-06-06T12:54:44Z","updated_at":"2016-06-06T12:54:44Z","body":"Hello, I've been interested in testing your benchmark program.\r\nBut I am surprised, because it seems to measure xxHash differently from other algorithms.\r\n\r\nIt invokes the heavier streaming API, requiring multiple calls :\r\n\r\n```\r\nstatic XXH64_state_t* state = NULL;\r\n\r\nstatic void xxHash64_test(const void *key, int len, uint32_t seed, void *out) {\r\n  if (! state) state = XXH64_createState ();\r\n  XXH64_reset (state, seed);\r\n  XXH64_update (state, key, len);\r\n  *(uint64_t*)out = XXH64_digest (state);\r\n}\r\n```\r\n\r\nIt could have been much simpler : \r\n```\r\nstatic void xxHash64_test(const void *key, int len, uint32_t seed, void *out) {\r\n  *(uint64_t*)out = XXH64(key, len, seed);\r\n}\r\n```\r\n\r\nThis is surprising because \"the short version\" is the one used for all other algorithms :\r\n\r\n```\r\n*(uint64*)out = CityHash64WithSeed((const char *)key,len,seed);\r\nsiphash(out, (const uint8_t *) key, len, (const uint8_t *) s);\r\nMetroHash64::Hash ((const uint8_t *)key, len, (uint8_t *) out, seed);\r\n*(uint64_t *)out = mum_hash (key, len, seed);\r\n```\r\n\r\nThe streaming API is meant for very long content. It has an additional burden with state maintenance. Using it for short keys is overkill and just reduces performance.\r\n\r\nAnyway, fixing this difference, the results are quite impacted. Similar to @vstakhov , I find xxHash to be 2x - 3x faster on short keys."}}
{"actor":{"display_login":"HashTang"}}