{"action":"created","comment":{"url":"https://api.github.com/repos/rth/py-speedyfx/pulls/comments/80450337","id":80450337,"diff_hunk":"@@ -0,0 +1,65 @@\n+#include <sys/mman.h>\n+#include <sys/stat.h>\n+#include <ctype.h>\n+#include <err.h>\n+#include <fcntl.h>\n+#include <math.h>\n+#include <stdio.h>\n+#include <stdlib.h>\n+#include <unistd.h>\n+\n+/*\n+ * SpeedyFx algorithm\n+ * ==================\n+ *\n+ * Tokenize/hash large amount of strings efficiently.\n+ * Original Java implementation: http://www.hpl.hp.com/techreports/2008/HPL-2008-91R1.pdf\n+ * Ported to C by Stanislaw Pusep (https://github.com/creaktive)\n+ * Compile with:\n+ *      clang -lm -O3 -o speedyfx speedyfx.c\n+ * or:\n+ *      gcc -lm -O3 -o speedyfx speedyfx.c\n+ * Then use as:\n+ *      ./speedyfx enwik9 > fv.bin\n+ * To generate 128KB feature vector for enwik9 text file.\n+ *\n+ * Benchmark\n+ * =========\n+ *\n+ * Test data: https://cs.fit.edu/~mmahoney/compression/enwik9.bz2\n+ * Hardware: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz\n+ * Average feature vector build speed: 213.83 MB/s\n+ */\n+\n+#define SetBit(a, b) (((unsigned char *) a)[(b) >> 3] |= (1 << ((b) & 7)))","path":"speedyfx/lib/speedyfx.c","position":34,"original_position":34,"commit_id":"7eda0cde4154f0fb819da57caf644243aaa5926b","original_commit_id":"7eda0cde4154f0fb819da57caf644243aaa5926b","user":{"login":"rth","id":630936,"avatar_url":"https://avatars.githubusercontent.com/u/630936?v=3","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"body":"I do not understand what is done here, shouldn't it be just `((unsigned char *) a)[ b ] += 1` ?","created_at":"2016-09-26T10:48:27Z","updated_at":"2016-09-26T10:53:54Z","html_url":"https://github.com/rth/py-speedyfx/pull/10#discussion_r80450337","pull_request_url":"https://api.github.com/repos/rth/py-speedyfx/pulls/10","_links":{"self":{"href":"https://api.github.com/repos/rth/py-speedyfx/pulls/comments/80450337"},"html":{"href":"https://github.com/rth/py-speedyfx/pull/10#discussion_r80450337"},"pull_request":{"href":"https://api.github.com/repos/rth/py-speedyfx/pulls/10"}}},"pull_request":{"url":"https://api.github.com/repos/rth/py-speedyfx/pulls/10","id":86719566,"html_url":"https://github.com/rth/py-speedyfx/pull/10","diff_url":"https://github.com/rth/py-speedyfx/pull/10.diff","patch_url":"https://github.com/rth/py-speedyfx/pull/10.patch","issue_url":"https://api.github.com/repos/rth/py-speedyfx/issues/10","number":10,"state":"open","locked":false,"title":"[WIP] Adding the fast C implementation","user":{"login":"rth","id":630936,"avatar_url":"https://avatars.githubusercontent.com/u/630936?v=3","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"body":"Work in progress on wrapping the fast C implementation.","created_at":"2016-09-26T10:47:16Z","updated_at":"2016-09-26T10:53:54Z","closed_at":null,"merged_at":null,"merge_commit_sha":"47ce5aec0f68187f958510b58701e3dde7a05de4","assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/rth/py-speedyfx/milestones/1","html_url":"https://github.com/rth/py-speedyfx/milestones/v0.1","labels_url":"https://api.github.com/repos/rth/py-speedyfx/milestones/1/labels","id":2010927,"number":1,"title":"v0.1","description":"A first release of pyspeedyfx with the basic functionality","creator":{"login":"rth","id":630936,"avatar_url":"https://avatars.githubusercontent.com/u/630936?v=3","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"open_issues":6,"closed_issues":4,"state":"open","created_at":"2016-09-17T12:17:35Z","updated_at":"2016-09-26T10:47:16Z","due_on":"2016-10-15T00:00:00Z","closed_at":null},"commits_url":"https://api.github.com/repos/rth/py-speedyfx/pulls/10/commits","review_comments_url":"https://api.github.com/repos/rth/py-speedyfx/pulls/10/comments","review_comment_url":"https://api.github.com/repos/rth/py-speedyfx/pulls/comments{/number}","comments_url":"https://api.github.com/repos/rth/py-speedyfx/issues/10/comments","statuses_url":"https://api.github.com/repos/rth/py-speedyfx/statuses/7eda0cde4154f0fb819da57caf644243aaa5926b","head":{"label":"rth:fast-c","ref":"fast-c","sha":"7eda0cde4154f0fb819da57caf644243aaa5926b","user":{"login":"rth","id":630936,"avatar_url":"https://avatars.githubusercontent.com/u/630936?v=3","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"repo":{"id":68128050,"name":"py-speedyfx","full_name":"rth/py-speedyfx","owner":{"login":"rth","id":630936,"avatar_url":"https://avatars.githubusercontent.com/u/630936?v=3","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/rth/py-speedyfx","description":"py-SpeedyFx: Fast Text Feature Extraction for Classification and Indexing ","fork":true,"url":"https://api.github.com/repos/rth/py-speedyfx","forks_url":"https://api.github.com/repos/rth/py-speedyfx/forks","keys_url":"https://api.github.com/repos/rth/py-speedyfx/keys{/key_id}","collaborators_url":"https://api.github.com/repos/rth/py-speedyfx/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/rth/py-speedyfx/teams","hooks_url":"https://api.github.com/repos/rth/py-speedyfx/hooks","issue_events_url":"https://api.github.com/repos/rth/py-speedyfx/issues/events{/number}","events_url":"https://api.github.com/repos/rth/py-speedyfx/events","assignees_url":"https://api.github.com/repos/rth/py-speedyfx/assignees{/user}","branches_url":"https://api.github.com/repos/rth/py-speedyfx/branches{/branch}","tags_url":"https://api.github.com/repos/rth/py-speedyfx/tags","blobs_url":"https://api.github.com/repos/rth/py-speedyfx/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/rth/py-speedyfx/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/rth/py-speedyfx/git/refs{/sha}","trees_url":"https://api.github.com/repos/rth/py-speedyfx/git/trees{/sha}","statuses_url":"https://api.github.com/repos/rth/py-speedyfx/statuses/{sha}","languages_url":"https://api.github.com/repos/rth/py-speedyfx/languages","stargazers_url":"https://api.github.com/repos/rth/py-speedyfx/stargazers","contributors_url":"https://api.github.com/repos/rth/py-speedyfx/contributors","subscribers_url":"https://api.github.com/repos/rth/py-speedyfx/subscribers","subscription_url":"https://api.github.com/repos/rth/py-speedyfx/subscription","commits_url":"https://api.github.com/repos/rth/py-speedyfx/commits{/sha}","git_commits_url":"https://api.github.com/repos/rth/py-speedyfx/git/commits{/sha}","comments_url":"https://api.github.com/repos/rth/py-speedyfx/comments{/number}","issue_comment_url":"https://api.github.com/repos/rth/py-speedyfx/issues/comments{/number}","contents_url":"https://api.github.com/repos/rth/py-speedyfx/contents/{+path}","compare_url":"https://api.github.com/repos/rth/py-speedyfx/compare/{base}...{head}","merges_url":"https://api.github.com/repos/rth/py-speedyfx/merges","archive_url":"https://api.github.com/repos/rth/py-speedyfx/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/rth/py-speedyfx/downloads","issues_url":"https://api.github.com/repos/rth/py-speedyfx/issues{/number}","pulls_url":"https://api.github.com/repos/rth/py-speedyfx/pulls{/number}","milestones_url":"https://api.github.com/repos/rth/py-speedyfx/milestones{/number}","notifications_url":"https://api.github.com/repos/rth/py-speedyfx/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/rth/py-speedyfx/labels{/name}","releases_url":"https://api.github.com/repos/rth/py-speedyfx/releases{/id}","deployments_url":"https://api.github.com/repos/rth/py-speedyfx/deployments","created_at":"2016-09-13T16:49:24Z","updated_at":"2016-09-17T12:48:46Z","pushed_at":"2016-09-26T10:47:16Z","git_url":"git://github.com/rth/py-speedyfx.git","ssh_url":"git@github.com:rth/py-speedyfx.git","clone_url":"https://github.com/rth/py-speedyfx.git","svn_url":"https://github.com/rth/py-speedyfx","homepage":"","size":42,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":6,"forks":0,"open_issues":6,"watchers":0,"default_branch":"master"}},"base":{"label":"rth:master","ref":"master","sha":"836251b6903ef1070e5abdd361bcb422175d391f","user":{"login":"rth","id":630936,"avatar_url":"https://avatars.githubusercontent.com/u/630936?v=3","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"repo":{"id":68128050,"name":"py-speedyfx","full_name":"rth/py-speedyfx","owner":{"login":"rth","id":630936,"avatar_url":"https://avatars.githubusercontent.com/u/630936?v=3","gravatar_id":"","url":"https://api.github.com/users/rth","html_url":"https://github.com/rth","followers_url":"https://api.github.com/users/rth/followers","following_url":"https://api.github.com/users/rth/following{/other_user}","gists_url":"https://api.github.com/users/rth/gists{/gist_id}","starred_url":"https://api.github.com/users/rth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rth/subscriptions","organizations_url":"https://api.github.com/users/rth/orgs","repos_url":"https://api.github.com/users/rth/repos","events_url":"https://api.github.com/users/rth/events{/privacy}","received_events_url":"https://api.github.com/users/rth/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/rth/py-speedyfx","description":"py-SpeedyFx: Fast Text Feature Extraction for Classification and Indexing ","fork":true,"url":"https://api.github.com/repos/rth/py-speedyfx","forks_url":"https://api.github.com/repos/rth/py-speedyfx/forks","keys_url":"https://api.github.com/repos/rth/py-speedyfx/keys{/key_id}","collaborators_url":"https://api.github.com/repos/rth/py-speedyfx/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/rth/py-speedyfx/teams","hooks_url":"https://api.github.com/repos/rth/py-speedyfx/hooks","issue_events_url":"https://api.github.com/repos/rth/py-speedyfx/issues/events{/number}","events_url":"https://api.github.com/repos/rth/py-speedyfx/events","assignees_url":"https://api.github.com/repos/rth/py-speedyfx/assignees{/user}","branches_url":"https://api.github.com/repos/rth/py-speedyfx/branches{/branch}","tags_url":"https://api.github.com/repos/rth/py-speedyfx/tags","blobs_url":"https://api.github.com/repos/rth/py-speedyfx/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/rth/py-speedyfx/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/rth/py-speedyfx/git/refs{/sha}","trees_url":"https://api.github.com/repos/rth/py-speedyfx/git/trees{/sha}","statuses_url":"https://api.github.com/repos/rth/py-speedyfx/statuses/{sha}","languages_url":"https://api.github.com/repos/rth/py-speedyfx/languages","stargazers_url":"https://api.github.com/repos/rth/py-speedyfx/stargazers","contributors_url":"https://api.github.com/repos/rth/py-speedyfx/contributors","subscribers_url":"https://api.github.com/repos/rth/py-speedyfx/subscribers","subscription_url":"https://api.github.com/repos/rth/py-speedyfx/subscription","commits_url":"https://api.github.com/repos/rth/py-speedyfx/commits{/sha}","git_commits_url":"https://api.github.com/repos/rth/py-speedyfx/git/commits{/sha}","comments_url":"https://api.github.com/repos/rth/py-speedyfx/comments{/number}","issue_comment_url":"https://api.github.com/repos/rth/py-speedyfx/issues/comments{/number}","contents_url":"https://api.github.com/repos/rth/py-speedyfx/contents/{+path}","compare_url":"https://api.github.com/repos/rth/py-speedyfx/compare/{base}...{head}","merges_url":"https://api.github.com/repos/rth/py-speedyfx/merges","archive_url":"https://api.github.com/repos/rth/py-speedyfx/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/rth/py-speedyfx/downloads","issues_url":"https://api.github.com/repos/rth/py-speedyfx/issues{/number}","pulls_url":"https://api.github.com/repos/rth/py-speedyfx/pulls{/number}","milestones_url":"https://api.github.com/repos/rth/py-speedyfx/milestones{/number}","notifications_url":"https://api.github.com/repos/rth/py-speedyfx/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/rth/py-speedyfx/labels{/name}","releases_url":"https://api.github.com/repos/rth/py-speedyfx/releases{/id}","deployments_url":"https://api.github.com/repos/rth/py-speedyfx/deployments","created_at":"2016-09-13T16:49:24Z","updated_at":"2016-09-17T12:48:46Z","pushed_at":"2016-09-26T10:47:16Z","git_url":"git://github.com/rth/py-speedyfx.git","ssh_url":"git@github.com:rth/py-speedyfx.git","clone_url":"https://github.com/rth/py-speedyfx.git","svn_url":"https://github.com/rth/py-speedyfx","homepage":"","size":42,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":6,"forks":0,"open_issues":6,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/rth/py-speedyfx/pulls/10"},"html":{"href":"https://github.com/rth/py-speedyfx/pull/10"},"issue":{"href":"https://api.github.com/repos/rth/py-speedyfx/issues/10"},"comments":{"href":"https://api.github.com/repos/rth/py-speedyfx/issues/10/comments"},"review_comments":{"href":"https://api.github.com/repos/rth/py-speedyfx/pulls/10/comments"},"review_comment":{"href":"https://api.github.com/repos/rth/py-speedyfx/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/rth/py-speedyfx/pulls/10/commits"},"statuses":{"href":"https://api.github.com/repos/rth/py-speedyfx/statuses/7eda0cde4154f0fb819da57caf644243aaa5926b"}}}}
{"actor":{"display_login":"rth"}}