{"action":"created","comment":{"url":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/comments/64886281","id":64886281,"diff_hunk":"@@ -13,6 +15,12 @@ def __init__(self):\n     def run(self):\n         self.kafkaConsumer.start_listens(self.listenStore)\n \n+    @property\n+    def listenStore(self):","path":"listenstore/bin/listenstore-kafka-to-cassandra-writer.py","position":null,"original_position":14,"commit_id":"22ab58e6a152958dea9de23d36f7ee2f166f6be7","original_commit_id":"caf0f86dc8da35bca84e1580b31b5b5a61f18d4f","user":{"login":"pinkeshbadjatiya","id":14854424,"avatar_url":"https://avatars.githubusercontent.com/u/14854424?v=3","gravatar_id":"","url":"https://api.github.com/users/pinkeshbadjatiya","html_url":"https://github.com/pinkeshbadjatiya","followers_url":"https://api.github.com/users/pinkeshbadjatiya/followers","following_url":"https://api.github.com/users/pinkeshbadjatiya/following{/other_user}","gists_url":"https://api.github.com/users/pinkeshbadjatiya/gists{/gist_id}","starred_url":"https://api.github.com/users/pinkeshbadjatiya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pinkeshbadjatiya/subscriptions","organizations_url":"https://api.github.com/users/pinkeshbadjatiya/orgs","repos_url":"https://api.github.com/users/pinkeshbadjatiya/repos","events_url":"https://api.github.com/users/pinkeshbadjatiya/events{/privacy}","received_events_url":"https://api.github.com/users/pinkeshbadjatiya/received_events","type":"User","site_admin":false},"body":"done!","created_at":"2016-05-27T10:27:07Z","updated_at":"2016-05-27T10:27:07Z","html_url":"https://github.com/metabrainz/listenbrainz-server/pull/80#discussion_r64886281","pull_request_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/80","_links":{"self":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/comments/64886281"},"html":{"href":"https://github.com/metabrainz/listenbrainz-server/pull/80#discussion_r64886281"},"pull_request":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/80"}}},"pull_request":{"url":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/80","id":69995208,"html_url":"https://github.com/metabrainz/listenbrainz-server/pull/80","diff_url":"https://github.com/metabrainz/listenbrainz-server/pull/80.diff","patch_url":"https://github.com/metabrainz/listenbrainz-server/pull/80.patch","issue_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/issues/80","number":80,"state":"open","locked":false,"title":"Storing Listens in Postgres DB","user":{"login":"pinkeshbadjatiya","id":14854424,"avatar_url":"https://avatars.githubusercontent.com/u/14854424?v=3","gravatar_id":"","url":"https://api.github.com/users/pinkeshbadjatiya","html_url":"https://github.com/pinkeshbadjatiya","followers_url":"https://api.github.com/users/pinkeshbadjatiya/followers","following_url":"https://api.github.com/users/pinkeshbadjatiya/following{/other_user}","gists_url":"https://api.github.com/users/pinkeshbadjatiya/gists{/gist_id}","starred_url":"https://api.github.com/users/pinkeshbadjatiya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pinkeshbadjatiya/subscriptions","organizations_url":"https://api.github.com/users/pinkeshbadjatiya/orgs","repos_url":"https://api.github.com/users/pinkeshbadjatiya/repos","events_url":"https://api.github.com/users/pinkeshbadjatiya/events{/privacy}","received_events_url":"https://api.github.com/users/pinkeshbadjatiya/received_events","type":"User","site_admin":false},"body":"Migrate the code to submit listens to postgres + cassandra.\r\nSome major changes made in the commits,\r\n- Use `postgresql` and `cassandra` both for storing listens. (Just need to uncomment a line to switch b/w the two)\r\n- Use `sqlalchemy` as DB Driver for postgresql commits.\r\n- Separate configuration file, `listenstore.conf`, for ListenStore. (Was mentioned as TODO inside code.)\r\n- `5 secs` timeout for a batch insert in postgresql.\r\n- Remove listens from Postgres which are older than `90 days`.\r\n- Run clearing job every `24 hours`.\r\n- Add `listenstore-kafka-to-postgres-writer.py` to save listens to postgres. Older script for Cassandra also works fine, provided DB of cassandra is still present.\r\n- Support showing listens from both `Cassandra` and `Postgres`. (Just need to swap a line.)","created_at":"2016-05-13T13:26:07Z","updated_at":"2016-05-27T10:27:07Z","closed_at":null,"merged_at":null,"merge_commit_sha":"f65620dc23586a935b263d771ef1185d061e275c","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/80/commits","review_comments_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/80/comments","review_comment_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/comments{/number}","comments_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/issues/80/comments","statuses_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/statuses/22ab58e6a152958dea9de23d36f7ee2f166f6be7","head":{"label":"pinkeshbadjatiya:migrate-to-postgres","ref":"migrate-to-postgres","sha":"22ab58e6a152958dea9de23d36f7ee2f166f6be7","user":{"login":"pinkeshbadjatiya","id":14854424,"avatar_url":"https://avatars.githubusercontent.com/u/14854424?v=3","gravatar_id":"","url":"https://api.github.com/users/pinkeshbadjatiya","html_url":"https://github.com/pinkeshbadjatiya","followers_url":"https://api.github.com/users/pinkeshbadjatiya/followers","following_url":"https://api.github.com/users/pinkeshbadjatiya/following{/other_user}","gists_url":"https://api.github.com/users/pinkeshbadjatiya/gists{/gist_id}","starred_url":"https://api.github.com/users/pinkeshbadjatiya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pinkeshbadjatiya/subscriptions","organizations_url":"https://api.github.com/users/pinkeshbadjatiya/orgs","repos_url":"https://api.github.com/users/pinkeshbadjatiya/repos","events_url":"https://api.github.com/users/pinkeshbadjatiya/events{/privacy}","received_events_url":"https://api.github.com/users/pinkeshbadjatiya/received_events","type":"User","site_admin":false},"repo":{"id":53163472,"name":"listenbrainz-server","full_name":"pinkeshbadjatiya/listenbrainz-server","owner":{"login":"pinkeshbadjatiya","id":14854424,"avatar_url":"https://avatars.githubusercontent.com/u/14854424?v=3","gravatar_id":"","url":"https://api.github.com/users/pinkeshbadjatiya","html_url":"https://github.com/pinkeshbadjatiya","followers_url":"https://api.github.com/users/pinkeshbadjatiya/followers","following_url":"https://api.github.com/users/pinkeshbadjatiya/following{/other_user}","gists_url":"https://api.github.com/users/pinkeshbadjatiya/gists{/gist_id}","starred_url":"https://api.github.com/users/pinkeshbadjatiya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pinkeshbadjatiya/subscriptions","organizations_url":"https://api.github.com/users/pinkeshbadjatiya/orgs","repos_url":"https://api.github.com/users/pinkeshbadjatiya/repos","events_url":"https://api.github.com/users/pinkeshbadjatiya/events{/privacy}","received_events_url":"https://api.github.com/users/pinkeshbadjatiya/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/pinkeshbadjatiya/listenbrainz-server","description":"Server for the ListenBrainz project","fork":true,"url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server","forks_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/forks","keys_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/teams","hooks_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/hooks","issue_events_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/issues/events{/number}","events_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/events","assignees_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/assignees{/user}","branches_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/branches{/branch}","tags_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/tags","blobs_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/git/refs{/sha}","trees_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/statuses/{sha}","languages_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/languages","stargazers_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/stargazers","contributors_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/contributors","subscribers_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/subscribers","subscription_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/subscription","commits_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/commits{/sha}","git_commits_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/git/commits{/sha}","comments_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/comments{/number}","issue_comment_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/issues/comments{/number}","contents_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/contents/{+path}","compare_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/merges","archive_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/downloads","issues_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/issues{/number}","pulls_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/pulls{/number}","milestones_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/milestones{/number}","notifications_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/labels{/name}","releases_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/releases{/id}","deployments_url":"https://api.github.com/repos/pinkeshbadjatiya/listenbrainz-server/deployments","created_at":"2016-03-04T20:28:41Z","updated_at":"2016-03-04T20:28:42Z","pushed_at":"2016-05-27T10:27:01Z","git_url":"git://github.com/pinkeshbadjatiya/listenbrainz-server.git","ssh_url":"git@github.com:pinkeshbadjatiya/listenbrainz-server.git","clone_url":"https://github.com/pinkeshbadjatiya/listenbrainz-server.git","svn_url":"https://github.com/pinkeshbadjatiya/listenbrainz-server","homepage":"https://listenbrainz.org/","size":1112,"stargazers_count":0,"watchers_count":0,"language":"CSS","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"metabrainz:master","ref":"master","sha":"c3eb1dc0a55b153a7d222fa8e7de50fb0f0c7f4e","user":{"login":"metabrainz","id":293421,"avatar_url":"https://avatars.githubusercontent.com/u/293421?v=3","gravatar_id":"","url":"https://api.github.com/users/metabrainz","html_url":"https://github.com/metabrainz","followers_url":"https://api.github.com/users/metabrainz/followers","following_url":"https://api.github.com/users/metabrainz/following{/other_user}","gists_url":"https://api.github.com/users/metabrainz/gists{/gist_id}","starred_url":"https://api.github.com/users/metabrainz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/metabrainz/subscriptions","organizations_url":"https://api.github.com/users/metabrainz/orgs","repos_url":"https://api.github.com/users/metabrainz/repos","events_url":"https://api.github.com/users/metabrainz/events{/privacy}","received_events_url":"https://api.github.com/users/metabrainz/received_events","type":"Organization","site_admin":false},"repo":{"id":41960013,"name":"listenbrainz-server","full_name":"metabrainz/listenbrainz-server","owner":{"login":"metabrainz","id":293421,"avatar_url":"https://avatars.githubusercontent.com/u/293421?v=3","gravatar_id":"","url":"https://api.github.com/users/metabrainz","html_url":"https://github.com/metabrainz","followers_url":"https://api.github.com/users/metabrainz/followers","following_url":"https://api.github.com/users/metabrainz/following{/other_user}","gists_url":"https://api.github.com/users/metabrainz/gists{/gist_id}","starred_url":"https://api.github.com/users/metabrainz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/metabrainz/subscriptions","organizations_url":"https://api.github.com/users/metabrainz/orgs","repos_url":"https://api.github.com/users/metabrainz/repos","events_url":"https://api.github.com/users/metabrainz/events{/privacy}","received_events_url":"https://api.github.com/users/metabrainz/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/metabrainz/listenbrainz-server","description":"Server for the ListenBrainz project","fork":false,"url":"https://api.github.com/repos/metabrainz/listenbrainz-server","forks_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/forks","keys_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/keys{/key_id}","collaborators_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/teams","hooks_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/hooks","issue_events_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/issues/events{/number}","events_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/events","assignees_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/assignees{/user}","branches_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/branches{/branch}","tags_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/tags","blobs_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/git/refs{/sha}","trees_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/git/trees{/sha}","statuses_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/statuses/{sha}","languages_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/languages","stargazers_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/stargazers","contributors_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/contributors","subscribers_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/subscribers","subscription_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/subscription","commits_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/commits{/sha}","git_commits_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/git/commits{/sha}","comments_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/comments{/number}","issue_comment_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/issues/comments{/number}","contents_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/contents/{+path}","compare_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/compare/{base}...{head}","merges_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/merges","archive_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/downloads","issues_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/issues{/number}","pulls_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls{/number}","milestones_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/milestones{/number}","notifications_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/labels{/name}","releases_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/releases{/id}","deployments_url":"https://api.github.com/repos/metabrainz/listenbrainz-server/deployments","created_at":"2015-09-05T12:30:49Z","updated_at":"2016-05-26T18:46:06Z","pushed_at":"2016-05-27T10:27:02Z","git_url":"git://github.com/metabrainz/listenbrainz-server.git","ssh_url":"git@github.com:metabrainz/listenbrainz-server.git","clone_url":"https://github.com/metabrainz/listenbrainz-server.git","svn_url":"https://github.com/metabrainz/listenbrainz-server","homepage":"https://listenbrainz.org/","size":1089,"stargazers_count":57,"watchers_count":57,"language":"CSS","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":13,"mirror_url":null,"open_issues_count":10,"forks":13,"open_issues":10,"watchers":57,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/80"},"html":{"href":"https://github.com/metabrainz/listenbrainz-server/pull/80"},"issue":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/issues/80"},"comments":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/issues/80/comments"},"review_comments":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/80/comments"},"review_comment":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/pulls/80/commits"},"statuses":{"href":"https://api.github.com/repos/metabrainz/listenbrainz-server/statuses/22ab58e6a152958dea9de23d36f7ee2f166f6be7"}}}}
{"actor":{"display_login":"pinkeshbadjatiya"}}