{"action":"opened","number":44,"pull_request":{"url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls/44","id":76097267,"html_url":"https://github.com/MachineLearningStudyGroup/Smart_Review_Summarization/pull/44","diff_url":"https://github.com/MachineLearningStudyGroup/Smart_Review_Summarization/pull/44.diff","patch_url":"https://github.com/MachineLearningStudyGroup/Smart_Review_Summarization/pull/44.patch","issue_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/issues/44","number":44,"state":"open","locked":false,"title":"Db peter","user":{"login":"zhilongz","id":11689770,"avatar_url":"https://avatars.githubusercontent.com/u/11689770?v=3","gravatar_id":"","url":"https://api.github.com/users/zhilongz","html_url":"https://github.com/zhilongz","followers_url":"https://api.github.com/users/zhilongz/followers","following_url":"https://api.github.com/users/zhilongz/following{/other_user}","gists_url":"https://api.github.com/users/zhilongz/gists{/gist_id}","starred_url":"https://api.github.com/users/zhilongz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhilongz/subscriptions","organizations_url":"https://api.github.com/users/zhilongz/orgs","repos_url":"https://api.github.com/users/zhilongz/repos","events_url":"https://api.github.com/users/zhilongz/events{/privacy}","received_events_url":"https://api.github.com/users/zhilongz/received_events","type":"User","site_admin":false},"body":"DB and scraper section update 2\r\n1. database.py: \r\n(a) added fields reviewID, review rating and total number of review to db\r\n(b) added has_review_id() to check if review already stored in db\r\n(c) added update_contents_for_product_id to append new review data to \r\nexisting data instead of overwrite existing data\r\n2update_contents_for_product_id. scraper.py: \r\n(a) backup scraper that scrape from html if amazon API failed \r\n(b) currently scrape contents, reviewID, review rating, and total number of reviews\r\n(c) checking if a review in db before scrape content \r\n3. srs_local.py: \r\n(a) logic: (i) check if product has any review\r\n\r\nNO -> trigger scraper, process review, store data\r\nOtherwise -> check if content is processed\r\nIf yes -> scrape only new review no in db, process review, update db\r\nTO DO: \r\n1. edit load_db_from_files() in database.py to handle stanford data instead of txt data\r\n2. processing review after scraping is slow, may consider speeding it up\r\n\r\nSigned-off-by: Zhilong Zhu peterz@dhcp-18-111-58-112.dyn.mit.edu","created_at":"2016-07-03T20:07:49Z","updated_at":"2016-07-03T20:07:50Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls/44/commits","review_comments_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls/44/comments","review_comment_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls/comments{/number}","comments_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/issues/44/comments","statuses_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/statuses/bcb69a8a34608b48acdde12242b0c402bebf73cd","head":{"label":"zhilongz:db_peter","ref":"db_peter","sha":"bcb69a8a34608b48acdde12242b0c402bebf73cd","user":{"login":"zhilongz","id":11689770,"avatar_url":"https://avatars.githubusercontent.com/u/11689770?v=3","gravatar_id":"","url":"https://api.github.com/users/zhilongz","html_url":"https://github.com/zhilongz","followers_url":"https://api.github.com/users/zhilongz/followers","following_url":"https://api.github.com/users/zhilongz/following{/other_user}","gists_url":"https://api.github.com/users/zhilongz/gists{/gist_id}","starred_url":"https://api.github.com/users/zhilongz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhilongz/subscriptions","organizations_url":"https://api.github.com/users/zhilongz/orgs","repos_url":"https://api.github.com/users/zhilongz/repos","events_url":"https://api.github.com/users/zhilongz/events{/privacy}","received_events_url":"https://api.github.com/users/zhilongz/received_events","type":"User","site_admin":false},"repo":{"id":53264634,"name":"Smart_Review_Summarization","full_name":"zhilongz/Smart_Review_Summarization","owner":{"login":"zhilongz","id":11689770,"avatar_url":"https://avatars.githubusercontent.com/u/11689770?v=3","gravatar_id":"","url":"https://api.github.com/users/zhilongz","html_url":"https://github.com/zhilongz","followers_url":"https://api.github.com/users/zhilongz/followers","following_url":"https://api.github.com/users/zhilongz/following{/other_user}","gists_url":"https://api.github.com/users/zhilongz/gists{/gist_id}","starred_url":"https://api.github.com/users/zhilongz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhilongz/subscriptions","organizations_url":"https://api.github.com/users/zhilongz/orgs","repos_url":"https://api.github.com/users/zhilongz/repos","events_url":"https://api.github.com/users/zhilongz/events{/privacy}","received_events_url":"https://api.github.com/users/zhilongz/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/zhilongz/Smart_Review_Summarization","description":"An automatic summarization tool for product reviews","fork":true,"url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization","forks_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/forks","keys_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/keys{/key_id}","collaborators_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/teams","hooks_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/hooks","issue_events_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/issues/events{/number}","events_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/events","assignees_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/assignees{/user}","branches_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/branches{/branch}","tags_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/tags","blobs_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/git/refs{/sha}","trees_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/git/trees{/sha}","statuses_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/statuses/{sha}","languages_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/languages","stargazers_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/stargazers","contributors_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/contributors","subscribers_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/subscribers","subscription_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/subscription","commits_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/commits{/sha}","git_commits_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/git/commits{/sha}","comments_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/comments{/number}","issue_comment_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/issues/comments{/number}","contents_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/contents/{+path}","compare_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/compare/{base}...{head}","merges_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/merges","archive_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/downloads","issues_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/issues{/number}","pulls_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/pulls{/number}","milestones_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/milestones{/number}","notifications_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/labels{/name}","releases_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/releases{/id}","deployments_url":"https://api.github.com/repos/zhilongz/Smart_Review_Summarization/deployments","created_at":"2016-03-06T16:32:43Z","updated_at":"2016-03-06T16:32:46Z","pushed_at":"2016-07-03T20:05:25Z","git_url":"git://github.com/zhilongz/Smart_Review_Summarization.git","ssh_url":"git@github.com:zhilongz/Smart_Review_Summarization.git","clone_url":"https://github.com/zhilongz/Smart_Review_Summarization.git","svn_url":"https://github.com/zhilongz/Smart_Review_Summarization","homepage":null,"size":693,"stargazers_count":0,"watchers_count":0,"language":"Jupyter Notebook","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"MachineLearningStudyGroup:master","ref":"master","sha":"e386ca83af29570cb767e9342d2e4f3f9292f41a","user":{"login":"MachineLearningStudyGroup","id":17015541,"avatar_url":"https://avatars.githubusercontent.com/u/17015541?v=3","gravatar_id":"","url":"https://api.github.com/users/MachineLearningStudyGroup","html_url":"https://github.com/MachineLearningStudyGroup","followers_url":"https://api.github.com/users/MachineLearningStudyGroup/followers","following_url":"https://api.github.com/users/MachineLearningStudyGroup/following{/other_user}","gists_url":"https://api.github.com/users/MachineLearningStudyGroup/gists{/gist_id}","starred_url":"https://api.github.com/users/MachineLearningStudyGroup/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MachineLearningStudyGroup/subscriptions","organizations_url":"https://api.github.com/users/MachineLearningStudyGroup/orgs","repos_url":"https://api.github.com/users/MachineLearningStudyGroup/repos","events_url":"https://api.github.com/users/MachineLearningStudyGroup/events{/privacy}","received_events_url":"https://api.github.com/users/MachineLearningStudyGroup/received_events","type":"Organization","site_admin":false},"repo":{"id":53262459,"name":"Smart_Review_Summarization","full_name":"MachineLearningStudyGroup/Smart_Review_Summarization","owner":{"login":"MachineLearningStudyGroup","id":17015541,"avatar_url":"https://avatars.githubusercontent.com/u/17015541?v=3","gravatar_id":"","url":"https://api.github.com/users/MachineLearningStudyGroup","html_url":"https://github.com/MachineLearningStudyGroup","followers_url":"https://api.github.com/users/MachineLearningStudyGroup/followers","following_url":"https://api.github.com/users/MachineLearningStudyGroup/following{/other_user}","gists_url":"https://api.github.com/users/MachineLearningStudyGroup/gists{/gist_id}","starred_url":"https://api.github.com/users/MachineLearningStudyGroup/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MachineLearningStudyGroup/subscriptions","organizations_url":"https://api.github.com/users/MachineLearningStudyGroup/orgs","repos_url":"https://api.github.com/users/MachineLearningStudyGroup/repos","events_url":"https://api.github.com/users/MachineLearningStudyGroup/events{/privacy}","received_events_url":"https://api.github.com/users/MachineLearningStudyGroup/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/MachineLearningStudyGroup/Smart_Review_Summarization","description":"An automatic summarization tool for product reviews","fork":false,"url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization","forks_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/forks","keys_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/keys{/key_id}","collaborators_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/teams","hooks_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/hooks","issue_events_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/issues/events{/number}","events_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/events","assignees_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/assignees{/user}","branches_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/branches{/branch}","tags_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/tags","blobs_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/git/refs{/sha}","trees_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/git/trees{/sha}","statuses_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/statuses/{sha}","languages_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/languages","stargazers_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/stargazers","contributors_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/contributors","subscribers_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/subscribers","subscription_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/subscription","commits_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/commits{/sha}","git_commits_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/git/commits{/sha}","comments_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/comments{/number}","issue_comment_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/issues/comments{/number}","contents_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/contents/{+path}","compare_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/compare/{base}...{head}","merges_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/merges","archive_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/downloads","issues_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/issues{/number}","pulls_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls{/number}","milestones_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/milestones{/number}","notifications_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/labels{/name}","releases_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/releases{/id}","deployments_url":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/deployments","created_at":"2016-03-06T15:44:56Z","updated_at":"2016-06-16T00:21:41Z","pushed_at":"2016-07-03T20:07:50Z","git_url":"git://github.com/MachineLearningStudyGroup/Smart_Review_Summarization.git","ssh_url":"git@github.com:MachineLearningStudyGroup/Smart_Review_Summarization.git","clone_url":"https://github.com/MachineLearningStudyGroup/Smart_Review_Summarization.git","svn_url":"https://github.com/MachineLearningStudyGroup/Smart_Review_Summarization","homepage":null,"size":37488,"stargazers_count":1,"watchers_count":1,"language":"Jupyter Notebook","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":4,"mirror_url":null,"open_issues_count":1,"forks":4,"open_issues":1,"watchers":1,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls/44"},"html":{"href":"https://github.com/MachineLearningStudyGroup/Smart_Review_Summarization/pull/44"},"issue":{"href":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/issues/44"},"comments":{"href":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/issues/44/comments"},"review_comments":{"href":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls/44/comments"},"review_comment":{"href":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/pulls/44/commits"},"statuses":{"href":"https://api.github.com/repos/MachineLearningStudyGroup/Smart_Review_Summarization/statuses/bcb69a8a34608b48acdde12242b0c402bebf73cd"}},"merged":false,"mergeable":null,"mergeable_state":"unknown","merged_by":null,"comments":0,"review_comments":0,"commits":3,"additions":238,"deletions":60,"changed_files":4}}
{"actor":{"display_login":"zhilongz"}}