{"action":"closed","issue":{"url":"https://api.github.com/repos/loklak/loklak_server/issues/598","repository_url":"https://api.github.com/repos/loklak/loklak_server","labels_url":"https://api.github.com/repos/loklak/loklak_server/issues/598/labels{/name}","comments_url":"https://api.github.com/repos/loklak/loklak_server/issues/598/comments","events_url":"https://api.github.com/repos/loklak/loklak_server/issues/598/events","html_url":"https://github.com/loklak/loklak_server/issues/598","id":160486608,"number":598,"title":"[Generic Scraper] Add more fields to GenericScraper.java","user":{"login":"daminisatya","id":6227784,"avatar_url":"https://avatars.githubusercontent.com/u/6227784?v=3","gravatar_id":"","url":"https://api.github.com/users/daminisatya","html_url":"https://github.com/daminisatya","followers_url":"https://api.github.com/users/daminisatya/followers","following_url":"https://api.github.com/users/daminisatya/following{/other_user}","gists_url":"https://api.github.com/users/daminisatya/gists{/gist_id}","starred_url":"https://api.github.com/users/daminisatya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/daminisatya/subscriptions","organizations_url":"https://api.github.com/users/daminisatya/orgs","repos_url":"https://api.github.com/users/daminisatya/repos","events_url":"https://api.github.com/users/daminisatya/events{/privacy}","received_events_url":"https://api.github.com/users/daminisatya/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/loklak/loklak_server/labels/apps","name":"apps","color":"d4c5f9"},{"url":"https://api.github.com/repos/loklak/loklak_server/labels/data%20crawler","name":"data crawler","color":"c2e0c6"},{"url":"https://api.github.com/repos/loklak/loklak_server/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"closed","locked":false,"assignee":{"login":"daminisatya","id":6227784,"avatar_url":"https://avatars.githubusercontent.com/u/6227784?v=3","gravatar_id":"","url":"https://api.github.com/users/daminisatya","html_url":"https://github.com/daminisatya","followers_url":"https://api.github.com/users/daminisatya/followers","following_url":"https://api.github.com/users/daminisatya/following{/other_user}","gists_url":"https://api.github.com/users/daminisatya/gists{/gist_id}","starred_url":"https://api.github.com/users/daminisatya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/daminisatya/subscriptions","organizations_url":"https://api.github.com/users/daminisatya/orgs","repos_url":"https://api.github.com/users/daminisatya/repos","events_url":"https://api.github.com/users/daminisatya/events{/privacy}","received_events_url":"https://api.github.com/users/daminisatya/received_events","type":"User","site_admin":false},"assignees":[{"login":"daminisatya","id":6227784,"avatar_url":"https://avatars.githubusercontent.com/u/6227784?v=3","gravatar_id":"","url":"https://api.github.com/users/daminisatya","html_url":"https://github.com/daminisatya","followers_url":"https://api.github.com/users/daminisatya/followers","following_url":"https://api.github.com/users/daminisatya/following{/other_user}","gists_url":"https://api.github.com/users/daminisatya/gists{/gist_id}","starred_url":"https://api.github.com/users/daminisatya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/daminisatya/subscriptions","organizations_url":"https://api.github.com/users/daminisatya/orgs","repos_url":"https://api.github.com/users/daminisatya/repos","events_url":"https://api.github.com/users/daminisatya/events{/privacy}","received_events_url":"https://api.github.com/users/daminisatya/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/loklak/loklak_server/milestones/14","html_url":"https://github.com/loklak/loklak_server/milestones/GSOC%202016%20Milestone%2014%20(Final%20Evalutation)","labels_url":"https://api.github.com/repos/loklak/loklak_server/milestones/14/labels","id":1776678,"number":14,"title":"GSOC 2016 Milestone 14 (Final Evalutation)","description":"GSOC 2016 Milestone 14 - All issues to be completed by 21st August 2016.\r\n\r\nInstructions: \r\n1. Create issues to work on.\r\n2. Assign the issue to the milestone.\r\n3. Assign the issue to a user (yourself)\r\n\r\nCompleted Milestone:\r\n1. Create a blog post describing what has been done\r\n2. Tag a release\r\n3. Deployment","creator":{"login":"hpdang","id":1658742,"avatar_url":"https://avatars.githubusercontent.com/u/1658742?v=3","gravatar_id":"","url":"https://api.github.com/users/hpdang","html_url":"https://github.com/hpdang","followers_url":"https://api.github.com/users/hpdang/followers","following_url":"https://api.github.com/users/hpdang/following{/other_user}","gists_url":"https://api.github.com/users/hpdang/gists{/gist_id}","starred_url":"https://api.github.com/users/hpdang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hpdang/subscriptions","organizations_url":"https://api.github.com/users/hpdang/orgs","repos_url":"https://api.github.com/users/hpdang/repos","events_url":"https://api.github.com/users/hpdang/events{/privacy}","received_events_url":"https://api.github.com/users/hpdang/received_events","type":"User","site_admin":false},"open_issues":57,"closed_issues":6,"state":"open","created_at":"2016-05-19T14:57:52Z","updated_at":"2016-08-20T18:36:31Z","due_on":"2016-08-20T22:00:00Z","closed_at":null},"comments":2,"created_at":"2016-06-15T18:05:37Z","updated_at":"2016-08-20T18:36:31Z","closed_at":"2016-08-20T18:36:31Z","body":"When compared to Diffbot, which scrapes only the basic information from a given URL, the current API is able to provide the following details. \r\n\r\n- Language\r\n- Links \r\n- images\r\n- title\r\n- source files\r\n- Text in the link tags\r\n\r\nFurther enhancements would be better if it is able to scrape most deeply parsed data. I want suggestions on this. These are one's from my side. \r\n\r\n- [ ] Video links\r\n- [ ] Social media links\r\n- [ ] Content of the meta tags\r\n- [ ] inline scripts\r\n- [ ] page URL\r\n\r\nAlong with some other generic fields we can have options(as shown below) where specific data can be retrieved accordingly.\r\n\r\n- [Article API](https://www.diffbot.com/dev/docs/article/) (Extracts all details about the articles and other blog posts)\r\n- [Product API](https://www.diffbot.com/dev/docs/product/) (Extracts all details about the product from a given E commerce website)\r\n- [Image API](https://www.diffbot.com/dev/docs/image/) (Extracts all the images and their attributes)\r\n- [DiscussionAPI](https://www.diffbot.com/dev/docs/discussion/) (Extracts all the discussion threads, comments and likes) \r\n\r\nThe above APIs are the services currently being offered by Diffbot. The links are to their docs and sample JSON objects which they have implemented. \r\n\r\n\r\n "}}
{"actor":{"display_login":"sudheesh001"}}