{"action":"created","issue":{"url":"https://api.github.com/repos/nirgn975/open_pension_scraper/issues/4","repository_url":"https://api.github.com/repos/nirgn975/open_pension_scraper","labels_url":"https://api.github.com/repos/nirgn975/open_pension_scraper/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/nirgn975/open_pension_scraper/issues/4/comments","events_url":"https://api.github.com/repos/nirgn975/open_pension_scraper/issues/4/events","html_url":"https://github.com/nirgn975/open_pension_scraper/issues/4","id":179754346,"number":4,"title":"Crawler","user":{"login":"nirgn975","id":3472902,"avatar_url":"https://avatars.githubusercontent.com/u/3472902?v=3","gravatar_id":"","url":"https://api.github.com/users/nirgn975","html_url":"https://github.com/nirgn975","followers_url":"https://api.github.com/users/nirgn975/followers","following_url":"https://api.github.com/users/nirgn975/following{/other_user}","gists_url":"https://api.github.com/users/nirgn975/gists{/gist_id}","starred_url":"https://api.github.com/users/nirgn975/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nirgn975/subscriptions","organizations_url":"https://api.github.com/users/nirgn975/orgs","repos_url":"https://api.github.com/users/nirgn975/repos","events_url":"https://api.github.com/users/nirgn975/events{/privacy}","received_events_url":"https://api.github.com/users/nirgn975/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"daniellevi1","id":19608168,"avatar_url":"https://avatars.githubusercontent.com/u/19608168?v=3","gravatar_id":"","url":"https://api.github.com/users/daniellevi1","html_url":"https://github.com/daniellevi1","followers_url":"https://api.github.com/users/daniellevi1/followers","following_url":"https://api.github.com/users/daniellevi1/following{/other_user}","gists_url":"https://api.github.com/users/daniellevi1/gists{/gist_id}","starred_url":"https://api.github.com/users/daniellevi1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/daniellevi1/subscriptions","organizations_url":"https://api.github.com/users/daniellevi1/orgs","repos_url":"https://api.github.com/users/daniellevi1/repos","events_url":"https://api.github.com/users/daniellevi1/events{/privacy}","received_events_url":"https://api.github.com/users/daniellevi1/received_events","type":"User","site_admin":false},"assignees":[{"login":"daniellevi1","id":19608168,"avatar_url":"https://avatars.githubusercontent.com/u/19608168?v=3","gravatar_id":"","url":"https://api.github.com/users/daniellevi1","html_url":"https://github.com/daniellevi1","followers_url":"https://api.github.com/users/daniellevi1/followers","following_url":"https://api.github.com/users/daniellevi1/following{/other_user}","gists_url":"https://api.github.com/users/daniellevi1/gists{/gist_id}","starred_url":"https://api.github.com/users/daniellevi1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/daniellevi1/subscriptions","organizations_url":"https://api.github.com/users/daniellevi1/orgs","repos_url":"https://api.github.com/users/daniellevi1/repos","events_url":"https://api.github.com/users/daniellevi1/events{/privacy}","received_events_url":"https://api.github.com/users/daniellevi1/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2016-09-28T12:07:13Z","updated_at":"2016-09-29T23:47:07Z","closed_at":null,"body":"We'll start with [ילין לפידות](https://www.yl-invest.co.il/).\r\nWrite a basic crawler (or spider as it called in Scrapy) that just start with the website basic url (which is `https://www.yl-invest.co.il/`) and extract all the links, save them, and then follow them to the next page and search links in there.\r\n\r\nFor start, let's say the crawler will stop follow links when he find the css rule `.sBodyTD > #ctlBody` (you can find it in this page [צור קשר](https://www.yl-invest.co.il/?SourceID=20&CategoryID=20&ArticleID=10)) (open the chrome inspect and search (`ctrl + f`) there `.sBodyTD > #ctlBody`. You'll see only 1 match).\r\n\r\nCheck the [scrapy documentation](https://doc.scrapy.org/en/latest/) for help."},"comment":{"url":"https://api.github.com/repos/nirgn975/open_pension_scraper/issues/comments/250623477","html_url":"https://github.com/nirgn975/open_pension_scraper/issues/4#issuecomment-250623477","issue_url":"https://api.github.com/repos/nirgn975/open_pension_scraper/issues/4","id":250623477,"user":{"login":"daniellevi1","id":19608168,"avatar_url":"https://avatars.githubusercontent.com/u/19608168?v=3","gravatar_id":"","url":"https://api.github.com/users/daniellevi1","html_url":"https://github.com/daniellevi1","followers_url":"https://api.github.com/users/daniellevi1/followers","following_url":"https://api.github.com/users/daniellevi1/following{/other_user}","gists_url":"https://api.github.com/users/daniellevi1/gists{/gist_id}","starred_url":"https://api.github.com/users/daniellevi1/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/daniellevi1/subscriptions","organizations_url":"https://api.github.com/users/daniellevi1/orgs","repos_url":"https://api.github.com/users/daniellevi1/repos","events_url":"https://api.github.com/users/daniellevi1/events{/privacy}","received_events_url":"https://api.github.com/users/daniellevi1/received_events","type":"User","site_admin":false},"created_at":"2016-09-29T23:47:07Z","updated_at":"2016-09-29T23:47:07Z","body":"@nirgn975 Ok great"}}
{"actor":{"display_login":"daniellevi1"}}