{"action":"created","issue":{"url":"https://api.github.com/repos/mozilla/prowac/issues/58","repository_url":"https://api.github.com/repos/mozilla/prowac","labels_url":"https://api.github.com/repos/mozilla/prowac/issues/58/labels{/name}","comments_url":"https://api.github.com/repos/mozilla/prowac/issues/58/comments","events_url":"https://api.github.com/repos/mozilla/prowac/issues/58/events","html_url":"https://github.com/mozilla/prowac/issues/58","id":144107732,"number":58,"title":"Consider open crawl data","user":{"login":"digitarald","id":8599,"avatar_url":"https://avatars.githubusercontent.com/u/8599?v=3","gravatar_id":"","url":"https://api.github.com/users/digitarald","html_url":"https://github.com/digitarald","followers_url":"https://api.github.com/users/digitarald/followers","following_url":"https://api.github.com/users/digitarald/following{/other_user}","gists_url":"https://api.github.com/users/digitarald/gists{/gist_id}","starred_url":"https://api.github.com/users/digitarald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/digitarald/subscriptions","organizations_url":"https://api.github.com/users/digitarald/orgs","repos_url":"https://api.github.com/users/digitarald/repos","events_url":"https://api.github.com/users/digitarald/events{/privacy}","received_events_url":"https://api.github.com/users/digitarald/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-03-28T23:15:59Z","updated_at":"2016-03-29T16:50:08Z","closed_at":null,"body":"Skimmed over both to assess what kind of data they could provide:\r\n\r\nhttps://commoncrawl.org/\r\n\r\n- Focused on text, doesn't crawl as much JS and CSS\r\n\r\nhttp://httparchive.org/\r\n\r\n- Mozilla is sponsor\r\n- Crawls images, CSS and JS\r\n\r\nWe need to do some more research, but httparchive might be good alternative to Prowac crawling sites."},"comment":{"url":"https://api.github.com/repos/mozilla/prowac/issues/comments/202994159","html_url":"https://github.com/mozilla/prowac/issues/58#issuecomment-202994159","issue_url":"https://api.github.com/repos/mozilla/prowac/issues/58","id":202994159,"user":{"login":"digitarald","id":8599,"avatar_url":"https://avatars.githubusercontent.com/u/8599?v=3","gravatar_id":"","url":"https://api.github.com/users/digitarald","html_url":"https://github.com/digitarald","followers_url":"https://api.github.com/users/digitarald/followers","following_url":"https://api.github.com/users/digitarald/following{/other_user}","gists_url":"https://api.github.com/users/digitarald/gists{/gist_id}","starred_url":"https://api.github.com/users/digitarald/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/digitarald/subscriptions","organizations_url":"https://api.github.com/users/digitarald/orgs","repos_url":"https://api.github.com/users/digitarald/repos","events_url":"https://api.github.com/users/digitarald/events{/privacy}","received_events_url":"https://api.github.com/users/digitarald/received_events","type":"User","site_admin":false},"created_at":"2016-03-29T16:50:08Z","updated_at":"2016-03-29T16:50:08Z","body":"httparchive\r\n\r\n> … The HTTP Archive provides this record. It is a permanent repository of web performance information such as size of pages, failed requests, and technologies utilized.\r\n\r\nIt is just request/response details and WebPagetest results on purpose.\r\n\r\nLooks like they are both not a good fit, not even giving us an edge in priming the database."}}