{"action":"opened","issue":{"url":"https://api.github.com/repos/wdiv-scrapers/dc-base-scrapers/issues/3","repository_url":"https://api.github.com/repos/wdiv-scrapers/dc-base-scrapers","labels_url":"https://api.github.com/repos/wdiv-scrapers/dc-base-scrapers/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/wdiv-scrapers/dc-base-scrapers/issues/3/comments","events_url":"https://api.github.com/repos/wdiv-scrapers/dc-base-scrapers/issues/3/events","html_url":"https://github.com/wdiv-scrapers/dc-base-scrapers/issues/3","id":362536381,"node_id":"MDU6SXNzdWUzNjI1MzYzODE=","number":3,"title":"Add base class for ArcGIS Open Data v2 API","user":{"login":"chris48s","id":6025893,"node_id":"MDQ6VXNlcjYwMjU4OTM=","avatar_url":"https://avatars1.githubusercontent.com/u/6025893?v=4","gravatar_id":"","url":"https://api.github.com/users/chris48s","html_url":"https://github.com/chris48s","followers_url":"https://api.github.com/users/chris48s/followers","following_url":"https://api.github.com/users/chris48s/following{/other_user}","gists_url":"https://api.github.com/users/chris48s/gists{/gist_id}","starred_url":"https://api.github.com/users/chris48s/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chris48s/subscriptions","organizations_url":"https://api.github.com/users/chris48s/orgs","repos_url":"https://api.github.com/users/chris48s/repos","events_url":"https://api.github.com/users/chris48s/events{/privacy}","received_events_url":"https://api.github.com/users/chris48s/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-09-21T09:44:49Z","updated_at":"2018-09-21T09:44:49Z","closed_at":null,"author_association":"MEMBER","body":"There are a number of councils where we are calling `https://opendata.arcgis.com/api/v2/datasets` to identify new data e.g:\r\n\r\n* https://opendata.arcgis.com/api/v2/datasets?filter%5Bcatalogs%5D=data-moray.opendata.arcgis.com&include=organizations%2Cgroups&page%5Bnumber%5D=1&page%5Bsize%5D=10&q=polling&sort=-updatedAt\r\n* https://opendata.arcgis.com/api/v2/datasets?filter%5Bcatalogs%5D=maps-south-ayrshire.opendata.arcgis.com&include=organizations%2Cgroups&page%5Bnumber%5D=1&page%5Bsize%5D=10&q=polling&sort=-updatedAt\r\n* https://opendata.arcgis.com/api/v2/datasets?filter%5Btags%5D=Elections&filter%5Bcatalogs%5D=maps-broxtowe.opendata.arcgis.com&include=organizations%2Cgroups&page%5Bnumber%5D=1&page%5Bsize%5D=10&sort=-updatedAt\r\n* http://gis-pkc.opendata.arcgis.com/datasets.json?q=polling&sort_by=-updatedAt\r\n\r\nAt the moment we just pass a general query and sync the data to GH so we can check a diff when something changes. If it changes, we know to update the scrapers. It would be useful to add a base class like we have for CKAN and Datapress that can parse that data a bit more intelligently and do it for us.\r\n\r\nTunbridge Wells is a starting point for this:\r\n\r\nhttps://github.com/wdiv-scrapers/DC-PollingStations-TunbridgeWells/blob/master/scraper.py"}}
{"actor":{"display_login":"chris48s"}}