{"action":"created","issue":{"url":"https://api.github.com/repos/trailbehind/DeepOSM/issues/59","repository_url":"https://api.github.com/repos/trailbehind/DeepOSM","labels_url":"https://api.github.com/repos/trailbehind/DeepOSM/issues/59/labels{/name}","comments_url":"https://api.github.com/repos/trailbehind/DeepOSM/issues/59/comments","events_url":"https://api.github.com/repos/trailbehind/DeepOSM/issues/59/events","html_url":"https://github.com/trailbehind/DeepOSM/issues/59","id":161731764,"number":59,"title":"automate/improve infrastructure","user":{"login":"andrewljohnson","id":455551,"avatar_url":"https://avatars.githubusercontent.com/u/455551?v=3","gravatar_id":"","url":"https://api.github.com/users/andrewljohnson","html_url":"https://github.com/andrewljohnson","followers_url":"https://api.github.com/users/andrewljohnson/followers","following_url":"https://api.github.com/users/andrewljohnson/following{/other_user}","gists_url":"https://api.github.com/users/andrewljohnson/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewljohnson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewljohnson/subscriptions","organizations_url":"https://api.github.com/users/andrewljohnson/orgs","repos_url":"https://api.github.com/users/andrewljohnson/repos","events_url":"https://api.github.com/users/andrewljohnson/events{/privacy}","received_events_url":"https://api.github.com/users/andrewljohnson/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/trailbehind/DeepOSM/labels/help%20wanted","name":"help wanted","color":"159818"},{"url":"https://api.github.com/repos/trailbehind/DeepOSM/labels/Infrastructure","name":"Infrastructure","color":"006b75"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-06-22T16:45:36Z","updated_at":"2016-06-22T16:55:39Z","closed_at":null,"body":"This issue describes how deeposm works now. Then it describes changes needed to improve the infrastructure. Also see notes/scripts on these issues: #8, #23, #30, and #39 (these issues were closed to merge with this issue, not completed).\r\n\r\n## Test Data and Training\r\n* one app does both the data prep, and neural net training\r\n* it uses a GPU/Tensorflow on a Linux box in my office\r\n* findings are then uploaded to S3\r\n\r\n## Display on deeposm.org\r\n* when a deeposm.org page is loaded, it checks S3, grabs findings, and updates the database\r\n* deeposm.org shows where DeepOSM detects mis-registered roads\r\n\r\n## Issues with this Setup\r\n* The scripts to gather data, train, and upload findings should run on a cycle, not manually when I press a button\r\n* The data prep and training modules should be separate - DeepOSM has one monster Dockerfile.gpu-devel that includes GDAL, Tensorflow, and more. This makes the build fragile and hard to deploy.\r\n* Actual work includes:\r\n * move the analysis to AWS, run on a cycle\r\n * set up a cron job to have deeposm.org check for new findings\r\n * parallelize the analysis, so we can do deeper nets and more area\r\n * import NAIPs into Postgres, instead of hacking them up and caching files\r\n * use Overpass or other approach to getting OSM data, instead of hacking up PBF extracts with Osmium\r\n"},"comment":{"url":"https://api.github.com/repos/trailbehind/DeepOSM/issues/comments/227807431","html_url":"https://github.com/trailbehind/DeepOSM/issues/59#issuecomment-227807431","issue_url":"https://api.github.com/repos/trailbehind/DeepOSM/issues/59","id":227807431,"user":{"login":"andrewljohnson","id":455551,"avatar_url":"https://avatars.githubusercontent.com/u/455551?v=3","gravatar_id":"","url":"https://api.github.com/users/andrewljohnson","html_url":"https://github.com/andrewljohnson","followers_url":"https://api.github.com/users/andrewljohnson/followers","following_url":"https://api.github.com/users/andrewljohnson/following{/other_user}","gists_url":"https://api.github.com/users/andrewljohnson/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewljohnson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewljohnson/subscriptions","organizations_url":"https://api.github.com/users/andrewljohnson/orgs","repos_url":"https://api.github.com/users/andrewljohnson/repos","events_url":"https://api.github.com/users/andrewljohnson/events{/privacy}","received_events_url":"https://api.github.com/users/andrewljohnson/received_events","type":"User","site_admin":false},"created_at":"2016-06-22T16:55:38Z","updated_at":"2016-06-22T16:55:38Z","body":"@anandthakker I'm probably wrong, but I decided it doesn't really help me to use \"tiled\" data, because my source imagery isn't projected to Mercator, or tiled into a TMS pyramid?\r\n\r\nWhen I did my first run at this, I used Mapzen vector tiles, plus TMS imagery tiles (since they align). But when I switched to NAIPs, it seemed to make more sense (less code) to clip some line strings to the bounds of my NAIPs or arbitrary tiles.\r\n\r\nSo that's also why I think my end solution is a planet DB with an API, and maybe that API is Overpass, or something simple I just cook up for this use case? "}}
{"actor":{"display_login":"andrewljohnson"}}