{"action":"created","comment":{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/comments/78159568","id":78159568,"diff_hunk":"@@ -21,13 +21,51 @@ psql -d gis -f indexes.sql\n Additionally you need some shapefiles.\n \n ## Scripted download\n+To download the shapefiles you can run the following script. No further steps should be needed as the data has been processed and placed in the requisite directories.\n \n-To download the shapefiles you can run the following script from this directory. No further steps should be needed as the data has been processed and placed in the requisite directories.\n+```\n+scripts/get-shapefiles.py\n+```\n+\n+This script generates and populates the *data* directory with all needed\n+shapefiles, including indexing them through *shapeindex*.\n \n+Command-line syntax:\n ```\n-./get-shapefiles.sh\n+usage: get-shapefiles.py [-h] [-c] [-d <directory name>] [-e] [-f] [-l] [-n]\n+                          [-p] [-r] [-s] [-u] [-1] [-2] [-3] [-4] [-5] [-6]\n+\n+optional arguments:\n+  -h, --help            show this help message and exit\n+  -c, --check           check whether the 'data' directory already exists\n+  -d <directory name>, --directory <directory name>\n+                        set the name of the data directory (default: 'data')\n+  -e, --no-extract      do not populate target directories with the expansion\n+                        of downloaded data\n+  -f, --force           force continuing even if project.yaml does not exist\n+  -l, --no-curl         do not use 'curl' even if available\n+  -n, --no-download     do not download archive if already existing locally\n+  -p, --pause           pause before starting\n+  -r, --remove          remove each downloaded archive after its expansion\n+  -s, --no-shape        do not run shapeindex\n+  -u, --update          force downloading files even if not newer than the\n+                        locally existing ones\n+  -1, --world-boundaries\n+                        only process world_boundaries\n+  -2, --simplified-land\n+                        only process simplified-land-polygons-complete-3857\n+  -3, --ne-admin        only process ne_110m_admin_0_boundary_lines_land\n+  -4, --land-polygons   only process land-polygons-split-3857\n+  -5, --icesheet-polygons\n+                        only process antarctica-icesheet-polygons-3857\n+  -6, --icesheet-outlines\n+                        only process antarctica-icesheet-outlines-3857","path":"INSTALL.md","position":44,"original_position":44,"commit_id":"ca7277f2901d7d80802f08c4ac444ffb317da461","original_commit_id":"ca7277f2901d7d80802f08c4ac444ffb317da461","user":{"login":"Ircama","id":8292987,"avatar_url":"https://avatars.githubusercontent.com/u/8292987?v=3","gravatar_id":"","url":"https://api.github.com/users/Ircama","html_url":"https://github.com/Ircama","followers_url":"https://api.github.com/users/Ircama/followers","following_url":"https://api.github.com/users/Ircama/following{/other_user}","gists_url":"https://api.github.com/users/Ircama/gists{/gist_id}","starred_url":"https://api.github.com/users/Ircama/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Ircama/subscriptions","organizations_url":"https://api.github.com/users/Ircama/orgs","repos_url":"https://api.github.com/users/Ircama/repos","events_url":"https://api.github.com/users/Ircama/events{/privacy}","received_events_url":"https://api.github.com/users/Ircama/received_events","type":"User","site_admin":false},"body":"Ok, I'll remove them. Also, they could be reported here https://github.com/gravitystorm/openstreetmap-carto/pull/2333","created_at":"2016-09-09T10:32:16Z","updated_at":"2016-09-09T10:32:16Z","html_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2137#discussion_r78159568","pull_request_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2137","_links":{"self":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/comments/78159568"},"html":{"href":"https://github.com/gravitystorm/openstreetmap-carto/pull/2137#discussion_r78159568"},"pull_request":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2137"}}},"pull_request":{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2137","id":71888807,"html_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2137","diff_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2137.diff","patch_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2137.patch","issue_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2137","number":2137,"state":"open","locked":false,"title":"Add scripts/get-shapefiles.py","user":{"login":"Ircama","id":8292987,"avatar_url":"https://avatars.githubusercontent.com/u/8292987?v=3","gravatar_id":"","url":"https://api.github.com/users/Ircama","html_url":"https://github.com/Ircama","followers_url":"https://api.github.com/users/Ircama/followers","following_url":"https://api.github.com/users/Ircama/following{/other_user}","gists_url":"https://api.github.com/users/Ircama/gists{/gist_id}","starred_url":"https://api.github.com/users/Ircama/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Ircama/subscriptions","organizations_url":"https://api.github.com/users/Ircama/orgs","repos_url":"https://api.github.com/users/Ircama/repos","events_url":"https://api.github.com/users/Ircama/events{/privacy}","received_events_url":"https://api.github.com/users/Ircama/received_events","type":"User","site_admin":false},"body":"[Edited summary for this PR, left out on my first push]\r\n\r\nThe aim of this PR is to replace *get-shapefiles.sh* with a Python version named *get-shapefiles.py* and positioned in the *scripts* directory of *openstreetmap-carto*.\r\n\r\nIt generates and populates the *data* directory with all needed shapefiles, including indexing them through *shapeindex*. Related features are the same as the original shell script, with the addition of some command line options.\r\n\r\nThe code exploits an internal dictionary for easy adaptation to future changes, supports Windows, UNIX, Python 2.7, Python 3 and *curl* when installed (even if *curl* is not required, for more comfortable Windows support). Installation of *shapeindex* is a prerequisite to index files, but is not strictly required if `-s` option is used.\r\n\r\nThe script can be run from the *scripts* directory of *openstreetmap-carto*, or from its base folder, or through absolute path from any directory. It should allow a default Web proxy and also the configuration of the HTTP_PROXY environment variable. With Windows, PAC and NTLM authentication are not supported.\r\n\r\nDocumentation has been updated accordingly.\r\n\r\nBasic invocation: `scripts/get-shapefiles.py` (or `./get-shapefiles.sh`)\r\n\r\n```\r\nCommand-line options: get-shapefiles.py [-h] [-c] [-d <directory name>] [-e] [-f] [-l] [-n]\r\n                          [-p] [-r] [-s] [-u] [-1] [-2] [-3] [-4] [-5] [-6]\r\n\r\nOptional arguments:\r\n  -h, --help            show this help message and exit\r\n  -c, --check           check whether the 'data' directory already exists\r\n  -d <directory name>, --directory <directory name>\r\n                        set the name of the data directory (default: 'data')\r\n  -e, --no-extract      do not populate target directories with the expansion\r\n                        of downloaded data\r\n  -f, --force           force continuing even if project.yaml does not exist\r\n  -l, --no-curl         do not use 'curl' even if available\r\n  -n, --no-download     do not download archive if already existing locally\r\n  -p, --pause           pause before starting\r\n  -r, --remove          remove each downloaded archive after its expansion\r\n  -s, --no-shape        do not run shapeindex\r\n  -u, --update          force downloading files even if not newer than the\r\n                        locally existing ones\r\n  -1, --world-boundaries\r\n                        only process world_boundaries\r\n  -2, --simplified-land\r\n                        only process simplified-land-polygons-complete-3857\r\n  -3, --ne-admin        only process ne_110m_admin_0_boundary_lines_land\r\n  -4, --land-polygons   only process land-polygons-split-3857\r\n  -5, --icesheet-polygons\r\n                        only process antarctica-icesheet-polygons-3857\r\n  -6, --icesheet-outlines\r\n                        only process antarctica-icesheet-outlines-3857\r\n```\r\n","created_at":"2016-05-30T11:27:26Z","updated_at":"2016-09-09T10:32:16Z","closed_at":null,"merged_at":null,"merge_commit_sha":"64c5825d5dc795e08722281bd459f1d393083761","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2137/commits","review_comments_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2137/comments","review_comment_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/comments{/number}","comments_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2137/comments","statuses_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/statuses/ca7277f2901d7d80802f08c4ac444ffb317da461","head":{"label":"Ircama:topo-extension-get-shapefiles","ref":"topo-extension-get-shapefiles","sha":"ca7277f2901d7d80802f08c4ac444ffb317da461","user":{"login":"Ircama","id":8292987,"avatar_url":"https://avatars.githubusercontent.com/u/8292987?v=3","gravatar_id":"","url":"https://api.github.com/users/Ircama","html_url":"https://github.com/Ircama","followers_url":"https://api.github.com/users/Ircama/followers","following_url":"https://api.github.com/users/Ircama/following{/other_user}","gists_url":"https://api.github.com/users/Ircama/gists{/gist_id}","starred_url":"https://api.github.com/users/Ircama/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Ircama/subscriptions","organizations_url":"https://api.github.com/users/Ircama/orgs","repos_url":"https://api.github.com/users/Ircama/repos","events_url":"https://api.github.com/users/Ircama/events{/privacy}","received_events_url":"https://api.github.com/users/Ircama/received_events","type":"User","site_admin":false},"repo":{"id":59918342,"name":"openstreetmap-carto","full_name":"Ircama/openstreetmap-carto","owner":{"login":"Ircama","id":8292987,"avatar_url":"https://avatars.githubusercontent.com/u/8292987?v=3","gravatar_id":"","url":"https://api.github.com/users/Ircama","html_url":"https://github.com/Ircama","followers_url":"https://api.github.com/users/Ircama/followers","following_url":"https://api.github.com/users/Ircama/following{/other_user}","gists_url":"https://api.github.com/users/Ircama/gists{/gist_id}","starred_url":"https://api.github.com/users/Ircama/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Ircama/subscriptions","organizations_url":"https://api.github.com/users/Ircama/orgs","repos_url":"https://api.github.com/users/Ircama/repos","events_url":"https://api.github.com/users/Ircama/events{/privacy}","received_events_url":"https://api.github.com/users/Ircama/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Ircama/openstreetmap-carto","description":"A general-purpose OpenStreetMap mapnik style, in CartoCSS","fork":true,"url":"https://api.github.com/repos/Ircama/openstreetmap-carto","forks_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/forks","keys_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/teams","hooks_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/hooks","issue_events_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/issues/events{/number}","events_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/events","assignees_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/assignees{/user}","branches_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/branches{/branch}","tags_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/tags","blobs_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/git/refs{/sha}","trees_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/statuses/{sha}","languages_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/languages","stargazers_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/stargazers","contributors_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/contributors","subscribers_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/subscribers","subscription_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/subscription","commits_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/commits{/sha}","git_commits_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/git/commits{/sha}","comments_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/comments{/number}","issue_comment_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/issues/comments{/number}","contents_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/contents/{+path}","compare_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/merges","archive_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/downloads","issues_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/issues{/number}","pulls_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/pulls{/number}","milestones_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/milestones{/number}","notifications_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/labels{/name}","releases_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/releases{/id}","deployments_url":"https://api.github.com/repos/Ircama/openstreetmap-carto/deployments","created_at":"2016-05-28T23:22:59Z","updated_at":"2016-05-28T23:23:01Z","pushed_at":"2016-09-08T23:27:17Z","git_url":"git://github.com/Ircama/openstreetmap-carto.git","ssh_url":"git@github.com:Ircama/openstreetmap-carto.git","clone_url":"https://github.com/Ircama/openstreetmap-carto.git","svn_url":"https://github.com/Ircama/openstreetmap-carto","homepage":"","size":7540,"stargazers_count":0,"watchers_count":0,"language":"CartoCSS","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"gravitystorm:master","ref":"master","sha":"cd2bcb1ebe9d0ea8a002921735878899b5c09875","user":{"login":"gravitystorm","id":360803,"avatar_url":"https://avatars.githubusercontent.com/u/360803?v=3","gravatar_id":"","url":"https://api.github.com/users/gravitystorm","html_url":"https://github.com/gravitystorm","followers_url":"https://api.github.com/users/gravitystorm/followers","following_url":"https://api.github.com/users/gravitystorm/following{/other_user}","gists_url":"https://api.github.com/users/gravitystorm/gists{/gist_id}","starred_url":"https://api.github.com/users/gravitystorm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gravitystorm/subscriptions","organizations_url":"https://api.github.com/users/gravitystorm/orgs","repos_url":"https://api.github.com/users/gravitystorm/repos","events_url":"https://api.github.com/users/gravitystorm/events{/privacy}","received_events_url":"https://api.github.com/users/gravitystorm/received_events","type":"User","site_admin":false},"repo":{"id":6707453,"name":"openstreetmap-carto","full_name":"gravitystorm/openstreetmap-carto","owner":{"login":"gravitystorm","id":360803,"avatar_url":"https://avatars.githubusercontent.com/u/360803?v=3","gravatar_id":"","url":"https://api.github.com/users/gravitystorm","html_url":"https://github.com/gravitystorm","followers_url":"https://api.github.com/users/gravitystorm/followers","following_url":"https://api.github.com/users/gravitystorm/following{/other_user}","gists_url":"https://api.github.com/users/gravitystorm/gists{/gist_id}","starred_url":"https://api.github.com/users/gravitystorm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gravitystorm/subscriptions","organizations_url":"https://api.github.com/users/gravitystorm/orgs","repos_url":"https://api.github.com/users/gravitystorm/repos","events_url":"https://api.github.com/users/gravitystorm/events{/privacy}","received_events_url":"https://api.github.com/users/gravitystorm/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/gravitystorm/openstreetmap-carto","description":"A general-purpose OpenStreetMap mapnik style, in CartoCSS","fork":false,"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto","forks_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/forks","keys_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/teams","hooks_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/hooks","issue_events_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/events{/number}","events_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/events","assignees_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/assignees{/user}","branches_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/branches{/branch}","tags_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/tags","blobs_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/git/refs{/sha}","trees_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/statuses/{sha}","languages_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/languages","stargazers_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/stargazers","contributors_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/contributors","subscribers_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/subscribers","subscription_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/subscription","commits_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/commits{/sha}","git_commits_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/git/commits{/sha}","comments_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/comments{/number}","issue_comment_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/comments{/number}","contents_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/contents/{+path}","compare_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/merges","archive_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/downloads","issues_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues{/number}","pulls_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls{/number}","milestones_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/milestones{/number}","notifications_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/labels{/name}","releases_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/releases{/id}","deployments_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/deployments","created_at":"2012-11-15T16:06:16Z","updated_at":"2016-09-06T10:21:09Z","pushed_at":"2016-09-08T23:33:53Z","git_url":"git://github.com/gravitystorm/openstreetmap-carto.git","ssh_url":"git@github.com:gravitystorm/openstreetmap-carto.git","clone_url":"https://github.com/gravitystorm/openstreetmap-carto.git","svn_url":"https://github.com/gravitystorm/openstreetmap-carto","homepage":"","size":11195,"stargazers_count":462,"watchers_count":462,"language":"CartoCSS","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":242,"mirror_url":null,"open_issues_count":392,"forks":242,"open_issues":392,"watchers":462,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2137"},"html":{"href":"https://github.com/gravitystorm/openstreetmap-carto/pull/2137"},"issue":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2137"},"comments":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2137/comments"},"review_comments":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2137/comments"},"review_comment":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2137/commits"},"statuses":{"href":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/statuses/ca7277f2901d7d80802f08c4ac444ffb317da461"}}}}
{"actor":{"display_login":"Ircama"}}