{"action":"created","issue":{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/1504","repository_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto","labels_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/1504/labels{/name}","comments_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/1504/comments","events_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/1504/events","html_url":"https://github.com/gravitystorm/openstreetmap-carto/issues/1504","id":71609976,"number":1504,"title":"Plan database re-import","user":{"login":"math1985","id":5251909,"avatar_url":"https://avatars.githubusercontent.com/u/5251909?v=3","gravatar_id":"","url":"https://api.github.com/users/math1985","html_url":"https://github.com/math1985","followers_url":"https://api.github.com/users/math1985/followers","following_url":"https://api.github.com/users/math1985/following{/other_user}","gists_url":"https://api.github.com/users/math1985/gists{/gist_id}","starred_url":"https://api.github.com/users/math1985/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/math1985/subscriptions","organizations_url":"https://api.github.com/users/math1985/orgs","repos_url":"https://api.github.com/users/math1985/repos","events_url":"https://api.github.com/users/math1985/events{/privacy}","received_events_url":"https://api.github.com/users/math1985/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/labels/general","name":"general","color":"cccccc"},{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/labels/lua","name":"lua","color":"bfd4f2"}],"state":"open","locked":false,"assignee":{"login":"pnorman","id":1190866,"avatar_url":"https://avatars.githubusercontent.com/u/1190866?v=3","gravatar_id":"","url":"https://api.github.com/users/pnorman","html_url":"https://github.com/pnorman","followers_url":"https://api.github.com/users/pnorman/followers","following_url":"https://api.github.com/users/pnorman/following{/other_user}","gists_url":"https://api.github.com/users/pnorman/gists{/gist_id}","starred_url":"https://api.github.com/users/pnorman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pnorman/subscriptions","organizations_url":"https://api.github.com/users/pnorman/orgs","repos_url":"https://api.github.com/users/pnorman/repos","events_url":"https://api.github.com/users/pnorman/events{/privacy}","received_events_url":"https://api.github.com/users/pnorman/received_events","type":"User","site_admin":false},"assignees":[{"login":"pnorman","id":1190866,"avatar_url":"https://avatars.githubusercontent.com/u/1190866?v=3","gravatar_id":"","url":"https://api.github.com/users/pnorman","html_url":"https://github.com/pnorman","followers_url":"https://api.github.com/users/pnorman/followers","following_url":"https://api.github.com/users/pnorman/following{/other_user}","gists_url":"https://api.github.com/users/pnorman/gists{/gist_id}","starred_url":"https://api.github.com/users/pnorman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pnorman/subscriptions","organizations_url":"https://api.github.com/users/pnorman/orgs","repos_url":"https://api.github.com/users/pnorman/repos","events_url":"https://api.github.com/users/pnorman/events{/privacy}","received_events_url":"https://api.github.com/users/pnorman/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/milestones/1","html_url":"https://github.com/gravitystorm/openstreetmap-carto/milestones/3.x%20-%20Needs%20upgrade%20to%20openstreetmap-carto.style","labels_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/milestones/1/labels","id":623947,"number":1,"title":"3.x - Needs upgrade to openstreetmap-carto.style","description":"Certain features can't be implemented without changing the database layout\r\n\r\nChanging this warrants a move to a 3.x series, and ideally, any changes to either the database or the mapnik version will happen at the same time.","creator":{"login":"gravitystorm","id":360803,"avatar_url":"https://avatars.githubusercontent.com/u/360803?v=3","gravatar_id":"","url":"https://api.github.com/users/gravitystorm","html_url":"https://github.com/gravitystorm","followers_url":"https://api.github.com/users/gravitystorm/followers","following_url":"https://api.github.com/users/gravitystorm/following{/other_user}","gists_url":"https://api.github.com/users/gravitystorm/gists{/gist_id}","starred_url":"https://api.github.com/users/gravitystorm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gravitystorm/subscriptions","organizations_url":"https://api.github.com/users/gravitystorm/orgs","repos_url":"https://api.github.com/users/gravitystorm/repos","events_url":"https://api.github.com/users/gravitystorm/events{/privacy}","received_events_url":"https://api.github.com/users/gravitystorm/received_events","type":"User","site_admin":false},"open_issues":50,"closed_issues":28,"state":"open","created_at":"2014-04-08T14:24:05Z","updated_at":"2016-07-07T23:48:14Z","due_on":null,"closed_at":null},"comments":53,"created_at":"2015-04-28T14:46:43Z","updated_at":"2016-07-18T22:54:02Z","closed_at":null,"body":"This is a management issue for the upcoming re-import of the database. A re-import of the database gives us the opportunity to change the keys that are available to the stylesheet.\r\n\r\nWe will use the lua tag transformation option from osm2pgsql, and enable the hstore option.\r\n\r\n# Questions to be answered\r\n* Decide for which keys we want to change the polygon vs linestring behaviour for closed ways.\r\n* Decide for which key/value combinations we want to change the polygon vs linestring behaviour for closed ways.\r\n* Is there anything in particular that needs to be done to prepare for hstore?\r\n* Do we want to move the z_order code for roads to lua?\r\n* Are there any other changes we want to make?\r\n\r\n# Action plan\r\n* [ ] Decide on the questions above.\r\n* [ ] Prepare SQL for hstore\r\n* [ ] Implement changes in lua file.\r\n* [ ] Contact server admins to discuss date of database reload.\r\n* [ ] Database reload.\r\n* [ ] Start resolving issues that need the new database scheme.\r\n\r\n# Requested changes\r\nThe following changes have been requested:\r\n\r\n## Keys to be added\r\nAdding hstore will avoid the need to manually add these keys.\r\n* emergency #318, #1160\r\n* entrance #740, #517\r\n* golf #661\r\n* addr:flats #264, #605\r\n* allotments #432\r\n* area:highway #180\r\n* mountain_pass #244\r\n* ford #267\r\n\r\n## Keys to be added\r\nAdding hstore will avoid the need to manually add these keys.\r\n* backcountry #1485\r\n* memorial:type #1066\r\n* lock_name #828\r\n* bridge_name #828\r\n* leaf_type #822\r\n* int_name #803\r\n* basin #709\r\n* location #552, #640\r\n* protect_class #603\r\n* proposed #592\r\n* information #304, #324\r\n* network #508\r\n* height #1017\r\n* generator:output:electricity #1017\r\n* lock_name #157\r\n* bus #311, #435\r\n* vehicle #214\r\n* motor_vehicle #214\r\n* motorcar #214 \r\n\r\n## Keys to be treated as linestring\r\n* sport https://github.com/gravitystorm/openstreetmap-carto/issues/1362#issuecomment-78860353\r\n\r\n## Keys to be treated as polygon\r\n* junction #1253\r\n* boundary #468\r\n\r\n## Key/value combinations to be treated as linestring\r\nPull request written, see https://github.com/openstreetmap/osm2pgsql/pull/346.\r\n* man_made=embankment #892\r\n* natural=cliff #268\r\n* historic=citywalls #1362\r\n* man_made=breakwater #1362\r\n* man_made=groyne #1362\r\n* natural=tree_row #1362\r\n* waterway=derelict_canal #1362\r\n* waterway=ditch #1362\r\n* waterway=drain #1362\r\n* waterway=river #1362\r\n* waterway=stream #1362\r\n* waterway=wadi #1362\r\n* waterway=weir #1362\r\n* power=line #1362\r\n* power=minor_line #1362\r\n\r\n## Key/value combinations to be treated as polygon\r\nPull request written, see https://github.com/openstreetmap/osm2pgsql/pull/346.\r\n* highway=services #137\r\n\r\n## Also include as way\r\nThis has already been fixed in the current default lua style file.\r\n* ele #101"},"comment":{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/comments/233483299","html_url":"https://github.com/gravitystorm/openstreetmap-carto/issues/1504#issuecomment-233483299","issue_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/1504","id":233483299,"user":{"login":"kocio-pl","id":5439713,"avatar_url":"https://avatars.githubusercontent.com/u/5439713?v=3","gravatar_id":"","url":"https://api.github.com/users/kocio-pl","html_url":"https://github.com/kocio-pl","followers_url":"https://api.github.com/users/kocio-pl/followers","following_url":"https://api.github.com/users/kocio-pl/following{/other_user}","gists_url":"https://api.github.com/users/kocio-pl/gists{/gist_id}","starred_url":"https://api.github.com/users/kocio-pl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kocio-pl/subscriptions","organizations_url":"https://api.github.com/users/kocio-pl/orgs","repos_url":"https://api.github.com/users/kocio-pl/repos","events_url":"https://api.github.com/users/kocio-pl/events{/privacy}","received_events_url":"https://api.github.com/users/kocio-pl/received_events","type":"User","site_admin":false},"created_at":"2016-07-18T22:54:02Z","updated_at":"2016-07-18T22:54:02Z","body":"I guess I'm still missing some important details. \r\n\r\nWhen trying to import Ivory Coast file like this (~/osm/kocio-osm/ is where the lua branch is located):\r\n\r\n```osm2pgsql -G --hstore -d gis --style ~/osm/kocio-osm/openstreetmap-carto.style --tag-transform-script ~/osm/kocio-osm/openstreetmap-carto.lua ivory-coast-latest.osm.pbf```\r\n\r\nit goes rather clean:\r\n\r\n```\r\nosm2pgsql SVN version 0.88.1 (64bit id space)\r\n\r\nUsing lua based tag processing pipeline with script /home/kocio/osm/kocio-osm/openstreetmap-carto.lua\r\nUsing projection SRS 900913 (Spherical Mercator)\r\nSetting up table: planet_osm_point\r\nSetting up table: planet_osm_line\r\nSetting up table: planet_osm_polygon\r\nSetting up table: planet_osm_roads\r\nAllocating memory for dense node cache\r\nAllocating dense node cache in one big chunk\r\nAllocating memory for sparse node cache\r\nSharing dense sparse\r\nNode-cache: cache=800MB, maxblocks=102400*8192, allocation method=3\r\nMid: Ram, scale=100\r\n\r\nReading in file: ivory-coast-latest.osm.pbf\r\nProcessing: Node(2192k 438.6k/s) Way(205k 8.58k/s) Relation(780 130.00/s)  parse time: 35s\r\nNode stats: total(2192887), max(4307593803) in 5s\r\nWay stats: total(205991), max(431547109) in 24s\r\nRelation stats: total(787), max(6419700) in 6s\r\nCommitting transaction for planet_osm_point\r\nCommitting transaction for planet_osm_line\r\nCommitting transaction for planet_osm_polygon\r\nCommitting transaction for planet_osm_roads\r\nUsing lua based tag processing pipeline with script /home/kocio/osm/kocio-osm/openstreetmap-carto.lua\r\n\r\nGoing over pending ways...\r\n\t134101 ways are pending\r\n\r\nUsing 1 helper-processes\r\nFinished processing 134101 ways in 24 sec\r\n\r\n134101 Pending ways took 24s at a rate of 5587.54/s\r\nCommitting transaction for planet_osm_point\r\nCommitting transaction for planet_osm_line\r\nCommitting transaction for planet_osm_polygon\r\nCommitting transaction for planet_osm_roads\r\n\r\nGoing over pending relations...\r\n\t0 relations are pending\r\n\r\nUsing 1 helper-processes\r\nFinished processing 0 relations in 0 sec\r\n\r\nCommitting transaction for planet_osm_point\r\nWARNING:  there is no transaction in progress\r\nCommitting transaction for planet_osm_line\r\nWARNING:  there is no transaction in progress\r\nCommitting transaction for planet_osm_polygon\r\nWARNING:  there is no transaction in progress\r\nCommitting transaction for planet_osm_roads\r\nWARNING:  there is no transaction in progress\r\nnode cache: stored: 2192887(100.00%), storage efficiency: 52.99% (dense blocks: 696, sparse nodes: 1712949), hit rate: 100.00%\r\nSorting data and creating indexes for planet_osm_point\r\nSorting data and creating indexes for planet_osm_line\r\nSorting data and creating indexes for planet_osm_polygon\r\nSorting data and creating indexes for planet_osm_roads\r\nCopying planet_osm_point to cluster by geometry finished\r\nCreating geometry index on  planet_osm_point\r\nCopying planet_osm_roads to cluster by geometry finished\r\nCreating geometry index on  planet_osm_roads\r\nCreating indexes on  planet_osm_point finished\r\nCreating indexes on  planet_osm_roads finished\r\nAll indexes on  planet_osm_roads created  in 6s\r\nCompleted planet_osm_roads\r\nAll indexes on  planet_osm_point created  in 6s\r\nCompleted planet_osm_point\r\nCopying planet_osm_line to cluster by geometry finished\r\nCreating geometry index on  planet_osm_line\r\nCopying planet_osm_polygon to cluster by geometry finished\r\nCreating geometry index on  planet_osm_polygon\r\nCreating indexes on  planet_osm_line finished\r\nAll indexes on  planet_osm_line created  in 9s\r\nCompleted planet_osm_line\r\nCreating indexes on  planet_osm_polygon finished\r\nAll indexes on  planet_osm_polygon created  in 12s\r\nCompleted planet_osm_polygon\r\n\r\nOsm2pgsql took 82s overall\r\n```\r\n\r\nbut when I start Kosmtik, I get long debugging message starting with:\r\n\r\n```\r\nPostgis Plugin: ERROR:  column \"ele\" does not exist\r\nLINE 113:              OR (ele IS NOT NULL AND (\"natural\" IN ('peak', ...\r\n```\r\n\r\nand no tiles. What could be the reason?"}}
{"actor":{"display_login":"kocio-pl"}}