{"action":"created","issue":{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2105","repository_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto","labels_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2105/labels{/name}","comments_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2105/comments","events_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2105/events","html_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2105","id":145548968,"number":2105,"title":"Database-reload","user":{"login":"math1985","id":5251909,"avatar_url":"https://avatars.githubusercontent.com/u/5251909?v=3","gravatar_id":"","url":"https://api.github.com/users/math1985","html_url":"https://github.com/math1985","followers_url":"https://api.github.com/users/math1985/followers","following_url":"https://api.github.com/users/math1985/following{/other_user}","gists_url":"https://api.github.com/users/math1985/gists{/gist_id}","starred_url":"https://api.github.com/users/math1985/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/math1985/subscriptions","organizations_url":"https://api.github.com/users/math1985/orgs","repos_url":"https://api.github.com/users/math1985/repos","events_url":"https://api.github.com/users/math1985/events{/privacy}","received_events_url":"https://api.github.com/users/math1985/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":40,"created_at":"2016-04-03T22:53:05Z","updated_at":"2016-05-13T22:25:09Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/pulls/2105","html_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2105","diff_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2105.diff","patch_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2105.patch"},"body":"**This is a pull-request against the 4.0-branch**\r\n\r\nThis PR proposes a database-reload. It changes the documentation on the use of\r\nosm2pgsql, and adds a .lua file for preprocessing.\r\n\r\nThis database import aims to be backwards compatible with old versions of the\r\nstyle, and with other styles making use a rendering database created with\r\ndefault osm2pgsql rendering settings.\r\n\r\nThis resolves (at least part of) #1504.\r\n\r\n# Hstore\r\nAdding the hstore option to osm2pgsql allows us to use all keys.\r\n\r\nWe could roll out hstore in two different ways. First, we could just use hstore\r\nfor new keys, and keep using traditional columns for old keys. Second, we could\r\ndrop some of the columns, and start using hstore for these, in addition to using\r\nhstore for new keys.\r\n\r\nI would prefer to go the first route. The second route might lead to some\r\nperformance improvements. However, it is not clear exactly how these can be\r\nachieved. @pnorman has been working on this over a year ago (see #1504), but as\r\nfar as I know, he currently has no time to continue this project. \r\nAccording to [@pnorman's benchmarks](http://www.paulnorman.ca/blog/2014/03/osm2pgsql-and-hstore/),\r\nusing hstore without dropping columns would lead to a 9% size increase and a\r\n0.38% speed decrease. Given the benefits of having all columns available, I\r\nwould consider this acceptable.\r\n\r\nIn addition, the second way will lead to loss of compatibility with other styles\r\nthat make use of the same rendering database.\r\n\r\nI would therefore propose to go the first route, and for now enable hstore\r\nwithout dropping columns.\r\n\r\n# Make polygon/linestring decision based on value\r\n\r\nThis is based on the following unmerged PR to osm2pgsql:\r\nhttps://github.com/openstreetmap/osm2pgsql/pull/346\r\n\r\nIt makes the polygon/linestring decision based on the tag value, in addition to\r\nthe tag key. In particular, highway=services and junction=yes are now treated as\r\npolygon, while leisure=track, man_made=embankment, man_made=breakwater,\r\nman_made=groyne, natural=cliff, natural=tree_row, historic=citywalls,\r\nwaterway=derelict_canal, waterway=ditch, waterway=drain, waterway=river,\r\nwaterway=stream, waterway=wadi, waterway=weir, power=line, and power=minor_line\r\nare now treated as linestring. \r\n\r\nThis resolves #1362, resolves #137, resolves #268, and resolves #892.\r\n\r\n# Rendering order\r\nThe new .lua file creates a osmcarto_z_order value in the database, which stores\r\nthe rendering order we use. This will allow us to simplify the road queries.\r\n\r\n# Recommend -G flag for multipolygons\r\nThis resolves #59.\r\n\r\n# Import ele on buildings\r\nThis resolves #101.\r\n\r\n# Delete more meta-tags from imports\r\nThe new .style file drops some more meta-information from imports that is of no\r\nneed for rendering.\r\n\r\nThis is based on the following unmerged PR to osm2pgsql:\r\n* https://github.com/openstreetmap/osm2pgsql/pull/532\r\n\r\n# Simplify treatment of multipolygons\r\nIn the default .lua style, multipolygons with the tags on outer way were\r\naccepted as a valid alternative for multipolygons with tags on the multipolygon.\r\nBecause almost all multipolygons have the tags now on the multipolygons, support\r\nfor multipolygons with tags on the outer multipolygon has been dropped in the\r\nproposed .lua file.\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/comments/219173955","html_url":"https://github.com/gravitystorm/openstreetmap-carto/pull/2105#issuecomment-219173955","issue_url":"https://api.github.com/repos/gravitystorm/openstreetmap-carto/issues/2105","id":219173955,"user":{"login":"math1985","id":5251909,"avatar_url":"https://avatars.githubusercontent.com/u/5251909?v=3","gravatar_id":"","url":"https://api.github.com/users/math1985","html_url":"https://github.com/math1985","followers_url":"https://api.github.com/users/math1985/followers","following_url":"https://api.github.com/users/math1985/following{/other_user}","gists_url":"https://api.github.com/users/math1985/gists{/gist_id}","starred_url":"https://api.github.com/users/math1985/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/math1985/subscriptions","organizations_url":"https://api.github.com/users/math1985/orgs","repos_url":"https://api.github.com/users/math1985/repos","events_url":"https://api.github.com/users/math1985/events{/privacy}","received_events_url":"https://api.github.com/users/math1985/received_events","type":"User","site_admin":false},"created_at":"2016-05-13T22:25:09Z","updated_at":"2016-05-13T22:25:09Z","body":"I adapted to PR to re-support old-style multipolygons. Data shows there is still a very large amount of them in use, and the amount is so large that I don't think retagging them on short notice is feasible.\r\n\r\nThis does come with a large performance cost, loading Luxembourg now takes 290 seconds versus 98 seconds without support for old-style multipolygons and 223 seconds currently. I think this is not great, but acceptable. As the old-style multipolygon handling is the bottleneck, I'd expect the performance hit to be linear in terms of database size.\r\n\r\nWith respect to backwards-compatibility: as we don't need to make large sacrifices for now, I'd propose to stay backwards compatible at least this reload. For the next reload, we can consider dropping this requirement, but for now I don't see large gains.\r\n\r\nI would really like to have this tested by somebody else before rolling it out to production. @pnorman or @gravitystorm, would you perhaps be able to test this, ideally on a larger dataset? Or somebody else?"}}