{"action":"created","issue":{"url":"https://api.github.com/repos/afimb/gtfslib-python/issues/55","repository_url":"https://api.github.com/repos/afimb/gtfslib-python","labels_url":"https://api.github.com/repos/afimb/gtfslib-python/issues/55/labels{/name}","comments_url":"https://api.github.com/repos/afimb/gtfslib-python/issues/55/comments","events_url":"https://api.github.com/repos/afimb/gtfslib-python/issues/55/events","html_url":"https://github.com/afimb/gtfslib-python/issues/55","id":196034842,"number":55,"title":"Extremely high memory usage when loading a big GTFS feed","user":{"login":"elad661","id":498903,"avatar_url":"https://avatars.githubusercontent.com/u/498903?v=3","gravatar_id":"","url":"https://api.github.com/users/elad661","html_url":"https://github.com/elad661","followers_url":"https://api.github.com/users/elad661/followers","following_url":"https://api.github.com/users/elad661/following{/other_user}","gists_url":"https://api.github.com/users/elad661/gists{/gist_id}","starred_url":"https://api.github.com/users/elad661/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/elad661/subscriptions","organizations_url":"https://api.github.com/users/elad661/orgs","repos_url":"https://api.github.com/users/elad661/repos","events_url":"https://api.github.com/users/elad661/events{/privacy}","received_events_url":"https://api.github.com/users/elad661/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-12-16T11:06:30Z","updated_at":"2016-12-16T13:20:55Z","closed_at":null,"body":"I'm trying to load this feed ftp://gtfs.mot.gov.il/israel-public-transportation.zip using gtfsdbloader. The zip file is approximately 1GB when decompressed.\r\n\r\ngtfsdbloader works okay until it gets to loading of shape points, during which it uses all the memory I have available (16GB) and the system starts swapping. If I let it run for a while, the memory usage keeps growing, which would probably invoke the OOM killer once I run out of swap.\r\n\r\nIs there any way to fix it apart from getting a machine with 32GB of RAM?"},"comment":{"url":"https://api.github.com/repos/afimb/gtfslib-python/issues/comments/267592595","html_url":"https://github.com/afimb/gtfslib-python/issues/55#issuecomment-267592595","issue_url":"https://api.github.com/repos/afimb/gtfslib-python/issues/55","id":267592595,"user":{"login":"laurentg","id":959548,"avatar_url":"https://avatars.githubusercontent.com/u/959548?v=3","gravatar_id":"","url":"https://api.github.com/users/laurentg","html_url":"https://github.com/laurentg","followers_url":"https://api.github.com/users/laurentg/followers","following_url":"https://api.github.com/users/laurentg/following{/other_user}","gists_url":"https://api.github.com/users/laurentg/gists{/gist_id}","starred_url":"https://api.github.com/users/laurentg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laurentg/subscriptions","organizations_url":"https://api.github.com/users/laurentg/orgs","repos_url":"https://api.github.com/users/laurentg/repos","events_url":"https://api.github.com/users/laurentg/events{/privacy}","received_events_url":"https://api.github.com/users/laurentg/received_events","type":"User","site_admin":false},"created_at":"2016-12-16T13:20:55Z","updated_at":"2016-12-16T13:20:55Z","body":"Where does it consume memory exactly? During shape import? You can try with the following PR: https://github.com/afimb/gtfslib-python/pull/46, it's meant to optimize memory usage."}}
{"actor":{"display_login":"laurentg"}}