{"action":"created","issue":{"url":"https://api.github.com/repos/psathyrella/partis/issues/160","repository_url":"https://api.github.com/repos/psathyrella/partis","labels_url":"https://api.github.com/repos/psathyrella/partis/issues/160/labels{/name}","comments_url":"https://api.github.com/repos/psathyrella/partis/issues/160/comments","events_url":"https://api.github.com/repos/psathyrella/partis/issues/160/events","html_url":"https://github.com/psathyrella/partis/issues/160","id":128713201,"number":160,"title":"Memory optimizations","user":{"login":"psathyrella","id":2960243,"avatar_url":"https://avatars.githubusercontent.com/u/2960243?v=3","gravatar_id":"","url":"https://api.github.com/users/psathyrella","html_url":"https://github.com/psathyrella","followers_url":"https://api.github.com/users/psathyrella/followers","following_url":"https://api.github.com/users/psathyrella/following{/other_user}","gists_url":"https://api.github.com/users/psathyrella/gists{/gist_id}","starred_url":"https://api.github.com/users/psathyrella/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/psathyrella/subscriptions","organizations_url":"https://api.github.com/users/psathyrella/orgs","repos_url":"https://api.github.com/users/psathyrella/repos","events_url":"https://api.github.com/users/psathyrella/events{/privacy}","received_events_url":"https://api.github.com/users/psathyrella/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-01-26T03:25:50Z","updated_at":"2016-06-02T21:46:06Z","closed_at":null,"body":"  - Throw out cached values once you've merged them into irrelevance\r\n    - somewhat nontrivial, since you have to also remove it from all the other processes' caches\r\n  - don't give the whole cache to every process\r\n    - probably not worthwhile, it'd introduce all sorts of shenanigans to work out who needs to go where, and then the final process still has to use just as much memory.\r\n  -  reduce the number of things the master python process keeps track of, or caching more of them on disc (although the latter isn't necessarily a great option, because we already spend a significant fraction of our time reading output files)."},"comment":{"url":"https://api.github.com/repos/psathyrella/partis/issues/comments/223432531","html_url":"https://github.com/psathyrella/partis/issues/160#issuecomment-223432531","issue_url":"https://api.github.com/repos/psathyrella/partis/issues/160","id":223432531,"user":{"login":"psathyrella","id":2960243,"avatar_url":"https://avatars.githubusercontent.com/u/2960243?v=3","gravatar_id":"","url":"https://api.github.com/users/psathyrella","html_url":"https://github.com/psathyrella","followers_url":"https://api.github.com/users/psathyrella/followers","following_url":"https://api.github.com/users/psathyrella/following{/other_user}","gists_url":"https://api.github.com/users/psathyrella/gists{/gist_id}","starred_url":"https://api.github.com/users/psathyrella/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/psathyrella/subscriptions","organizations_url":"https://api.github.com/users/psathyrella/orgs","repos_url":"https://api.github.com/users/psathyrella/repos","events_url":"https://api.github.com/users/psathyrella/events{/privacy}","received_events_url":"https://api.github.com/users/psathyrella/received_events","type":"User","site_admin":false},"created_at":"2016-06-02T21:46:06Z","updated_at":"2016-06-02T21:46:06Z","body":"  - rearrange Run() and RunKSet() so you can run all the ksets for each gene at the same time -- then you can only cache one gene's values at a time\r\n  - combine trellis_, paths_, and, uh, that other map that has the same keys ('cause the keys take up a lot of space, and are time-consuming to look through)\r\n  - if fragmentation is the reason the memory usage keeps going up, it could be that pre-allocating a map that's larger than you'll need for any query will fix this."}}
{"actor":{"display_login":"psathyrella"}}