{"action":"created","issue":{"url":"https://api.github.com/repos/astropy/halotools/issues/568","repository_url":"https://api.github.com/repos/astropy/halotools","labels_url":"https://api.github.com/repos/astropy/halotools/issues/568/labels{/name}","comments_url":"https://api.github.com/repos/astropy/halotools/issues/568/comments","events_url":"https://api.github.com/repos/astropy/halotools/issues/568/events","html_url":"https://github.com/astropy/halotools/issues/568","id":162481532,"number":568,"title":"Suggestion: Improved memory use","user":{"login":"johannesulf","id":4167911,"avatar_url":"https://avatars.githubusercontent.com/u/4167911?v=3","gravatar_id":"","url":"https://api.github.com/users/johannesulf","html_url":"https://github.com/johannesulf","followers_url":"https://api.github.com/users/johannesulf/followers","following_url":"https://api.github.com/users/johannesulf/following{/other_user}","gists_url":"https://api.github.com/users/johannesulf/gists{/gist_id}","starred_url":"https://api.github.com/users/johannesulf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/johannesulf/subscriptions","organizations_url":"https://api.github.com/users/johannesulf/orgs","repos_url":"https://api.github.com/users/johannesulf/repos","events_url":"https://api.github.com/users/johannesulf/events{/privacy}","received_events_url":"https://api.github.com/users/johannesulf/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/astropy/halotools/labels/empirical-models","name":"empirical-models","color":"fef2c0"},{"url":"https://api.github.com/repos/astropy/halotools/labels/performance-enhancement","name":"performance-enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":{"login":"aphearin","id":6951595,"avatar_url":"https://avatars.githubusercontent.com/u/6951595?v=3","gravatar_id":"","url":"https://api.github.com/users/aphearin","html_url":"https://github.com/aphearin","followers_url":"https://api.github.com/users/aphearin/followers","following_url":"https://api.github.com/users/aphearin/following{/other_user}","gists_url":"https://api.github.com/users/aphearin/gists{/gist_id}","starred_url":"https://api.github.com/users/aphearin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aphearin/subscriptions","organizations_url":"https://api.github.com/users/aphearin/orgs","repos_url":"https://api.github.com/users/aphearin/repos","events_url":"https://api.github.com/users/aphearin/events{/privacy}","received_events_url":"https://api.github.com/users/aphearin/received_events","type":"User","site_admin":false},"assignees":[{"login":"aphearin","id":6951595,"avatar_url":"https://avatars.githubusercontent.com/u/6951595?v=3","gravatar_id":"","url":"https://api.github.com/users/aphearin","html_url":"https://github.com/aphearin","followers_url":"https://api.github.com/users/aphearin/followers","following_url":"https://api.github.com/users/aphearin/following{/other_user}","gists_url":"https://api.github.com/users/aphearin/gists{/gist_id}","starred_url":"https://api.github.com/users/aphearin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aphearin/subscriptions","organizations_url":"https://api.github.com/users/aphearin/orgs","repos_url":"https://api.github.com/users/aphearin/repos","events_url":"https://api.github.com/users/aphearin/events{/privacy}","received_events_url":"https://api.github.com/users/aphearin/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/astropy/halotools/milestones/8","html_url":"https://github.com/astropy/halotools/milestones/next-release","labels_url":"https://api.github.com/repos/astropy/halotools/milestones/8/labels","id":1849131,"number":8,"title":"next-release","description":"An issue tagged with the next-release milestone may or may not be critical enough to mandate resolution prior to the next minor release. However, all next-release milestones should at least be reviewed and evaluated prior to any official release. ","creator":{"login":"aphearin","id":6951595,"avatar_url":"https://avatars.githubusercontent.com/u/6951595?v=3","gravatar_id":"","url":"https://api.github.com/users/aphearin","html_url":"https://github.com/aphearin","followers_url":"https://api.github.com/users/aphearin/followers","following_url":"https://api.github.com/users/aphearin/following{/other_user}","gists_url":"https://api.github.com/users/aphearin/gists{/gist_id}","starred_url":"https://api.github.com/users/aphearin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aphearin/subscriptions","organizations_url":"https://api.github.com/users/aphearin/orgs","repos_url":"https://api.github.com/users/aphearin/repos","events_url":"https://api.github.com/users/aphearin/events{/privacy}","received_events_url":"https://api.github.com/users/aphearin/received_events","type":"User","site_admin":false},"open_issues":12,"closed_issues":1,"state":"open","created_at":"2016-06-26T21:18:27Z","updated_at":"2016-07-25T15:31:13Z","due_on":null,"closed_at":null},"comments":13,"created_at":"2016-06-27T15:34:00Z","updated_at":"2016-07-25T22:31:16Z","closed_at":null,"body":"The following code which continuously repopulates the same halo catalog with galaxies uses a lot of memory. Most importantly, the memory usage increases even after the first hundred iterations and only reaches a maximum after around 300 iterations. The amount of memory used after 300 iterations is roughly 3 times the memory used after one iteration. It's not obvious why halotools should at all increase in memory usage after each iteration.\r\n\r\nObviously, re-populating the same halo catalog over and over again is important for MCMC and there one commonly has a lot of memory, anyway. However, re-populating the same halo catalog is also important when plotting results of an MCMC chain. It might be desirable that this could be done on a computer with less than 16GB of RAM. It would be great if the memory usage of halotools could be reduced in such scenarios.\r\n\r\n```\r\nfrom halotools.empirical_models import PrebuiltHodModelFactory\r\nfrom halotools.sim_manager import CachedHaloCatalog\r\n\r\nhalocat = CachedHaloCatalog(simname ='bolplanck', redshift=0)\r\nmodel = PrebuiltHodModelFactory('zheng07')\r\nmodel.param_dict['sigma_logM'] = 1.0\r\n\r\nfor i in range(1000):\r\n  try:\r\n    model.mock.populate()\r\n  except:\r\n    model.populate_mock(halocat)\r\n```"},"comment":{"url":"https://api.github.com/repos/astropy/halotools/issues/comments/235106392","html_url":"https://github.com/astropy/halotools/issues/568#issuecomment-235106392","issue_url":"https://api.github.com/repos/astropy/halotools/issues/568","id":235106392,"user":{"login":"aphearin","id":6951595,"avatar_url":"https://avatars.githubusercontent.com/u/6951595?v=3","gravatar_id":"","url":"https://api.github.com/users/aphearin","html_url":"https://github.com/aphearin","followers_url":"https://api.github.com/users/aphearin/followers","following_url":"https://api.github.com/users/aphearin/following{/other_user}","gists_url":"https://api.github.com/users/aphearin/gists{/gist_id}","starred_url":"https://api.github.com/users/aphearin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aphearin/subscriptions","organizations_url":"https://api.github.com/users/aphearin/orgs","repos_url":"https://api.github.com/users/aphearin/repos","events_url":"https://api.github.com/users/aphearin/events{/privacy}","received_events_url":"https://api.github.com/users/aphearin/received_events","type":"User","site_admin":false},"created_at":"2016-07-25T22:31:16Z","updated_at":"2016-07-25T22:31:16Z","body":"It would be nice to limit the proliferation of keyword arguments. Maybe the best option is to simply add a `Notes` section to the docstring of mock.populate notifying users that they can call gc.collect() after calling mock.populate() if they need to limit memory usage, since this particular example happens to be so effective. "}}
{"actor":{"display_login":"aphearin"}}