{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7811","repository_url":"https://api.github.com/repos/scikit-learn/scikit-learn","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7811/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7811/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7811/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/7811","id":186816030,"number":7811,"title":"mae DecisionTree does not free used memory after refit","user":{"login":"mbochk","id":15201622,"avatar_url":"https://avatars.githubusercontent.com/u/15201622?v=3","gravatar_id":"","url":"https://api.github.com/users/mbochk","html_url":"https://github.com/mbochk","followers_url":"https://api.github.com/users/mbochk/followers","following_url":"https://api.github.com/users/mbochk/following{/other_user}","gists_url":"https://api.github.com/users/mbochk/gists{/gist_id}","starred_url":"https://api.github.com/users/mbochk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbochk/subscriptions","organizations_url":"https://api.github.com/users/mbochk/orgs","repos_url":"https://api.github.com/users/mbochk/repos","events_url":"https://api.github.com/users/mbochk/events{/privacy}","received_events_url":"https://api.github.com/users/mbochk/received_events","type":"User","site_admin":false},"labels":[{"id":12995755,"url":"https://api.github.com/repos/scikit-learn/scikit-learn/labels/Bug","name":"Bug","color":"e10c02","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/milestones/21","html_url":"https://github.com/scikit-learn/scikit-learn/milestones/0.19","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/milestones/21/labels","id":1970916,"number":21,"title":"0.19","description":"","creator":{"login":"amueller","id":449558,"avatar_url":"https://avatars.githubusercontent.com/u/449558?v=3","gravatar_id":"","url":"https://api.github.com/users/amueller","html_url":"https://github.com/amueller","followers_url":"https://api.github.com/users/amueller/followers","following_url":"https://api.github.com/users/amueller/following{/other_user}","gists_url":"https://api.github.com/users/amueller/gists{/gist_id}","starred_url":"https://api.github.com/users/amueller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amueller/subscriptions","organizations_url":"https://api.github.com/users/amueller/orgs","repos_url":"https://api.github.com/users/amueller/repos","events_url":"https://api.github.com/users/amueller/events{/privacy}","received_events_url":"https://api.github.com/users/amueller/received_events","type":"User","site_admin":false},"open_issues":179,"closed_issues":35,"state":"open","created_at":"2016-08-29T22:34:33Z","updated_at":"2016-11-23T09:03:08Z","due_on":"2016-12-31T00:00:00Z","closed_at":null},"comments":10,"created_at":"2016-11-02T14:28:58Z","updated_at":"2016-11-28T08:44:17Z","closed_at":null,"body":"<!--\r\nIf your issue is a usage question, submit it here instead:\r\n- StackOverflow with the scikit-learn tag: http://stackoverflow.com/questions/tagged/scikit-learn\r\n- Mailing List: https://mail.python.org/mailman/listinfo/scikit-learn\r\nFor more information, see User Questions: http://scikit-learn.org/stable/support.html#user-questions\r\n-->\r\n\r\n<!-- Instructions For Filing a Bug: https://github.com/scikit-learn/scikit-learn/blob/master/CONTRIBUTING.md#filing-bugs -->\r\n\r\n#### Description\r\nDecisionTree with mae criterion does not free used memory.\r\nAs a result that kind of trees is unusable in cycle, forests and cross-validation.\r\n(originally i encoutered this problem while trying use grid search with mae forest)\r\n\r\n#### Steps/Code to Reproduce\r\n\r\n```\r\nimport sklearn\r\n\r\nparams = dict(max_depth=range(1, 10))\r\nparams = sklearn.model_selection.ParameterGrid(params)\r\nreg = sklearn.tree.DecisionTreeRegressor(criterion='mae')\r\n\r\n# repeated several times in jupyter cell\r\nfor p in params:\r\n    print p\r\n    for i in range(100):\r\n        reg.set_params(**p)\r\n        reg.fit(xxx, yyy)\r\n```\r\n\r\n#### Expected Results\r\nNo additional memory usage after first completion of cycle.\r\nCell with 'for' cycle is runnable unlimited times.\r\n\r\nReplacing 'mae' with 'mse' does give expected result.\r\n\r\n#### Actual Results\r\nAdditional memory is used after each cell execution.\r\nDeleteing tree with 'del' and enforcing 'gc.collect()' does not help.\r\nWhen there is no memory left OS (windows) shuts down python process.\r\n\r\nMy guess is that there is memory leak in cython.\r\n\r\n#### Versions\r\n<!--\r\nPlease run the following snippet and paste the output below.\r\nimport platform; print(platform.platform())\r\nimport sys; print(\"Python\", sys.version)\r\nimport numpy; print(\"NumPy\", numpy.__version__)\r\nimport scipy; print(\"SciPy\", scipy.__version__)\r\nimport sklearn; print(\"Scikit-Learn\", sklearn.__version__)\r\n-->\r\n\r\nWindows-7-6.1.7601-SP1\r\n('Python', '2.7.12 | packaged by conda-forge | (default, Aug 12 2016, 18:53:38) [MSC v.1500 64 bit (AMD64)]')\r\n('NumPy', '1.11.2')\r\n('SciPy', '0.18.0')\r\n('Scikit-Learn', '0.18')\r\n\r\n<!-- Thanks for contributing! -->\r\n"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/263213376","html_url":"https://github.com/scikit-learn/scikit-learn/issues/7811#issuecomment-263213376","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/7811","id":263213376,"user":{"login":"nelson-liu","id":7272031,"avatar_url":"https://avatars.githubusercontent.com/u/7272031?v=3","gravatar_id":"","url":"https://api.github.com/users/nelson-liu","html_url":"https://github.com/nelson-liu","followers_url":"https://api.github.com/users/nelson-liu/followers","following_url":"https://api.github.com/users/nelson-liu/following{/other_user}","gists_url":"https://api.github.com/users/nelson-liu/gists{/gist_id}","starred_url":"https://api.github.com/users/nelson-liu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nelson-liu/subscriptions","organizations_url":"https://api.github.com/users/nelson-liu/orgs","repos_url":"https://api.github.com/users/nelson-liu/repos","events_url":"https://api.github.com/users/nelson-liu/events{/privacy}","received_events_url":"https://api.github.com/users/nelson-liu/received_events","type":"User","site_admin":false},"created_at":"2016-11-28T08:44:17Z","updated_at":"2016-11-28T08:44:17Z","body":"if by investigate you mean debug, I think it's [possible to use valgrind](https://github.com/cython/cython/wiki/UsingValgrindToDebug). I'm not super familiar with debugging memory issues in Cython, though; I think it's fairly intuitive where the leak is happening here, though (the numpy array of Cython objects is improperly freed)..."}}
{"actor":{"display_login":"nelson-liu"}}