{"action":"created","issue":{"url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/6839","repository_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j","labels_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/6839/labels{/name}","comments_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/6839/comments","events_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/6839/events","html_url":"https://github.com/deeplearning4j/deeplearning4j/issues/6839","id":389984122,"node_id":"MDU6SXNzdWUzODk5ODQxMjI=","number":6839,"title":"Implement INDArray.close() semantics.","user":{"login":"stolsvik","id":3123344,"node_id":"MDQ6VXNlcjMxMjMzNDQ=","avatar_url":"https://avatars2.githubusercontent.com/u/3123344?v=4","gravatar_id":"","url":"https://api.github.com/users/stolsvik","html_url":"https://github.com/stolsvik","followers_url":"https://api.github.com/users/stolsvik/followers","following_url":"https://api.github.com/users/stolsvik/following{/other_user}","gists_url":"https://api.github.com/users/stolsvik/gists{/gist_id}","starred_url":"https://api.github.com/users/stolsvik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stolsvik/subscriptions","organizations_url":"https://api.github.com/users/stolsvik/orgs","repos_url":"https://api.github.com/users/stolsvik/repos","events_url":"https://api.github.com/users/stolsvik/events{/privacy}","received_events_url":"https://api.github.com/users/stolsvik/received_events","type":"User","site_admin":false},"labels":[{"id":67900999,"node_id":"MDU6TGFiZWw2NzkwMDk5OQ==","url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/labels/Bug","name":"Bug","color":"d93f0b","default":false},{"id":934492059,"node_id":"MDU6TGFiZWw5MzQ0OTIwNTk=","url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/labels/ND4J","name":"ND4J","color":"c7cdfc","default":false}],"state":"closed","locked":false,"assignee":{"login":"raver119","id":12250879,"node_id":"MDQ6VXNlcjEyMjUwODc5","avatar_url":"https://avatars2.githubusercontent.com/u/12250879?v=4","gravatar_id":"","url":"https://api.github.com/users/raver119","html_url":"https://github.com/raver119","followers_url":"https://api.github.com/users/raver119/followers","following_url":"https://api.github.com/users/raver119/following{/other_user}","gists_url":"https://api.github.com/users/raver119/gists{/gist_id}","starred_url":"https://api.github.com/users/raver119/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/raver119/subscriptions","organizations_url":"https://api.github.com/users/raver119/orgs","repos_url":"https://api.github.com/users/raver119/repos","events_url":"https://api.github.com/users/raver119/events{/privacy}","received_events_url":"https://api.github.com/users/raver119/received_events","type":"User","site_admin":false},"assignees":[{"login":"raver119","id":12250879,"node_id":"MDQ6VXNlcjEyMjUwODc5","avatar_url":"https://avatars2.githubusercontent.com/u/12250879?v=4","gravatar_id":"","url":"https://api.github.com/users/raver119","html_url":"https://github.com/raver119","followers_url":"https://api.github.com/users/raver119/followers","following_url":"https://api.github.com/users/raver119/following{/other_user}","gists_url":"https://api.github.com/users/raver119/gists{/gist_id}","starred_url":"https://api.github.com/users/raver119/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/raver119/subscriptions","organizations_url":"https://api.github.com/users/raver119/orgs","repos_url":"https://api.github.com/users/raver119/repos","events_url":"https://api.github.com/users/raver119/events{/privacy}","received_events_url":"https://api.github.com/users/raver119/received_events","type":"User","site_admin":false}],"milestone":null,"comments":42,"created_at":"2018-12-11T22:36:11Z","updated_at":"2018-12-19T06:52:55Z","closed_at":"2018-12-19T06:52:55Z","author_association":"CONTRIBUTOR","body":"_{This was originally an issue about AutoGcWindow being set to 100 ms, but it \"derailed\" into a discussion of implementing close()-semantics on INDArrays. I moved the original issue to https://github.com/deeplearning4j/deeplearning4j/issues/6846, so that it can be closed without closing the discussion about close() semantics.}_\r\n\r\nOriginal issue was as follows, but the discussion ended up with INDArray.close() semantics.\r\n\r\n> Given that no-one should _ever_ rely on GC for doing cleanup of external resources (as per every single explanation of resource handling in every manual ever written), and that most of Dl4j is now using explicit cleanup via workspaces, the default AutoGcWindow setting should be looked into.\r\n>\r\n>I was _really_ hit by the GC-logic of dl4j when trying out something I hadn't done before: Restoring a previously stored network and inferring on it..! The loading took much longer time than saving, but worse, the inferring of some hundreds-of-thousands datapoints, which while training took literally seconds, took _hours_ on this restored network.\r\n>\r\n>After intense debugging and \"triangulating\" between different setups, barking up every tree in the forest, whipping out the debugger and profiler, I suddenly looked directly at that _one_ single line which I had in the training code, which I hadn't copied over to the restore-and-infer code:\r\n`Nd4j.getMemoryManager().setAutoGcWindow(600_000); // 600k = Every 10 minutes`\r\n>\r\n>It suddenly hit me that I vaguely remember that one should change this setting if using Workspaces. And I then copied it over. And that made all the difference. When looking into the code, I see that it is set to 100 ms by default (!!), both on the BasicMemoryManager and the CudaMemoryManager. When using a heap of 20GB+, doing GC every 100 ms kinda explains why this was .. not working.\r\n>\r\n>This default seems pretty strange. Unless there is something extraordinary with my setup, this is what will hit most people unless they dig pretty deep, and I can just see the contours of how many people will wrongfully experience dl4j as a really slow system."},"comment":{"url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/comments/448488868","html_url":"https://github.com/deeplearning4j/deeplearning4j/issues/6839#issuecomment-448488868","issue_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/6839","id":448488868,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0ODQ4ODg2OA==","user":{"login":"raver119","id":12250879,"node_id":"MDQ6VXNlcjEyMjUwODc5","avatar_url":"https://avatars2.githubusercontent.com/u/12250879?v=4","gravatar_id":"","url":"https://api.github.com/users/raver119","html_url":"https://github.com/raver119","followers_url":"https://api.github.com/users/raver119/followers","following_url":"https://api.github.com/users/raver119/following{/other_user}","gists_url":"https://api.github.com/users/raver119/gists{/gist_id}","starred_url":"https://api.github.com/users/raver119/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/raver119/subscriptions","organizations_url":"https://api.github.com/users/raver119/orgs","repos_url":"https://api.github.com/users/raver119/repos","events_url":"https://api.github.com/users/raver119/events{/privacy}","received_events_url":"https://api.github.com/users/raver119/received_events","type":"User","site_admin":false},"created_at":"2018-12-19T06:52:55Z","updated_at":"2018-12-19T06:52:55Z","author_association":"CONTRIBUTOR","body":"close() and closeable() were implemented for INDArray and DataBuffer"}}
{"actor":{"display_login":"raver119"}}