{"action":"created","issue":{"url":"https://api.github.com/repos/senecajs/seneca/issues/401","repository_url":"https://api.github.com/repos/senecajs/seneca","labels_url":"https://api.github.com/repos/senecajs/seneca/issues/401/labels{/name}","comments_url":"https://api.github.com/repos/senecajs/seneca/issues/401/comments","events_url":"https://api.github.com/repos/senecajs/seneca/issues/401/events","html_url":"https://github.com/senecajs/seneca/issues/401","id":149577225,"number":401,"title":"leak from plugin when using toString on large data","user":{"login":"psnider","id":940931,"avatar_url":"https://avatars.githubusercontent.com/u/940931?v=3","gravatar_id":"","url":"https://api.github.com/users/psnider","html_url":"https://github.com/psnider","followers_url":"https://api.github.com/users/psnider/followers","following_url":"https://api.github.com/users/psnider/following{/other_user}","gists_url":"https://api.github.com/users/psnider/gists{/gist_id}","starred_url":"https://api.github.com/users/psnider/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/psnider/subscriptions","organizations_url":"https://api.github.com/users/psnider/orgs","repos_url":"https://api.github.com/users/psnider/repos","events_url":"https://api.github.com/users/psnider/events{/privacy}","received_events_url":"https://api.github.com/users/psnider/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/senecajs/seneca/labels/investigate","name":"investigate","color":"f7c6c7"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-04-19T20:37:20Z","updated_at":"2016-04-20T02:22:32Z","closed_at":null,"body":"I've encountered a severe memory leak when an external service started sending very large XML responses (~40MB). \r\n\r\n_Note that there are some bug reports in node with very large strings leaking, but they must be around 250MB, and seem to be unrelated to this issue. Also , I couldn't reproduce this problem when I don't use seneca._\r\n\r\nnode v5.0.0\r\nseneca ^1.3.0\r\n\r\nHere is minimal code I use from a REPL that reproduces the issue reliably.\r\nEnable garbage collection API to help with testing:\r\n\r\n**node  --expose-gc**\r\n\r\n```\r\n// if true, toString() is called within the plugin code, if false, it is called within the triggering act callback\r\nvar perform_toString_within_plugin\r\n\r\nfunction generateLargeBuffer() {\r\n    const buf = new Buffer(43632013)\r\n    buf.fill(0)\r\n    return buf\r\n}\r\n\r\n/////// SIMPLE PLUGLIN\r\nvar fs = require('fs')\r\nfunction leaking_plugin( options ) {\r\n    const PLUGIN_NAME = 'leaking'\r\n    this.add('role:test,cmd:leak', (msg, respond) => {\r\n        var data = generateLargeBuffer()\r\n        if (perform_toString_within_plugin) {\r\n            var contents = data.toString()\r\n            console.log('toString() in leaking_plugin')\r\n        } else {\r\n            var contents = data\r\n        }\r\n        respond(null, {contents})\r\n    })\r\n    this.add( {init: PLUGIN_NAME}, function( args, done ) {\r\n        done()\r\n    })\r\n    return PLUGIN_NAME\r\n}\r\n\r\n\r\nfunction senecaAction(msg, done) {\r\n    seneca.act(msg, (error, response) => {\r\n        if (!error) {\r\n            if (perform_toString_within_plugin) {\r\n                var s = response.contents\r\n            } else {\r\n                s = response.contents.toString()\r\n                console.log('toString() in senecaAction()')\r\n            }\r\n            console.log('OK to process data')\r\n        } else {\r\n            console.log('error=' + error);\r\n        }\r\n        done(error, s)\r\n    })\r\n}\r\n\r\n\r\nfs = require('fs')\r\nSENECA = require('seneca')\r\nvar seneca = SENECA({\r\n    \"log\": \"silent\",\r\n    \"default_plugins\": {\r\n        \"mem-store\": true\r\n    },\r\n    \"debug\": {\r\n        \"undead\": true\r\n    },\r\n    \"timeout\": 99999\r\n})\r\nseneca.use(leaking_plugin)\r\nseneca.error((error) => {\r\n    console.log('Seneca error: ' + error)\r\n})\r\n\r\nfunction test() {\r\n    senecaAction({role: 'test',cmd: 'leak'}, function(error, s) {\r\n        console.log('test finished...')\r\n    })\r\n}\r\n\r\n```\r\n\r\nFind the base heap size:\r\n```\r\nglobal.gc()\r\nprocess.memoryUsage()\r\n```\r\nFor me this gives: { rss: 52281344, heapTotal: 32966400, heapUsed: **26,996,888** }\r\n\r\nThen excercise the above code from the REPL with combinations of tests and measurements and running the GC.\r\n```\r\nperform_toString_within_plugin = false\r\ntest()\r\ntest()\r\ntest()\r\ntest()\r\n```\r\nWait for the tests to finish, then run:\r\n```\r\nprocess.memoryUsage()\r\n```\r\nFor me this gives: { rss: 293875712, heapTotal: 152241920, heapUsed: 115,433,200 }\r\n\r\nVerify that this memory can be reclaimed by running the GC:\r\n```\r\nglobal.gc()\r\nprocess.memoryUsage()\r\n```\r\nFor me this gives: { rss: 182329344, heapTotal: 31946496, heapUsed: **26,413,976** }\r\nSo it appears that memory is reclaimed correctly.\r\n\r\n*Now switch to performing the toString() from within the plugin*:\r\n```\r\nperform_toString_within_plugin = true\r\ntest()\r\ntest()\r\ntest()\r\ntest()\r\n```\r\nAgain, wait for the tests to finish, then run:\r\n```\r\nprocess.memoryUsage()\r\n```\r\nFor me this gives: { rss: 527720448, heapTotal: 206558208, heapUsed: 198,439,760 }\r\n\r\nNow confirm that the GC cannot reclaim this memory any longer.\r\n```\r\nglobal.gc()\r\nprocess.memoryUsage()\r\n```\r\nFor me this gives: { rss: 523902976, heapTotal: 204523008, heapUsed: **198,208,656** }\r\n\r\n*In contrast to before, it has dropped only by about 200k !*"},"comment":{"url":"https://api.github.com/repos/senecajs/seneca/issues/comments/212214834","html_url":"https://github.com/senecajs/seneca/issues/401#issuecomment-212214834","issue_url":"https://api.github.com/repos/senecajs/seneca/issues/401","id":212214834,"user":{"login":"psnider","id":940931,"avatar_url":"https://avatars.githubusercontent.com/u/940931?v=3","gravatar_id":"","url":"https://api.github.com/users/psnider","html_url":"https://github.com/psnider","followers_url":"https://api.github.com/users/psnider/followers","following_url":"https://api.github.com/users/psnider/following{/other_user}","gists_url":"https://api.github.com/users/psnider/gists{/gist_id}","starred_url":"https://api.github.com/users/psnider/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/psnider/subscriptions","organizations_url":"https://api.github.com/users/psnider/orgs","repos_url":"https://api.github.com/users/psnider/repos","events_url":"https://api.github.com/users/psnider/events{/privacy}","received_events_url":"https://api.github.com/users/psnider/received_events","type":"User","site_admin":false},"created_at":"2016-04-20T02:22:32Z","updated_at":"2016-04-20T02:22:32Z","body":"FWIW:\r\n- [Fatal Error (Abort) creating string larger than 268435440 bytes](https://github.com/nodejs/node/issues/1374)\r\n- [src: fix memory leak in ExternString](https://github.com/nodejs/node/pull/2402)"}}