{"action":"created","issue":{"url":"https://api.github.com/repos/1lobby/mailgun-js/issues/90","repository_url":"https://api.github.com/repos/1lobby/mailgun-js","labels_url":"https://api.github.com/repos/1lobby/mailgun-js/issues/90/labels{/name}","comments_url":"https://api.github.com/repos/1lobby/mailgun-js/issues/90/comments","events_url":"https://api.github.com/repos/1lobby/mailgun-js/issues/90/events","html_url":"https://github.com/1lobby/mailgun-js/issues/90","id":137575812,"number":90,"title":"Batch sending stack overflow","user":{"login":"bevacqua","id":934293,"avatar_url":"https://avatars.githubusercontent.com/u/934293?v=3","gravatar_id":"","url":"https://api.github.com/users/bevacqua","html_url":"https://github.com/bevacqua","followers_url":"https://api.github.com/users/bevacqua/followers","following_url":"https://api.github.com/users/bevacqua/following{/other_user}","gists_url":"https://api.github.com/users/bevacqua/gists{/gist_id}","starred_url":"https://api.github.com/users/bevacqua/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bevacqua/subscriptions","organizations_url":"https://api.github.com/users/bevacqua/orgs","repos_url":"https://api.github.com/users/bevacqua/repos","events_url":"https://api.github.com/users/bevacqua/events{/privacy}","received_events_url":"https://api.github.com/users/bevacqua/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-03-01T13:34:44Z","updated_at":"2016-03-02T00:18:47Z","closed_at":null,"body":"Attempting to send 1000 emails in a single batch fails with a stack overflow. The stack is quite cryptic.\r\n\r\n```shell\r\npostBatch 1000\r\n27:10 - error: Uncaught exception stack=RangeError: Maximum call stack size exceeded\r\n    at ClientRequest.OutgoingMessage._writeRaw (_http_outgoing.js:143:47)\r\n    at ClientRequest.OutgoingMessage._send (_http_outgoing.js:139:15)\r\n    at ClientRequest.OutgoingMessage.write (_http_outgoing.js:464:16)\r\n    at FormData.ondata (stream.js:31:26)\r\n    at emitOne (events.js:77:13)\r\n    at FormData.emit (events.js:169:7)\r\n    at FormData.CombinedStream.write (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:118:8)\r\n    at FormData.CombinedStream._pipeNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:106:8)\r\n    at FormData.<anonymous> (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:91:10)\r\n    at FormData.<anonymous> (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/lib/form_data.js:201:5)\r\n    at FormData.CombinedStream._getNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:84:3)\r\n    at FormData.CombinedStream._pipeNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:107:8)\r\n    at FormData.CombinedStream._getNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:79:10)\r\n    at FormData.CombinedStream._pipeNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:107:8)\r\n    at FormData.CombinedStream._getNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:79:10)\r\n    at FormData.CombinedStream._pipeNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:107:8)\r\n    at FormData.<anonymous> (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:91:10)\r\n    at FormData.<anonymous> (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/lib/form_data.js:201:5)\r\n    at FormData.CombinedStream._getNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:84:3)\r\n    at FormData.CombinedStream._pipeNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:107:8)\r\n    at FormData.CombinedStream._getNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:79:10)\r\n    at FormData.CombinedStream._pipeNext (~/campaign-mailgun/node_modules/mailgun-js/node_modules/form-data/node_modules/combined-stream/lib/combined_stream.js:107:8)\r\npostBatch 431\r\n1 Mar 10:27:10 - [nodemon] app crashed - waiting for file changes before starting...\r\n```\r\n\r\nExact same configuration except I sent emails in batches of 100:\r\n\r\n```shell\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 100\r\npostBatch 31\r\n```\r\n\r\nEach `postBatch` line in the logs is a call to `client.messages().send`.\r\n\r\nI haven't tried other batch sizes because these were production recipients. I use merge variables. I also have an inline image which I suspect may be the culprit due to the nature of the stack trace. Is there any \"generous\" looping going on that would end up in a stack overflow with relatively small batches (1000) but not on even smaller batches (100)?\r\n\r\nReally wish I could give more information than this."},"comment":{"url":"https://api.github.com/repos/1lobby/mailgun-js/issues/comments/190984862","html_url":"https://github.com/1lobby/mailgun-js/issues/90#issuecomment-190984862","issue_url":"https://api.github.com/repos/1lobby/mailgun-js/issues/90","id":190984862,"user":{"login":"bojand","id":2081100,"avatar_url":"https://avatars.githubusercontent.com/u/2081100?v=3","gravatar_id":"","url":"https://api.github.com/users/bojand","html_url":"https://github.com/bojand","followers_url":"https://api.github.com/users/bojand/followers","following_url":"https://api.github.com/users/bojand/following{/other_user}","gists_url":"https://api.github.com/users/bojand/gists{/gist_id}","starred_url":"https://api.github.com/users/bojand/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bojand/subscriptions","organizations_url":"https://api.github.com/users/bojand/orgs","repos_url":"https://api.github.com/users/bojand/repos","events_url":"https://api.github.com/users/bojand/events{/privacy}","received_events_url":"https://api.github.com/users/bojand/received_events","type":"User","site_admin":false},"created_at":"2016-03-02T00:18:47Z","updated_at":"2016-03-02T00:18:47Z","body":"May not be related but one thing of interest is your setting of `knownLength` is not guaranteed, and unlikely, to be incorrect considering we are dealing with image / base64 data. I believe `Buffer.length` or `Buffer.byteLength()` should be used to get the true byte size of the data.\r\n\r\n```\r\nvar file = path.join(__dirname, 'mailgun_logo.png');\r\n\r\nfunction filename (file) {\r\n  var basename = path.basename(file);\r\n  var lio = basename.lastIndexOf('.');\r\n  return lio === -1 ? basename : basename.substr(0, lio);\r\n}\r\n\r\nvar imagedata = fs.readFileSync(file);\r\n\r\nvar image = {\r\n  data: new Buffer(imagedata).toString('base64'),\r\n  mime: mime.lookup(file),\r\n  name: filename(file)\r\n};\r\n\r\nvar imageBuffer = new Buffer(image.data, 'base64');\r\n\r\nconsole.log('buffer length: %s', imageBuffer.length);\r\nconsole.log('buffer byte length: %s', Buffer.byteLength(image.data, 'base64'));\r\nconsole.log('string length: %s', image.data.length);\r\n```\r\n\r\n```\r\nbuffer length: 16298\r\nbuffer byte length: 16298\r\nstring length: 21732\r\n```\r\n\r\nThe `Attachment` creation should be setting the byte length, so something like:\r\n\r\n```\r\nfunction transform (image) {\r\n        var buff = new Buffer(image.data, 'base64')\r\n        return new client.Attachment({\r\n          data: buff,\r\n          filename: image.name,\r\n          contentType: image.mime,\r\n          knownLength: buff.length\r\n        });\r\n      }\r\n```"}}