{"action":"created","issue":{"url":"https://api.github.com/repos/aws/aws-sdk-js/issues/883","labels_url":"https://api.github.com/repos/aws/aws-sdk-js/issues/883/labels{/name}","comments_url":"https://api.github.com/repos/aws/aws-sdk-js/issues/883/comments","events_url":"https://api.github.com/repos/aws/aws-sdk-js/issues/883/events","html_url":"https://github.com/aws/aws-sdk-js/issues/883","id":127896173,"number":883,"title":"Uploading multiple files to S3 gets \"Killed\"","user":{"login":"ruandre","id":5968651,"avatar_url":"https://avatars.githubusercontent.com/u/5968651?v=3","gravatar_id":"","url":"https://api.github.com/users/ruandre","html_url":"https://github.com/ruandre","followers_url":"https://api.github.com/users/ruandre/followers","following_url":"https://api.github.com/users/ruandre/following{/other_user}","gists_url":"https://api.github.com/users/ruandre/gists{/gist_id}","starred_url":"https://api.github.com/users/ruandre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ruandre/subscriptions","organizations_url":"https://api.github.com/users/ruandre/orgs","repos_url":"https://api.github.com/users/ruandre/repos","events_url":"https://api.github.com/users/ruandre/events{/privacy}","received_events_url":"https://api.github.com/users/ruandre/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-01-21T10:45:03Z","updated_at":"2016-01-21T16:59:13Z","closed_at":null,"body":"When I try to upload a large number of files it fails and returns \"Killed\" (uploading fewer files work fine).\r\n\r\n```js\r\nfor ( var i = files.length - 1; i >= 0; i-- ) {\r\n\r\n  var file     = files[ i ];\r\n  var template = fs.readFileSync( file, 'utf8' );\r\n  var inject   = handlebars.compile( template );\r\n  var html     = inject( obj ); // add data\r\n\r\n  console.log( 'Processing '+ file +' for '+ key +' ...' );\r\n\r\n  var params = {\r\n    Bucket : bucket,\r\n    Key    : 'generate/' + key + '/index.html',\r\n    Body   : html\r\n  };\r\n\r\n  s3.upload( params, function( err, data ) {\r\n    if ( err ) console.log( err, err.stack );\r\n    else       console.log( 'success' );\r\n  });\r\n\r\n};\r\n```\r\n\r\nWith a few files I get:\r\n```\r\nProcessing src/html/index.html for 1234567890 ...\r\nProcessing src/html/example.html for 1234567890 ...\r\nProcessing src/html/index.html for 1234567891 ...\r\nProcessing src/html/example.html for 1234567891 ...\r\nsuccess\r\nsuccess\r\nsuccess\r\nsuccess\r\n```\r\n\r\nI should be getting something like this, which leads me to believe it's an async issue.\r\n```\r\nProcessing src/html/index.html for 1234567890 ...\r\nsuccess\r\nProcessing src/html/example.html for 1234567890 ...\r\nsuccess\r\nProcessing src/html/index.html for 1234567891 ...\r\nsuccess\r\nProcessing src/html/example.html for 1234567891 ...\r\nsuccess\r\n```\r\n\r\nWith many files I get:\r\n```\r\n...\r\nProcessing src/html/index.html for 99994565 ...\r\nProcessing src/html/example.html for 99994565 ...\r\nProcessing src/html/index.html for 99997328 ...\r\nProcessing src/html/example.html for 99997328 ...\r\nKilled\r\n```\r\n\r\nThe loop should wait for the current file to finish uploading before proceeding to the next.\r\nIs this caused by a bug in AWS SDK or do I need to change something in my code?"},"comment":{"url":"https://api.github.com/repos/aws/aws-sdk-js/issues/comments/173634450","html_url":"https://github.com/aws/aws-sdk-js/issues/883#issuecomment-173634450","issue_url":"https://api.github.com/repos/aws/aws-sdk-js/issues/883","id":173634450,"user":{"login":"chrisradek","id":14189820,"avatar_url":"https://avatars.githubusercontent.com/u/14189820?v=3","gravatar_id":"","url":"https://api.github.com/users/chrisradek","html_url":"https://github.com/chrisradek","followers_url":"https://api.github.com/users/chrisradek/followers","following_url":"https://api.github.com/users/chrisradek/following{/other_user}","gists_url":"https://api.github.com/users/chrisradek/gists{/gist_id}","starred_url":"https://api.github.com/users/chrisradek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chrisradek/subscriptions","organizations_url":"https://api.github.com/users/chrisradek/orgs","repos_url":"https://api.github.com/users/chrisradek/repos","events_url":"https://api.github.com/users/chrisradek/events{/privacy}","received_events_url":"https://api.github.com/users/chrisradek/received_events","type":"User","site_admin":false},"created_at":"2016-01-21T16:59:13Z","updated_at":"2016-01-21T16:59:13Z","body":"@ruandre \r\nThe SDK will perform it's service operations asynchronously, so you're actually kicking off as many uploads as you have files at once. \r\n\r\nIf you want each upload to occur one after another, you can start the next upload in the callback you pass into the upload function. The `async` module can likely help you accomplish this as well.\r\n\r\nLet me know if this helps!"}}