{"action":"created","issue":{"url":"https://api.github.com/repos/mscdex/ssh2/issues/459","repository_url":"https://api.github.com/repos/mscdex/ssh2","labels_url":"https://api.github.com/repos/mscdex/ssh2/issues/459/labels{/name}","comments_url":"https://api.github.com/repos/mscdex/ssh2/issues/459/comments","events_url":"https://api.github.com/repos/mscdex/ssh2/issues/459/events","html_url":"https://github.com/mscdex/ssh2/issues/459","id":171902318,"number":459,"title":"SFTP: low speed in uploading","user":{"login":"eidosam","id":17527017,"avatar_url":"https://avatars.githubusercontent.com/u/17527017?v=3","gravatar_id":"","url":"https://api.github.com/users/eidosam","html_url":"https://github.com/eidosam","followers_url":"https://api.github.com/users/eidosam/followers","following_url":"https://api.github.com/users/eidosam/following{/other_user}","gists_url":"https://api.github.com/users/eidosam/gists{/gist_id}","starred_url":"https://api.github.com/users/eidosam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eidosam/subscriptions","organizations_url":"https://api.github.com/users/eidosam/orgs","repos_url":"https://api.github.com/users/eidosam/repos","events_url":"https://api.github.com/users/eidosam/events{/privacy}","received_events_url":"https://api.github.com/users/eidosam/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-08-18T13:34:51Z","updated_at":"2016-08-18T15:00:43Z","closed_at":null,"body":"Hi,\r\n\r\nI'm trying to use **ssh2** replacing the current used method (spawn'ing sftp command)\r\nIt is 10times slower!\r\nThe files used in this case 450MB *3 + 150MB *1\r\nUploading in parallel limit:4\r\n\r\n```js\r\nssh2 = require('ssh2').Client;\r\n/*\r\nfile: instance of Google Cloud Storage File \r\nhttps://googlecloudplatform.github.io/gcloud-node/#/docs/google-cloud/0.38.3/storage/file\r\n*/\r\nuploadFile = function(file, opts, cb) {\r\n  var ssh2Conn, gcsReadStream, fileName;\r\n  ssh2Conn = new ssh2();\r\n  fileName = _.last(file.name.split('/'));\r\n  gcsReadStream = file.createReadStream();\r\n  gcsReadStream.on('error', function(e) {\r\n    cb(e);\r\n  });\r\n  ssh2Conn.connect(opts.connectionCfg);\r\n  ssh2Conn.on('ready', function() {\r\n    console.log(\"Connection with SFTP server \" + opts.connectionCfg.host + \" is established\");\r\n    ssh2Conn.sftp(function(err, sftp) {\r\n      var pathOfDestinationFile, sftpWriteStream;\r\n      if (err) {\r\n        cb(err);\r\n      } else {\r\n        pathOfDestinationFile = opts.path + \"/\" + fileName;\r\n        sftpWriteStream = sftp.createWriteStream(pathOfDestinationFile);\r\n        sftpWriteStream.on('open', function() {\r\n          console.log(\"File \" + sftpWriteStream.path + \" uploading started\");\r\n        });\r\n        sftpWriteStream.on('finish', function() {\r\n          console.log(\"File \" + sftpWriteStream.path + \" uploaded successfully\");\r\n          sftpWriteStream.end();\r\n          sftp.end();\r\n          ssh2Conn.end();\r\n        });\r\n        sftpWriteStream.on('error', function(err) {\r\n          console.log(\"Error at \" + sftpWriteStream.path + \", \" + err);\r\n          cb(err);\r\n        });\r\n        gcsReadStream.pipe(sftpWriteStream);\r\n      }\r\n    });\r\n  });\r\n}\r\n``` \r\n\r\nany hints?\r\n\r\nRegards!"},"comment":{"url":"https://api.github.com/repos/mscdex/ssh2/issues/comments/240750857","html_url":"https://github.com/mscdex/ssh2/issues/459#issuecomment-240750857","issue_url":"https://api.github.com/repos/mscdex/ssh2/issues/459","id":240750857,"user":{"login":"mscdex","id":54666,"avatar_url":"https://avatars.githubusercontent.com/u/54666?v=3","gravatar_id":"","url":"https://api.github.com/users/mscdex","html_url":"https://github.com/mscdex","followers_url":"https://api.github.com/users/mscdex/followers","following_url":"https://api.github.com/users/mscdex/following{/other_user}","gists_url":"https://api.github.com/users/mscdex/gists{/gist_id}","starred_url":"https://api.github.com/users/mscdex/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mscdex/subscriptions","organizations_url":"https://api.github.com/users/mscdex/orgs","repos_url":"https://api.github.com/users/mscdex/repos","events_url":"https://api.github.com/users/mscdex/events{/privacy}","received_events_url":"https://api.github.com/users/mscdex/received_events","type":"User","site_admin":false},"created_at":"2016-08-18T15:00:43Z","updated_at":"2016-08-18T15:00:43Z","body":"If you are able to manually do multiple, parallel reads from GCS for a single file, then you could try to implement a `sftp.fastPut()`-like function. You could look at the source code for that function and use it as a guide/reference if you want to see one way to accomplish the parallel reads/writes.\r\n\r\nAnother solution which may/may not be feasible is to have the destination server pull down the file directly from GCS so that the traffic doesn't have to go through a middle system. I'm not familiar with GCS, but it would be something like:\r\n\r\n```js\r\nssh.exec('curl -L https://gcs-server.example.org/myfile > /path/to/save/file',\r\n          (err, stream) => {\r\n  /*..*/\r\n});\r\n```\r\ninstead of using `sftp.*` to transfer."}}
{"actor":{"display_login":"mscdex"}}