{"action":"created","issue":{"url":"https://api.github.com/repos/SciLifeLab/taca-ngi-pipeline/issues/22","repository_url":"https://api.github.com/repos/SciLifeLab/taca-ngi-pipeline","labels_url":"https://api.github.com/repos/SciLifeLab/taca-ngi-pipeline/issues/22/labels{/name}","comments_url":"https://api.github.com/repos/SciLifeLab/taca-ngi-pipeline/issues/22/comments","events_url":"https://api.github.com/repos/SciLifeLab/taca-ngi-pipeline/issues/22/events","html_url":"https://github.com/SciLifeLab/taca-ngi-pipeline/pull/22","id":141800480,"number":22,"title":"Mosler delivery support","user":{"login":"vezzi","id":1209695,"avatar_url":"https://avatars.githubusercontent.com/u/1209695?v=3","gravatar_id":"","url":"https://api.github.com/users/vezzi","html_url":"https://github.com/vezzi","followers_url":"https://api.github.com/users/vezzi/followers","following_url":"https://api.github.com/users/vezzi/following{/other_user}","gists_url":"https://api.github.com/users/vezzi/gists{/gist_id}","starred_url":"https://api.github.com/users/vezzi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vezzi/subscriptions","organizations_url":"https://api.github.com/users/vezzi/orgs","repos_url":"https://api.github.com/users/vezzi/repos","events_url":"https://api.github.com/users/vezzi/events{/privacy}","received_events_url":"https://api.github.com/users/vezzi/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-03-18T08:12:06Z","updated_at":"2016-03-21T14:31:47Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/SciLifeLab/taca-ngi-pipeline/pulls/22","html_url":"https://github.com/SciLifeLab/taca-ngi-pipeline/pull/22","diff_url":"https://github.com/SciLifeLab/taca-ngi-pipeline/pull/22.diff","patch_url":"https://github.com/SciLifeLab/taca-ngi-pipeline/pull/22.patch"},"body":"Command to be run looks like:\r\n `taca -c ~/.taca/taca.deliver.sthlm.v2.yaml deliver --force --uppnexid bils2016002  mosler P2262`\r\nThe following fields need to be added to the config file:\r\n```\r\n    moslerdeliverypath: /<UPPNEXID>  \r\n    moslersftpserver: mosler.bils.se\r\n    moslersftpserver_user: vezzi  #mosler username (same of milou, nestor)\r\n    moslersftpmaxfiles: 9 # max number of files that can be present on mosler sftp server \r\n```\r\n\r\nThe script does the following:\r\n - stage the delivery\r\n - compute checksums \r\n - tar ball the sample dir\r\n - move the tar to mosler sftp server\r\n\r\nthe password and token auth need to be entered only once. The person need to be a member of the mosler project.\r\nFor now the limiting factor is the delivery time. Last test I performed a delivery took 100 minutes. However there are too many \"limiting\" factors:\r\n - tar-balling on nestor is extremely slow and cannot be multi-threaded (can be but I kill the system)\r\n - moving at the same time more than one sample saturate the bandwith... but I need to test this out\r\n\r\nFor now reports are not generated. Delivery to mosler should be a one time thing... so we can generate them independently and send them at once. Also checking the success of the transfer...  I want to do it but need to think a bit more about it (the success corresponds to the file disappearing from mosler sftp... which can happen at any time...)\r\n\r\nIt is better to fist stage and compute the checksums, and then delivery, in this way the deliveries of samples should start in a quicker way.  But again this needs to be tested on irma, where performances should improve a lot.\r\n\r\nIt is unclear if we will deliver with this solution or with something else, but for now this is stable enough to have and I do not want to develop more until Irma is fully in place and we know how we will deliver things.\r\n\r\n@b97pla \r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/SciLifeLab/taca-ngi-pipeline/issues/comments/199315989","html_url":"https://github.com/SciLifeLab/taca-ngi-pipeline/pull/22#issuecomment-199315989","issue_url":"https://api.github.com/repos/SciLifeLab/taca-ngi-pipeline/issues/22","id":199315989,"user":{"login":"vezzi","id":1209695,"avatar_url":"https://avatars.githubusercontent.com/u/1209695?v=3","gravatar_id":"","url":"https://api.github.com/users/vezzi","html_url":"https://github.com/vezzi","followers_url":"https://api.github.com/users/vezzi/followers","following_url":"https://api.github.com/users/vezzi/following{/other_user}","gists_url":"https://api.github.com/users/vezzi/gists{/gist_id}","starred_url":"https://api.github.com/users/vezzi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vezzi/subscriptions","organizations_url":"https://api.github.com/users/vezzi/orgs","repos_url":"https://api.github.com/users/vezzi/repos","events_url":"https://api.github.com/users/vezzi/events{/privacy}","received_events_url":"https://api.github.com/users/vezzi/received_events","type":"User","site_admin":false},"created_at":"2016-03-21T14:31:47Z","updated_at":"2016-03-21T14:31:47Z","body":"@b97pla tests have passed, needs to be tested on Irma anyway and it will likely require further development. "}}