{"action":"created","issue":{"url":"https://api.github.com/repos/nextflow-io/nextflow/issues/160","repository_url":"https://api.github.com/repos/nextflow-io/nextflow","labels_url":"https://api.github.com/repos/nextflow-io/nextflow/issues/160/labels{/name}","comments_url":"https://api.github.com/repos/nextflow-io/nextflow/issues/160/comments","events_url":"https://api.github.com/repos/nextflow-io/nextflow/issues/160/events","html_url":"https://github.com/nextflow-io/nextflow/issues/160","id":155697271,"number":160,"title":"PublishDir fail to move the output directory when the target is a S3 path","user":{"login":"pditommaso","id":816968,"avatar_url":"https://avatars.githubusercontent.com/u/816968?v=3","gravatar_id":"","url":"https://api.github.com/users/pditommaso","html_url":"https://github.com/pditommaso","followers_url":"https://api.github.com/users/pditommaso/followers","following_url":"https://api.github.com/users/pditommaso/following{/other_user}","gists_url":"https://api.github.com/users/pditommaso/gists{/gist_id}","starred_url":"https://api.github.com/users/pditommaso/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pditommaso/subscriptions","organizations_url":"https://api.github.com/users/pditommaso/orgs","repos_url":"https://api.github.com/users/pditommaso/repos","events_url":"https://api.github.com/users/pditommaso/events{/privacy}","received_events_url":"https://api.github.com/users/pditommaso/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-05-19T09:53:42Z","updated_at":"2016-05-19T15:52:29Z","closed_at":null,"body":"For example: \r\n\r\n\tprocess foo {\r\n\t  publishDir 's3://cbcrg-eu/work/hello/', mode: 'move'\r\n\r\n\t  output: \r\n\t  file('x')\r\n\r\n\t  \"\"\"\r\n\t  mkdir x \r\n\t  touch x/one\r\n\t  touch x/two\r\n\t  \"\"\"\r\n\t}\r\n\r\nIt create an empty `s3://cbcrg-eu/work/hello/x` path, instead it should contain the file `one` and `two`. \r\n"},"comment":{"url":"https://api.github.com/repos/nextflow-io/nextflow/issues/comments/220368336","html_url":"https://github.com/nextflow-io/nextflow/issues/160#issuecomment-220368336","issue_url":"https://api.github.com/repos/nextflow-io/nextflow/issues/160","id":220368336,"user":{"login":"jbyars","id":2440592,"avatar_url":"https://avatars.githubusercontent.com/u/2440592?v=3","gravatar_id":"","url":"https://api.github.com/users/jbyars","html_url":"https://github.com/jbyars","followers_url":"https://api.github.com/users/jbyars/followers","following_url":"https://api.github.com/users/jbyars/following{/other_user}","gists_url":"https://api.github.com/users/jbyars/gists{/gist_id}","starred_url":"https://api.github.com/users/jbyars/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jbyars/subscriptions","organizations_url":"https://api.github.com/users/jbyars/orgs","repos_url":"https://api.github.com/users/jbyars/repos","events_url":"https://api.github.com/users/jbyars/events{/privacy}","received_events_url":"https://api.github.com/users/jbyars/received_events","type":"User","site_admin":false},"created_at":"2016-05-19T15:52:29Z","updated_at":"2016-05-19T15:52:29Z","body":"Here is my example case. I'm probably just making a simple mistake. This is running on nextflow 0.18.3, ubuntu 14.04, aws region us-east-1, on r3.4xlarge instances. `mode: 'copy'` behaves as expected for me. `mode: 'move'` just creates empty folders in the bucket.\r\n```\r\n#!/usr/bin/env nextflow\r\n\r\n/*\r\n  This tests the behavior of publishDir outputting folders to buckets.\r\n  Create a few txt files in the working folder before running this.\r\n*/\r\n\r\nparams.bucket = \"s3://nextflow-demo-output\"\r\n\r\nChannel\r\n    .fromPath( \"*.txt\" )\r\n    .set { copyfiles }\r\n\r\nChannel\r\n    .fromPath( \"*.txt\" )\r\n    .set { movefiles }\r\n\r\nprocess copytest {\r\n    tag { sam }\r\n\r\n    publishDir params.bucket + '/copy', mode: 'copy'\r\n\r\n    input:\r\n    file sam from copyfiles\r\n\r\n    output:\r\n    file out into copyresults\r\n\r\n    script:\r\n    out=sam.getBaseName()\r\n    \"\"\"\r\n    mkdir \"$out\"\r\n    touch \"${out}/result1.txt\"\r\n    touch \"${out}/result2.txt\"\r\n    touch \"${out}/result3.txt\"\r\n    \"\"\"\r\n}\r\n\r\nprocess movetest {\r\n    tag { sam }\r\n\r\n    publishDir params.bucket + '/move', mode: 'move'\r\n\r\n    input:\r\n    file sam from movefiles\r\n\r\n    output:\r\n    file out into copyresults\r\n\r\n    script:\r\n    out=sam.getBaseName()\r\n    \"\"\"\r\n    mkdir \"$out\"\r\n    touch \"${out}/result1.txt\"\r\n    touch \"${out}/result2.txt\"\r\n    touch \"${out}/result3.txt\"\r\n    \"\"\"\r\n}\r\n```"}}