{"action":"closed","number":1415,"pull_request":{"url":"https://api.github.com/repos/broadinstitute/cromwell/pulls/1415","id":85134208,"html_url":"https://github.com/broadinstitute/cromwell/pull/1415","diff_url":"https://github.com/broadinstitute/cromwell/pull/1415.diff","patch_url":"https://github.com/broadinstitute/cromwell/pull/1415.patch","issue_url":"https://api.github.com/repos/broadinstitute/cromwell/issues/1415","number":1415,"state":"closed","locked":false,"title":"Some globbing Closes #1321 Closes #1308 Closes #1245","user":{"login":"Horneth","id":2978948,"avatar_url":"https://avatars.githubusercontent.com/u/2978948?v=3","gravatar_id":"","url":"https://api.github.com/users/Horneth","html_url":"https://github.com/Horneth","followers_url":"https://api.github.com/users/Horneth/followers","following_url":"https://api.github.com/users/Horneth/following{/other_user}","gists_url":"https://api.github.com/users/Horneth/gists{/gist_id}","starred_url":"https://api.github.com/users/Horneth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Horneth/subscriptions","organizations_url":"https://api.github.com/users/Horneth/orgs","repos_url":"https://api.github.com/users/Horneth/repos","events_url":"https://api.github.com/users/Horneth/events{/privacy}","received_events_url":"https://api.github.com/users/Horneth/received_events","type":"User","site_admin":false},"body":"\r\n* Commit 1: Creates 2 subdirectories in the call directory.\r\n\r\n```\r\ncromwell-executions/globbinginput/\r\n└── 970c776f-3b9d-4c0e-a2ff-dbd76395b4ba\r\n    ├── call-globtask\r\n    │   ├── execution\r\n    │   │   ├── outputfile1.txt\r\n    │   │   ├── outputfile2.txt\r\n    │   │   ├── rc\r\n    │   │   ├── script\r\n    │   │   ├── script.background\r\n    │   │   ├── script.submit\r\n    │   │   ├── stderr\r\n    │   │   ├── stderr.background\r\n    │   │   ├── stdout\r\n    │   │   └── stdout.background\r\n    │   └── inputs\r\n    │       └── 90e8d77e2a99e99efa82c33e27365d71\r\n    │           └── somefile.txt\r\n```\r\n\r\nThis prevents globbing from matching input files as they're not in the same branch.\r\n\r\n* Commit 2: Instead of recreating the whole directory structure underneath the call directory, hash the path and create a single directory named with this hash. \r\n\r\n    * This reduces the depth of the call directory, and is hopefully a bit less confusing (people tend to be confused when they see the full path of their input file appended to the call directory)\r\n\r\n    * The hash is only computed from the fullpath of the input's parent directory (not including the filename). This ensures that multiple files from the same directory will still end up in the same directory once localized).\r\n\r\nThe hashing part is not necessary to fix this bug at all, and I remember the choice was made not to use hashes because it's unreadable and makes it hard to know where the inputs came from but I feel like recreating the full path underneath the call directory is even worse...\r\n\r\n* Commit 3: Change the globbing pattern so it does NOT traverse recursively the hierarchy looking for matches by default. If that's the desired behaviour it should be reflected in the glob pattern in the wdl (with `**`). This is necessary to allow for this kind of usage https://github.com/broadinstitute/cromwell/issues/1245\r\n","created_at":"2016-09-13T22:05:58Z","updated_at":"2016-09-14T13:00:39Z","closed_at":"2016-09-14T13:00:39Z","merged_at":null,"merge_commit_sha":"bbeb5a6abe4913bedf3adc82f3ee834203da2d98","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/broadinstitute/cromwell/pulls/1415/commits","review_comments_url":"https://api.github.com/repos/broadinstitute/cromwell/pulls/1415/comments","review_comment_url":"https://api.github.com/repos/broadinstitute/cromwell/pulls/comments{/number}","comments_url":"https://api.github.com/repos/broadinstitute/cromwell/issues/1415/comments","statuses_url":"https://api.github.com/repos/broadinstitute/cromwell/statuses/55dc0deccf410cee77ec9813f136cb638100672a","head":{"label":"broadinstitute:cromwell-1308","ref":"cromwell-1308","sha":"55dc0deccf410cee77ec9813f136cb638100672a","user":{"login":"broadinstitute","id":393552,"avatar_url":"https://avatars.githubusercontent.com/u/393552?v=3","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"repo":{"id":34136406,"name":"cromwell","full_name":"broadinstitute/cromwell","owner":{"login":"broadinstitute","id":393552,"avatar_url":"https://avatars.githubusercontent.com/u/393552?v=3","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/broadinstitute/cromwell","description":"Workflow Execution Engine using WDL","fork":false,"url":"https://api.github.com/repos/broadinstitute/cromwell","forks_url":"https://api.github.com/repos/broadinstitute/cromwell/forks","keys_url":"https://api.github.com/repos/broadinstitute/cromwell/keys{/key_id}","collaborators_url":"https://api.github.com/repos/broadinstitute/cromwell/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/broadinstitute/cromwell/teams","hooks_url":"https://api.github.com/repos/broadinstitute/cromwell/hooks","issue_events_url":"https://api.github.com/repos/broadinstitute/cromwell/issues/events{/number}","events_url":"https://api.github.com/repos/broadinstitute/cromwell/events","assignees_url":"https://api.github.com/repos/broadinstitute/cromwell/assignees{/user}","branches_url":"https://api.github.com/repos/broadinstitute/cromwell/branches{/branch}","tags_url":"https://api.github.com/repos/broadinstitute/cromwell/tags","blobs_url":"https://api.github.com/repos/broadinstitute/cromwell/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/broadinstitute/cromwell/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/broadinstitute/cromwell/git/refs{/sha}","trees_url":"https://api.github.com/repos/broadinstitute/cromwell/git/trees{/sha}","statuses_url":"https://api.github.com/repos/broadinstitute/cromwell/statuses/{sha}","languages_url":"https://api.github.com/repos/broadinstitute/cromwell/languages","stargazers_url":"https://api.github.com/repos/broadinstitute/cromwell/stargazers","contributors_url":"https://api.github.com/repos/broadinstitute/cromwell/contributors","subscribers_url":"https://api.github.com/repos/broadinstitute/cromwell/subscribers","subscription_url":"https://api.github.com/repos/broadinstitute/cromwell/subscription","commits_url":"https://api.github.com/repos/broadinstitute/cromwell/commits{/sha}","git_commits_url":"https://api.github.com/repos/broadinstitute/cromwell/git/commits{/sha}","comments_url":"https://api.github.com/repos/broadinstitute/cromwell/comments{/number}","issue_comment_url":"https://api.github.com/repos/broadinstitute/cromwell/issues/comments{/number}","contents_url":"https://api.github.com/repos/broadinstitute/cromwell/contents/{+path}","compare_url":"https://api.github.com/repos/broadinstitute/cromwell/compare/{base}...{head}","merges_url":"https://api.github.com/repos/broadinstitute/cromwell/merges","archive_url":"https://api.github.com/repos/broadinstitute/cromwell/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/broadinstitute/cromwell/downloads","issues_url":"https://api.github.com/repos/broadinstitute/cromwell/issues{/number}","pulls_url":"https://api.github.com/repos/broadinstitute/cromwell/pulls{/number}","milestones_url":"https://api.github.com/repos/broadinstitute/cromwell/milestones{/number}","notifications_url":"https://api.github.com/repos/broadinstitute/cromwell/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/broadinstitute/cromwell/labels{/name}","releases_url":"https://api.github.com/repos/broadinstitute/cromwell/releases{/id}","deployments_url":"https://api.github.com/repos/broadinstitute/cromwell/deployments","created_at":"2015-04-17T19:39:36Z","updated_at":"2016-09-09T09:42:54Z","pushed_at":"2016-09-14T10:54:10Z","git_url":"git://github.com/broadinstitute/cromwell.git","ssh_url":"git@github.com:broadinstitute/cromwell.git","clone_url":"https://github.com/broadinstitute/cromwell.git","svn_url":"https://github.com/broadinstitute/cromwell","homepage":"","size":9121,"stargazers_count":82,"watchers_count":82,"language":"Scala","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":21,"mirror_url":null,"open_issues_count":185,"forks":21,"open_issues":185,"watchers":82,"default_branch":"develop"}},"base":{"label":"broadinstitute:develop","ref":"develop","sha":"fba786821302a1ac34e234adf8c28421a2c15f4e","user":{"login":"broadinstitute","id":393552,"avatar_url":"https://avatars.githubusercontent.com/u/393552?v=3","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"repo":{"id":34136406,"name":"cromwell","full_name":"broadinstitute/cromwell","owner":{"login":"broadinstitute","id":393552,"avatar_url":"https://avatars.githubusercontent.com/u/393552?v=3","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/broadinstitute/cromwell","description":"Workflow Execution Engine using WDL","fork":false,"url":"https://api.github.com/repos/broadinstitute/cromwell","forks_url":"https://api.github.com/repos/broadinstitute/cromwell/forks","keys_url":"https://api.github.com/repos/broadinstitute/cromwell/keys{/key_id}","collaborators_url":"https://api.github.com/repos/broadinstitute/cromwell/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/broadinstitute/cromwell/teams","hooks_url":"https://api.github.com/repos/broadinstitute/cromwell/hooks","issue_events_url":"https://api.github.com/repos/broadinstitute/cromwell/issues/events{/number}","events_url":"https://api.github.com/repos/broadinstitute/cromwell/events","assignees_url":"https://api.github.com/repos/broadinstitute/cromwell/assignees{/user}","branches_url":"https://api.github.com/repos/broadinstitute/cromwell/branches{/branch}","tags_url":"https://api.github.com/repos/broadinstitute/cromwell/tags","blobs_url":"https://api.github.com/repos/broadinstitute/cromwell/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/broadinstitute/cromwell/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/broadinstitute/cromwell/git/refs{/sha}","trees_url":"https://api.github.com/repos/broadinstitute/cromwell/git/trees{/sha}","statuses_url":"https://api.github.com/repos/broadinstitute/cromwell/statuses/{sha}","languages_url":"https://api.github.com/repos/broadinstitute/cromwell/languages","stargazers_url":"https://api.github.com/repos/broadinstitute/cromwell/stargazers","contributors_url":"https://api.github.com/repos/broadinstitute/cromwell/contributors","subscribers_url":"https://api.github.com/repos/broadinstitute/cromwell/subscribers","subscription_url":"https://api.github.com/repos/broadinstitute/cromwell/subscription","commits_url":"https://api.github.com/repos/broadinstitute/cromwell/commits{/sha}","git_commits_url":"https://api.github.com/repos/broadinstitute/cromwell/git/commits{/sha}","comments_url":"https://api.github.com/repos/broadinstitute/cromwell/comments{/number}","issue_comment_url":"https://api.github.com/repos/broadinstitute/cromwell/issues/comments{/number}","contents_url":"https://api.github.com/repos/broadinstitute/cromwell/contents/{+path}","compare_url":"https://api.github.com/repos/broadinstitute/cromwell/compare/{base}...{head}","merges_url":"https://api.github.com/repos/broadinstitute/cromwell/merges","archive_url":"https://api.github.com/repos/broadinstitute/cromwell/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/broadinstitute/cromwell/downloads","issues_url":"https://api.github.com/repos/broadinstitute/cromwell/issues{/number}","pulls_url":"https://api.github.com/repos/broadinstitute/cromwell/pulls{/number}","milestones_url":"https://api.github.com/repos/broadinstitute/cromwell/milestones{/number}","notifications_url":"https://api.github.com/repos/broadinstitute/cromwell/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/broadinstitute/cromwell/labels{/name}","releases_url":"https://api.github.com/repos/broadinstitute/cromwell/releases{/id}","deployments_url":"https://api.github.com/repos/broadinstitute/cromwell/deployments","created_at":"2015-04-17T19:39:36Z","updated_at":"2016-09-09T09:42:54Z","pushed_at":"2016-09-14T10:54:10Z","git_url":"git://github.com/broadinstitute/cromwell.git","ssh_url":"git@github.com:broadinstitute/cromwell.git","clone_url":"https://github.com/broadinstitute/cromwell.git","svn_url":"https://github.com/broadinstitute/cromwell","homepage":"","size":9121,"stargazers_count":82,"watchers_count":82,"language":"Scala","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":21,"mirror_url":null,"open_issues_count":185,"forks":21,"open_issues":185,"watchers":82,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/broadinstitute/cromwell/pulls/1415"},"html":{"href":"https://github.com/broadinstitute/cromwell/pull/1415"},"issue":{"href":"https://api.github.com/repos/broadinstitute/cromwell/issues/1415"},"comments":{"href":"https://api.github.com/repos/broadinstitute/cromwell/issues/1415/comments"},"review_comments":{"href":"https://api.github.com/repos/broadinstitute/cromwell/pulls/1415/comments"},"review_comment":{"href":"https://api.github.com/repos/broadinstitute/cromwell/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/broadinstitute/cromwell/pulls/1415/commits"},"statuses":{"href":"https://api.github.com/repos/broadinstitute/cromwell/statuses/55dc0deccf410cee77ec9813f136cb638100672a"}},"merged":false,"mergeable":true,"mergeable_state":"unstable","merged_by":null,"comments":0,"review_comments":0,"commits":3,"additions":83,"deletions":68,"changed_files":16}}
{"actor":{"display_login":"Horneth"}}