{"action":"created","issue":{"url":"https://api.github.com/repos/BD2KGenomics/toil/issues/966","repository_url":"https://api.github.com/repos/BD2KGenomics/toil","labels_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/966/labels{/name}","comments_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/966/comments","events_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/966/events","html_url":"https://github.com/BD2KGenomics/toil/issues/966","id":159848493,"number":966,"title":"Wrong permissions on file in fileJobStore","user":{"login":"adderan","id":4521830,"avatar_url":"https://avatars.githubusercontent.com/u/4521830?v=3","gravatar_id":"","url":"https://api.github.com/users/adderan","html_url":"https://github.com/adderan","followers_url":"https://api.github.com/users/adderan/followers","following_url":"https://api.github.com/users/adderan/following{/other_user}","gists_url":"https://api.github.com/users/adderan/gists{/gist_id}","starred_url":"https://api.github.com/users/adderan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adderan/subscriptions","organizations_url":"https://api.github.com/users/adderan/orgs","repos_url":"https://api.github.com/users/adderan/repos","events_url":"https://api.github.com/users/adderan/events{/privacy}","received_events_url":"https://api.github.com/users/adderan/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/BD2KGenomics/toil/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/BD2KGenomics/toil/labels/caching","name":"caching","color":"bfe5bf"}],"state":"open","locked":false,"assignee":{"login":"arkal","id":11653944,"avatar_url":"https://avatars.githubusercontent.com/u/11653944?v=3","gravatar_id":"","url":"https://api.github.com/users/arkal","html_url":"https://github.com/arkal","followers_url":"https://api.github.com/users/arkal/followers","following_url":"https://api.github.com/users/arkal/following{/other_user}","gists_url":"https://api.github.com/users/arkal/gists{/gist_id}","starred_url":"https://api.github.com/users/arkal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arkal/subscriptions","organizations_url":"https://api.github.com/users/arkal/orgs","repos_url":"https://api.github.com/users/arkal/repos","events_url":"https://api.github.com/users/arkal/events{/privacy}","received_events_url":"https://api.github.com/users/arkal/received_events","type":"User","site_admin":false},"milestone":null,"comments":3,"created_at":"2016-06-12T21:24:04Z","updated_at":"2016-06-13T20:07:59Z","closed_at":null,"body":"```\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Obtained lock on file /tmp/toil-bf91efaf-3d0a-441d-8e55-40a040a201d7/cache-bf91efaf-3d0a-441d-8e55-40a040a201d7/.cacheLock\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Released lock\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Obtained lock on file /tmp/toil-bf91efaf-3d0a-441d-8e55-40a040a201d7/cache-bf91efaf-3d0a-441d-8e55-40a040a201d7/.cacheLock\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Released lock\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Obtained lock on file /tmp/toil-bf91efaf-3d0a-441d-8e55-40a040a201d7/cache-bf91efaf-3d0a-441d-8e55-40a040a201d7/.cacheLock\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Released lock\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Obtained lock on file /tmp/toil-bf91efaf-3d0a-441d-8e55-40a040a201d7/cache-bf91efaf-3d0a-441d-8e55-40a040a201d7/.cacheLock\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Cache miss on file with ID 'R/q/tmp4jyd3q.tmp'.\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Obtained lock on file /tmp/toil-bf91efaf-3d0a-441d-8e55-40a040a201d7/cache-bf91efaf-3d0a-441d-8e55-40a040a201d7/.cacheLock\r\nT/O/jobugCtyo:  INFO:toil.job:CACHE: Read file with ID 'R/q/tmp4jyd3q.tmp' from the cache.\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Released lock\r\nT/O/jobugCtyo:  DEBUG:toil.job:CACHE: Released lock\r\nT/O/jobugCtyo:  DEBUG:root:Running the command: cactus_workflow_getFlowers CRITICAL '<st_kv_database_conf type=\"kyoto_tycoon\">\r\nT/O/jobugCtyo:                          <kyoto_tycoon database_dir=\"/tmp/toil-bf91efaf-3d0a-441d-8e55-40a040a201d7/tmp4egzRz/2087b518-a4bc-4613-9299-5c6b2ac71f11/tL3udCx/cactusDB\" database_name=\"Anc2.kch\" host=\"thinkpad\" in_me\r\nmory=\"1\" port=\"1979\" snapshot=\"0\" />\r\nT/O/jobugCtyo:                  </st_kv_database_conf>' 0 100000000 50000000\r\nT/O/jobugCtyo:  Exception in thread Thread-25:\r\nT/O/jobugCtyo:  Traceback (most recent call last):\r\nT/O/jobugCtyo:    File \"/usr/lib/python2.7/threading.py\", line 801, in __bootstrap_inner\r\nT/O/jobugCtyo:      self.run()\r\nT/O/jobugCtyo:    File \"/usr/lib/python2.7/threading.py\", line 754, in run\r\nT/O/jobugCtyo:      self.__target(*self.__args, **self.__kwargs)\r\nT/O/jobugCtyo:    File \"/home/alden/progressiveCactus-new/python/local/lib/python2.7/site-packages/toil/job.py\", line 571, in asyncWrite\r\nT/O/jobugCtyo:      with jobStore.updateFileStream(jobStoreFileID) as outputFileHandle:\r\nT/O/jobugCtyo:    File \"/usr/lib/python2.7/contextlib.py\", line 17, in __enter__\r\nT/O/jobugCtyo:      return self.gen.next()\r\nT/O/jobugCtyo:    File \"/home/alden/progressiveCactus-new/python/local/lib/python2.7/site-packages/toil/jobStores/fileJobStore.py\", line 255, in updateFileStream\r\nT/O/jobugCtyo:      with open(self._getAbsPath(jobStoreFileID), 'w') as f:\r\nT/O/jobugCtyo:  IOError: [Errno 13] Permission denied: '/home/alden/alignments/toil/work/toil/tmp/R/q/tmp4jyd3q.tmp'\r\n```\r\n\r\nI tried to make a test case for this outside of cactus but wasn't able to recreate it exactly. It looks like the file is ending up with 444 permissions - I'm not sure if this is ever intended to occur or not. This appears to happen with graphs like this:\r\n\r\nA --> C \r\n| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |\r\nB &nbsp;&nbsp;&nbsp;&nbsp; D\r\n\r\nwhere A writes a file to the file store, B downloads it with mutable=True, then modifies it and writes it back to the file store with cleanup=False, and D downloads it. The test case I made with the above graph doesn't trigger the asyncWrite error, but the file store ID does have 444 permissions in job D."},"comment":{"url":"https://api.github.com/repos/BD2KGenomics/toil/issues/comments/225693050","html_url":"https://github.com/BD2KGenomics/toil/issues/966#issuecomment-225693050","issue_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/966","id":225693050,"user":{"login":"arkal","id":11653944,"avatar_url":"https://avatars.githubusercontent.com/u/11653944?v=3","gravatar_id":"","url":"https://api.github.com/users/arkal","html_url":"https://github.com/arkal","followers_url":"https://api.github.com/users/arkal/followers","following_url":"https://api.github.com/users/arkal/following{/other_user}","gists_url":"https://api.github.com/users/arkal/gists{/gist_id}","starred_url":"https://api.github.com/users/arkal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arkal/subscriptions","organizations_url":"https://api.github.com/users/arkal/orgs","repos_url":"https://api.github.com/users/arkal/repos","events_url":"https://api.github.com/users/arkal/events{/privacy}","received_events_url":"https://api.github.com/users/arkal/received_events","type":"User","site_admin":false},"created_at":"2016-06-13T20:07:59Z","updated_at":"2016-06-13T20:07:59Z","body":"Are you putting it back in the jobstore under the same jobstore ID?\n\n@hannes-ucsc didn't we decide that you can't overwrite the file in the job store? If you download a file and modify it, you need to upload it as a new jobstore ID"}}
{"actor":{"display_login":"arkal"}}