{"action":"closed","issue":{"url":"https://api.github.com/repos/mar-file-system/marfs/issues/74","labels_url":"https://api.github.com/repos/mar-file-system/marfs/issues/74/labels{/name}","comments_url":"https://api.github.com/repos/mar-file-system/marfs/issues/74/comments","events_url":"https://api.github.com/repos/mar-file-system/marfs/issues/74/events","html_url":"https://github.com/mar-file-system/marfs/issues/74","id":124816952,"number":74,"title":"prevent access to incomplete large files","user":{"login":"jti-lanl","id":8238841,"avatar_url":"https://avatars.githubusercontent.com/u/8238841?v=3","gravatar_id":"","url":"https://api.github.com/users/jti-lanl","html_url":"https://github.com/jti-lanl","followers_url":"https://api.github.com/users/jti-lanl/followers","following_url":"https://api.github.com/users/jti-lanl/following{/other_user}","gists_url":"https://api.github.com/users/jti-lanl/gists{/gist_id}","starred_url":"https://api.github.com/users/jti-lanl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jti-lanl/subscriptions","organizations_url":"https://api.github.com/users/jti-lanl/orgs","repos_url":"https://api.github.com/users/jti-lanl/repos","events_url":"https://api.github.com/users/jti-lanl/events{/privacy}","received_events_url":"https://api.github.com/users/jti-lanl/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"jti-lanl","id":8238841,"avatar_url":"https://avatars.githubusercontent.com/u/8238841?v=3","gravatar_id":"","url":"https://api.github.com/users/jti-lanl","html_url":"https://github.com/jti-lanl","followers_url":"https://api.github.com/users/jti-lanl/followers","following_url":"https://api.github.com/users/jti-lanl/following{/other_user}","gists_url":"https://api.github.com/users/jti-lanl/gists{/gist_id}","starred_url":"https://api.github.com/users/jti-lanl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jti-lanl/subscriptions","organizations_url":"https://api.github.com/users/jti-lanl/orgs","repos_url":"https://api.github.com/users/jti-lanl/repos","events_url":"https://api.github.com/users/jti-lanl/events{/privacy}","received_events_url":"https://api.github.com/users/jti-lanl/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2016-01-04T18:57:39Z","updated_at":"2016-01-05T17:51:25Z","closed_at":"2016-01-05T17:51:25Z","body":"We accept some degree of failures from pftool, writing files.\r\n\r\nIn the case of N:N writes, any failed destination files should probably just be unlinked.\r\n\r\nBut in the case of N:1, we will potentially have a single large file with \"holes\" in it where some writes failed.  pftool should eventually be able to fill these holes during a \"restart\" run.  But, in the meantime, attempts to read the file should fail.  (Or, more ambitiously, we could fail any reads that touch a \"hole\".)\r\n\r\nEven if we wanted to (i.e. ignoring the waste of effort), we can't just unlink the N:1 destination file at the time a write-failure is detected, because multiple writers may be writing to the file, some of which haven't even started yet.\r\n\r\nThe best approach is to rigorously apply the implicit scheme we are already using, where files are given a \"marfs_restart\" xattr at open time, which is removed upon successful close.  Then, we should also have MarFS refuse/control read-access to files that still have this flag.  In the case of N:1, the flag should not be removed at close-time, but at post_process time, from pftool.  This might be awkward.  It implies MarFS internals must check the object-type at close time, and only clear the xattr for non-Nto1.\r\n"}}