{"action":"created","issue":{"url":"https://api.github.com/repos/HenrikBengtsson/Wishlist-for-R/issues/20","repository_url":"https://api.github.com/repos/HenrikBengtsson/Wishlist-for-R","labels_url":"https://api.github.com/repos/HenrikBengtsson/Wishlist-for-R/issues/20/labels{/name}","comments_url":"https://api.github.com/repos/HenrikBengtsson/Wishlist-for-R/issues/20/comments","events_url":"https://api.github.com/repos/HenrikBengtsson/Wishlist-for-R/issues/20/events","html_url":"https://github.com/HenrikBengtsson/Wishlist-for-R/issues/20","id":145550887,"number":20,"title":"WISH: Atomic writing to file","user":{"login":"HenrikBengtsson","id":1616850,"avatar_url":"https://avatars.githubusercontent.com/u/1616850?v=3","gravatar_id":"","url":"https://api.github.com/users/HenrikBengtsson","html_url":"https://github.com/HenrikBengtsson","followers_url":"https://api.github.com/users/HenrikBengtsson/followers","following_url":"https://api.github.com/users/HenrikBengtsson/following{/other_user}","gists_url":"https://api.github.com/users/HenrikBengtsson/gists{/gist_id}","starred_url":"https://api.github.com/users/HenrikBengtsson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HenrikBengtsson/subscriptions","organizations_url":"https://api.github.com/users/HenrikBengtsson/orgs","repos_url":"https://api.github.com/users/HenrikBengtsson/repos","events_url":"https://api.github.com/users/HenrikBengtsson/events{/privacy}","received_events_url":"https://api.github.com/users/HenrikBengtsson/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/HenrikBengtsson/Wishlist-for-R/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-03T23:11:12Z","updated_at":"2016-04-03T23:15:40Z","closed_at":null,"body":"(Adopted from [Wiki entry](https://github.com/HenrikBengtsson/Wishlist-for-R/wiki))\r\n\r\n# Background\r\nWhen writing to file, there is always the risk that the process is interrupted which may result in an incomplete file.  Depending on file format, it can be extremely hard, or even impossible, to detect that the file is incomplete.  For instance, if writing a data frame with 100,000 rows to a comma-delimited file using `write.csv()`, if we're unlucky, the writing may be interrupted at the end of a row, e.g. when 98,953 complete rows have been written.  If so, `data <- read.csv()` will happily read the 98,953 rows and there is no way for us to know that the file is incomplete.  Even if it is possible to detect incomplete and/or corrupt files, it can be extremely tedious to identify them.\r\n\r\nThis is a real problem when generating a large number of files, especially large files for which the risk of being exposed to an interrupt increases.   \r\n\r\n# Suggestion / Wish\r\nIf the file are written _atomically_, that is, either all of the file is there at the end or not at all, then the problem of knowing whether the file is complete or not would not exist.  One approach for writing files atomically is to write using a temporary file name and then rename on completion.\r\n\r\n## Prototype / example\r\n\r\nAssume we save the file using `saveRDS(x, file=\"foo.rds\", atomic=TRUE)`.  This could in principle be done as:\r\n\r\n1. `saveRDS(x, file=\"foo.rds.tmp\")`\r\n2. `file.rename(\"foo.rds.tmp\", \"foo.rds\")`\r\n\r\nIf there is an interrupt, there will be a left-over `*.rds.tmp` file, but not the final `*.rds` file.  There could be options for automatically cleaning up incomplete files, or renaming the temporary file to, say, `*.rds.error` if an error was thrown while writing the file.\r\n"},"comment":{"url":"https://api.github.com/repos/HenrikBengtsson/Wishlist-for-R/issues/comments/205075448","html_url":"https://github.com/HenrikBengtsson/Wishlist-for-R/issues/20#issuecomment-205075448","issue_url":"https://api.github.com/repos/HenrikBengtsson/Wishlist-for-R/issues/20","id":205075448,"user":{"login":"HenrikBengtsson","id":1616850,"avatar_url":"https://avatars.githubusercontent.com/u/1616850?v=3","gravatar_id":"","url":"https://api.github.com/users/HenrikBengtsson","html_url":"https://github.com/HenrikBengtsson","followers_url":"https://api.github.com/users/HenrikBengtsson/followers","following_url":"https://api.github.com/users/HenrikBengtsson/following{/other_user}","gists_url":"https://api.github.com/users/HenrikBengtsson/gists{/gist_id}","starred_url":"https://api.github.com/users/HenrikBengtsson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HenrikBengtsson/subscriptions","organizations_url":"https://api.github.com/users/HenrikBengtsson/orgs","repos_url":"https://api.github.com/users/HenrikBengtsson/repos","events_url":"https://api.github.com/users/HenrikBengtsson/events{/privacy}","received_events_url":"https://api.github.com/users/HenrikBengtsson/received_events","type":"User","site_admin":false},"created_at":"2016-04-03T23:15:40Z","updated_at":"2016-04-03T23:15:40Z","body":"On 2016-04-01, @gaborcsardi proposed that one could may design a specific connection type that does write-to-temporary-file-name-and-rename-when-done for us, e.g.\r\n```r\r\nsaveRDS(x, file=atomic(\"foo.rds\"))\r\nwrite.csv(data, file=atomic(\"data.csv\"))\r\n```\r\n"}}