{"action":"created","issue":{"url":"https://api.github.com/repos/caolan/highland/issues/431","repository_url":"https://api.github.com/repos/caolan/highland","labels_url":"https://api.github.com/repos/caolan/highland/issues/431/labels{/name}","comments_url":"https://api.github.com/repos/caolan/highland/issues/431/comments","events_url":"https://api.github.com/repos/caolan/highland/issues/431/events","html_url":"https://github.com/caolan/highland/issues/431","id":127270071,"number":431,"title":"Feedback: Filebuffer through stream","user":{"login":"kolpax","id":3372392,"avatar_url":"https://avatars.githubusercontent.com/u/3372392?v=3","gravatar_id":"","url":"https://api.github.com/users/kolpax","html_url":"https://github.com/kolpax","followers_url":"https://api.github.com/users/kolpax/followers","following_url":"https://api.github.com/users/kolpax/following{/other_user}","gists_url":"https://api.github.com/users/kolpax/gists{/gist_id}","starred_url":"https://api.github.com/users/kolpax/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kolpax/subscriptions","organizations_url":"https://api.github.com/users/kolpax/orgs","repos_url":"https://api.github.com/users/kolpax/repos","events_url":"https://api.github.com/users/kolpax/events{/privacy}","received_events_url":"https://api.github.com/users/kolpax/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2016-01-18T17:03:50Z","updated_at":"2016-02-03T06:48:10Z","closed_at":null,"body":"I implemented a filebuffer through stream that writes and reads to and from a file simultaneously. I am planning to use it in situations where we have to cache large incoming files to disk while they're being sent to the user over HTTP. As I in my case can't control the speed of the read stream, this implementation allows me buffer to disk instead of memory.\r\n\r\nAny feedback would be much appreciated!\r\n\r\n```es6\r\nimport fs from 'fs-promise'\r\nimport _ from 'highland'\r\n\r\nexport default function(filePath) {\r\n  return s => {\r\n    let eof = false\r\n\r\n    const tmpFilePath = `${filePath}.tmp`\r\n    const ws = fs.createWriteStream(tmpFilePath)\r\n\r\n    const errors = _((push) => {\r\n      s.pipe(ws)\r\n        .on('finish', () => {\r\n          eof = true\r\n          push(null, _.nil)\r\n          fs.rename(tmpFilePath, filePath)\r\n            .catch((err) => fs.unlink(tmpFilePath))\r\n        })\r\n        .on('error', (err) => push(err))\r\n    })\r\n\r\n    const readStream = ({ start = 0 } = {}) => {\r\n      return _(fs.createReadStream(eof ? filePath : tmpFilePath, { start }))\r\n        .consume((err, x, push, next) => {\r\n          if (err) {\r\n            push(err, null)\r\n          } else if (x === _.nil) {\r\n            if (eof) {\r\n              push(null, _.nil)\r\n            } else {\r\n              setTimeout(() => next(readStream({ start })), 1000)\r\n            }\r\n          } else {\r\n            start += x.length\r\n            push(null, x)\r\n            next()\r\n          }\r\n        })\r\n    }\r\n\r\n    return _([errors, readStream()]).merge()\r\n  }\r\n}\r\n```"},"comment":{"url":"https://api.github.com/repos/caolan/highland/issues/comments/179046260","html_url":"https://github.com/caolan/highland/issues/431#issuecomment-179046260","issue_url":"https://api.github.com/repos/caolan/highland/issues/431","id":179046260,"user":{"login":"vqvu","id":6435429,"avatar_url":"https://avatars.githubusercontent.com/u/6435429?v=3","gravatar_id":"","url":"https://api.github.com/users/vqvu","html_url":"https://github.com/vqvu","followers_url":"https://api.github.com/users/vqvu/followers","following_url":"https://api.github.com/users/vqvu/following{/other_user}","gists_url":"https://api.github.com/users/vqvu/gists{/gist_id}","starred_url":"https://api.github.com/users/vqvu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vqvu/subscriptions","organizations_url":"https://api.github.com/users/vqvu/orgs","repos_url":"https://api.github.com/users/vqvu/repos","events_url":"https://api.github.com/users/vqvu/events{/privacy}","received_events_url":"https://api.github.com/users/vqvu/received_events","type":"User","site_admin":false},"created_at":"2016-02-03T06:48:10Z","updated_at":"2016-02-03T06:48:10Z","body":"Hmm, that would suggest that that problem is either Highland or `PassThrough`. Does this case also use `filebuffer`?"}}