{"action":"created","issue":{"url":"https://api.github.com/repos/github/git-lfs/issues/1621","repository_url":"https://api.github.com/repos/github/git-lfs","labels_url":"https://api.github.com/repos/github/git-lfs/issues/1621/labels{/name}","comments_url":"https://api.github.com/repos/github/git-lfs/issues/1621/comments","events_url":"https://api.github.com/repos/github/git-lfs/issues/1621/events","html_url":"https://github.com/github/git-lfs/pull/1621","id":186927920,"number":1621,"title":"filter_stream: provide io.Reader implementation","user":{"login":"ttaylorr","id":443245,"avatar_url":"https://avatars.githubusercontent.com/u/443245?v=3","gravatar_id":"","url":"https://api.github.com/users/ttaylorr","html_url":"https://github.com/ttaylorr","followers_url":"https://api.github.com/users/ttaylorr/followers","following_url":"https://api.github.com/users/ttaylorr/following{/other_user}","gists_url":"https://api.github.com/users/ttaylorr/gists{/gist_id}","starred_url":"https://api.github.com/users/ttaylorr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ttaylorr/subscriptions","organizations_url":"https://api.github.com/users/ttaylorr/orgs","repos_url":"https://api.github.com/users/ttaylorr/repos","events_url":"https://api.github.com/users/ttaylorr/events{/privacy}","received_events_url":"https://api.github.com/users/ttaylorr/received_events","type":"User","site_admin":true},"labels":[{"id":459606653,"url":"https://api.github.com/repos/github/git-lfs/labels/1.5.0","name":"1.5.0","color":"f3f3f3","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/github/git-lfs/milestones/1","html_url":"https://github.com/github/git-lfs/milestones/Filter%20protocol%20support","labels_url":"https://api.github.com/repos/github/git-lfs/milestones/1/labels","id":2110220,"number":1,"title":"Filter protocol support","description":"","creator":{"login":"ttaylorr","id":443245,"avatar_url":"https://avatars.githubusercontent.com/u/443245?v=3","gravatar_id":"","url":"https://api.github.com/users/ttaylorr","html_url":"https://github.com/ttaylorr","followers_url":"https://api.github.com/users/ttaylorr/followers","following_url":"https://api.github.com/users/ttaylorr/following{/other_user}","gists_url":"https://api.github.com/users/ttaylorr/gists{/gist_id}","starred_url":"https://api.github.com/users/ttaylorr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ttaylorr/subscriptions","organizations_url":"https://api.github.com/users/ttaylorr/orgs","repos_url":"https://api.github.com/users/ttaylorr/repos","events_url":"https://api.github.com/users/ttaylorr/events{/privacy}","received_events_url":"https://api.github.com/users/ttaylorr/received_events","type":"User","site_admin":true},"open_issues":5,"closed_issues":1,"state":"open","created_at":"2016-11-02T17:25:24Z","updated_at":"2016-11-03T20:51:14Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2016-11-02T21:41:37Z","updated_at":"2016-11-04T21:30:48Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/github/git-lfs/pulls/1621","html_url":"https://github.com/github/git-lfs/pull/1621","diff_url":"https://github.com/github/git-lfs/pull/1621.diff","patch_url":"https://github.com/github/git-lfs/pull/1621.patch"},"body":"This pull-request provides an implementation of the `io.Reader` interface that can read data encoded with the protocol used for filter process communication.\r\n\r\nThis provides a huge savings on memory usage when cleaning files, as the shasum of the contents can be calculated as data is streamed in, and then discarded.\r\n\r\nThis pull request is implemented in three parts:\r\n\r\n- https://github.com/github/git-lfs/commit/4524d1221efc54d82f4a409208e0d83a56a1c78e: Implement the underlying `PacketReader`.\r\n- https://github.com/github/git-lfs/commit/d23c0445f4580d6b82345ae3613bf47ecaff482a: Make the `WriteStatus` method public since reading packets no longer reports success/error statuses, and this responsibility will be delegated to the caller in the following commit...\r\n- https://github.com/github/git-lfs/commit/2db63047c028811deeab1b8462e44251304164ca: use an `io.Reader` as the `Payload` field of a request, instead of buffering the data upfront.\r\n\r\nSome notes on the implementation:\r\n\r\n- There is no \"offset\" encoded into where the reader starts and stops. This means that a given packet run must be _completely_ consumed before the next one is read. The code as it currently stands within this branch, does obey that rule as the synchronization is done with the loop condition, `scanner.Scan()`.\r\n- The implementation is safe to call with buffers shorter or longer than the packet that is being read. Any data that wont fit into the current given buffer will be sent to an internal buffer, and subsequent calls to `Read(p []byte)` will read first from the internal buffer until the internal buffer is drained.\r\n- An `io.EOF` is returned on the first call to `Read()` _after_ all of the data has been consumed. This is done to avoid a complicated implementation dealing with peeking bytes to see if more data is incoming. Luckily, the Go documentation allows for this (meaning things like `io.Copy`, etc will respect it):\r\n\r\n> An instance of this general case is that a Reader returning a non-zero number of bytes at the end of the input stream may return either err == EOF or err == nil. The next Read should return 0, EOF.\r\n\r\n(See: https://golang.org/pkg/io#Reader).\r\n\r\n---\r\n\r\n/cc @larsxschneider @technoweenie @sinbad @rubyist @sschuberth "},"comment":{"url":"https://api.github.com/repos/github/git-lfs/issues/comments/258552609","html_url":"https://github.com/github/git-lfs/pull/1621#issuecomment-258552609","issue_url":"https://api.github.com/repos/github/git-lfs/issues/1621","id":258552609,"user":{"login":"sschuberth","id":349154,"avatar_url":"https://avatars.githubusercontent.com/u/349154?v=3","gravatar_id":"","url":"https://api.github.com/users/sschuberth","html_url":"https://github.com/sschuberth","followers_url":"https://api.github.com/users/sschuberth/followers","following_url":"https://api.github.com/users/sschuberth/following{/other_user}","gists_url":"https://api.github.com/users/sschuberth/gists{/gist_id}","starred_url":"https://api.github.com/users/sschuberth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sschuberth/subscriptions","organizations_url":"https://api.github.com/users/sschuberth/orgs","repos_url":"https://api.github.com/users/sschuberth/repos","events_url":"https://api.github.com/users/sschuberth/events{/privacy}","received_events_url":"https://api.github.com/users/sschuberth/received_events","type":"User","site_admin":false},"created_at":"2016-11-04T21:30:48Z","updated_at":"2016-11-04T21:30:48Z","body":"> We could also just specify that tracking empty files is not allowed and just enforce that requirement earlier on in LFS\r\n\r\nI agree here. **L**FS is for **large** files after all, and zero is not large ;-)"}}
{"actor":{"display_login":"sschuberth"}}