{"action":"created","issue":{"url":"https://api.github.com/repos/qri-io/dataset/issues/160","repository_url":"https://api.github.com/repos/qri-io/dataset","labels_url":"https://api.github.com/repos/qri-io/dataset/issues/160/labels{/name}","comments_url":"https://api.github.com/repos/qri-io/dataset/issues/160/comments","events_url":"https://api.github.com/repos/qri-io/dataset/issues/160/events","html_url":"https://github.com/qri-io/dataset/pull/160","id":387441604,"node_id":"MDExOlB1bGxSZXF1ZXN0MjM1OTI5MzEz","number":160,"title":"refactor(cbor): Rewrite of CBOR reader. Faster and simpler.","user":{"login":"dustmop","id":2515490,"node_id":"MDQ6VXNlcjI1MTU0OTA=","avatar_url":"https://avatars2.githubusercontent.com/u/2515490?v=4","gravatar_id":"","url":"https://api.github.com/users/dustmop","html_url":"https://github.com/dustmop","followers_url":"https://api.github.com/users/dustmop/followers","following_url":"https://api.github.com/users/dustmop/following{/other_user}","gists_url":"https://api.github.com/users/dustmop/gists{/gist_id}","starred_url":"https://api.github.com/users/dustmop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dustmop/subscriptions","organizations_url":"https://api.github.com/users/dustmop/orgs","repos_url":"https://api.github.com/users/dustmop/repos","events_url":"https://api.github.com/users/dustmop/events{/privacy}","received_events_url":"https://api.github.com/users/dustmop/received_events","type":"User","site_admin":false},"labels":[{"id":702211389,"node_id":"MDU6TGFiZWw3MDIyMTEzODk=","url":"https://api.github.com/repos/qri-io/dataset/labels/in%20progress","name":"in progress","color":"ededed","default":false}],"state":"open","locked":false,"assignee":{"login":"dustmop","id":2515490,"node_id":"MDQ6VXNlcjI1MTU0OTA=","avatar_url":"https://avatars2.githubusercontent.com/u/2515490?v=4","gravatar_id":"","url":"https://api.github.com/users/dustmop","html_url":"https://github.com/dustmop","followers_url":"https://api.github.com/users/dustmop/followers","following_url":"https://api.github.com/users/dustmop/following{/other_user}","gists_url":"https://api.github.com/users/dustmop/gists{/gist_id}","starred_url":"https://api.github.com/users/dustmop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dustmop/subscriptions","organizations_url":"https://api.github.com/users/dustmop/orgs","repos_url":"https://api.github.com/users/dustmop/repos","events_url":"https://api.github.com/users/dustmop/events{/privacy}","received_events_url":"https://api.github.com/users/dustmop/received_events","type":"User","site_admin":false},"assignees":[{"login":"dustmop","id":2515490,"node_id":"MDQ6VXNlcjI1MTU0OTA=","avatar_url":"https://avatars2.githubusercontent.com/u/2515490?v=4","gravatar_id":"","url":"https://api.github.com/users/dustmop","html_url":"https://github.com/dustmop","followers_url":"https://api.github.com/users/dustmop/followers","following_url":"https://api.github.com/users/dustmop/following{/other_user}","gists_url":"https://api.github.com/users/dustmop/gists{/gist_id}","starred_url":"https://api.github.com/users/dustmop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dustmop/subscriptions","organizations_url":"https://api.github.com/users/dustmop/orgs","repos_url":"https://api.github.com/users/dustmop/repos","events_url":"https://api.github.com/users/dustmop/events{/privacy}","received_events_url":"https://api.github.com/users/dustmop/received_events","type":"User","site_admin":false}],"milestone":null,"comments":0,"created_at":"2018-12-04T19:22:22Z","updated_at":"2018-12-04T21:42:41Z","closed_at":null,"author_association":"COLLABORATOR","pull_request":{"url":"https://api.github.com/repos/qri-io/dataset/pulls/160","html_url":"https://github.com/qri-io/dataset/pull/160","diff_url":"https://github.com/qri-io/dataset/pull/160.diff","patch_url":"https://github.com/qri-io/dataset/pull/160.patch"},"body":"Rewrite CBOR reader so that it avoids buffering tokens. Instead, output parsed elements as they are read off the input reader. Decode CBOR bytes on our own, instead of relying on another library (which is what required buffering).\r\n\r\nPerformance is about 3x better. Before:\r\n\r\n```\r\n> go test ./dsio/. -bench=.\r\n\r\nBenchmarkCBORWriterArrays-2    \t    3000\t    441960 ns/op\r\nBenchmarkCBORWriterObjects-2   \t    3000\t    617959 ns/op\r\nBenchmarkCBORReader-2          \t     200\t   6453132 ns/op\r\nBenchmarkCSVWriterArrays-2     \t    1000\t   1547526 ns/op\r\nBenchmarkCSVWriterObjects-2    \t    1000\t   1970756 ns/op\r\nBenchmarkCSVReader-2           \t     500\t   2300161 ns/op\r\nBenchmarkJSONWriterArrays-2    \t    1000\t   1890182 ns/op\r\nBenchmarkJSONWriterObjects-2   \t     500\t   2021165 ns/op\r\nBenchmarkJSONReader-2          \t     500\t   3043123 ns/op\r\n```\r\n\r\n(compare to https://github.com/qri-io/dataset/commit/819375c81ba49a904b97dd1f670e90920b4edbca)\r\n\r\nAfter:\r\n\r\n```\r\nBenchmarkCBORWriterArrays-2    \t    3000\t    431290 ns/op\r\nBenchmarkCBORWriterObjects-2   \t    2000\t    698920 ns/op\r\nBenchmarkCBORReader-2          \t    1000\t   1764549 ns/op\r\nBenchmarkCSVWriterArrays-2     \t    1000\t   1548509 ns/op\r\nBenchmarkCSVWriterObjects-2    \t    1000\t   1458219 ns/op\r\nBenchmarkCSVReader-2           \t    1000\t   2008097 ns/op\r\nBenchmarkJSONWriterArrays-2    \t    1000\t   1556416 ns/op\r\nBenchmarkJSONWriterObjects-2   \t    1000\t   1562488 ns/op\r\nBenchmarkJSONReader-2          \t     500\t   2984057 ns/op\r\n```"},"comment":{"url":"https://api.github.com/repos/qri-io/dataset/issues/comments/444269858","html_url":"https://github.com/qri-io/dataset/pull/160#issuecomment-444269858","issue_url":"https://api.github.com/repos/qri-io/dataset/issues/160","id":444269858,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0NDI2OTg1OA==","user":{"login":"b5","id":1154390,"node_id":"MDQ6VXNlcjExNTQzOTA=","avatar_url":"https://avatars1.githubusercontent.com/u/1154390?v=4","gravatar_id":"","url":"https://api.github.com/users/b5","html_url":"https://github.com/b5","followers_url":"https://api.github.com/users/b5/followers","following_url":"https://api.github.com/users/b5/following{/other_user}","gists_url":"https://api.github.com/users/b5/gists{/gist_id}","starred_url":"https://api.github.com/users/b5/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/b5/subscriptions","organizations_url":"https://api.github.com/users/b5/orgs","repos_url":"https://api.github.com/users/b5/repos","events_url":"https://api.github.com/users/b5/events{/privacy}","received_events_url":"https://api.github.com/users/b5/received_events","type":"User","site_admin":false},"created_at":"2018-12-04T21:42:41Z","updated_at":"2018-12-04T21:42:41Z","author_association":"MEMBER","body":"[benchcmp](https://godoc.org/golang.org/x/tools/cmd/benchcmp) can make these comparisons even more delightful so we can see just how much faster things got 😄 \r\n```\r\nbenchmark                        old ns/op     new ns/op     delta\r\nBenchmarkCBORWriterArrays-4      156580        154605        -1.26%\r\nBenchmarkCBORWriterObjects-4     225393        227865        +1.10%\r\nBenchmarkCBORReader-4            1599268       585046        -63.42%\r\nBenchmarkCSVWriterArrays-4       530450        510958        -3.67%\r\nBenchmarkCSVWriterObjects-4      517925        508732        -1.77%\r\nBenchmarkCSVReader-4             598331        586055        -2.05%\r\nBenchmarkJSONWriterArrays-4      604918        593041        -1.96%\r\nBenchmarkJSONWriterObjects-4     647054        645048        -0.31%\r\nBenchmarkJSONReader-4            1012359       1008421       -0.39%\r\n```\r\n\r\ngot this by running:\r\n```\r\n$ go get golang.org/x/tools/cmd/benchcmp\r\n$ git checkout master\r\n$ go test ./dsio/. -bench=. > before.txt\r\n$ git checkout cbor-improve\r\n$ go test ./dsio/. -bench=. > after.txt\r\n$ benchcmp before.txt after.txt\r\n```"}}
{"actor":{"display_login":"b5"}}