{"action":"created","comment":{"url":"https://api.github.com/repos/npm/npm/pulls/comments/77746130","id":77746130,"diff_hunk":"@@ -33,8 +33,9 @@ var union = function (streamA, streamB, toKey) {\n         var keyB = toKey(dataB)\n \n         if (keyA === keyB) {","path":"node_modules/sorted-union-stream/index.js","position":null,"original_position":3,"commit_id":"8d36ba24b9b56c8c8b777b403eb92d0b5afe2c78","original_commit_id":"e79ceae6b9531752a4a8d6515ebed28b2989b8cd","user":{"login":"iarna","id":983798,"avatar_url":"https://avatars.githubusercontent.com/u/983798?v=3","gravatar_id":"","url":"https://api.github.com/users/iarna","html_url":"https://github.com/iarna","followers_url":"https://api.github.com/users/iarna/followers","following_url":"https://api.github.com/users/iarna/following{/other_user}","gists_url":"https://api.github.com/users/iarna/gists{/gist_id}","starred_url":"https://api.github.com/users/iarna/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/iarna/subscriptions","organizations_url":"https://api.github.com/users/iarna/orgs","repos_url":"https://api.github.com/users/iarna/repos","events_url":"https://api.github.com/users/iarna/events{/privacy}","received_events_url":"https://api.github.com/users/iarna/received_events","type":"User","site_admin":false},"body":"Can you elaborate on what's going on here? And what this being fixed looks like?","created_at":"2016-09-07T01:05:42Z","updated_at":"2016-09-07T01:05:42Z","html_url":"https://github.com/npm/npm/pull/13746#discussion_r77746130","pull_request_url":"https://api.github.com/repos/npm/npm/pulls/13746","_links":{"self":{"href":"https://api.github.com/repos/npm/npm/pulls/comments/77746130"},"html":{"href":"https://github.com/npm/npm/pull/13746#discussion_r77746130"},"pull_request":{"href":"https://api.github.com/repos/npm/npm/pulls/13746"}}},"pull_request":{"url":"https://api.github.com/repos/npm/npm/pulls/13746","id":82458838,"html_url":"https://github.com/npm/npm/pull/13746","diff_url":"https://github.com/npm/npm/pull/13746.diff","patch_url":"https://github.com/npm/npm/pull/13746.patch","issue_url":"https://api.github.com/repos/npm/npm/issues/13746","number":13746,"state":"open","locked":false,"title":"Streaming CLI Search","user":{"login":"zkat","id":17535,"avatar_url":"https://avatars.githubusercontent.com/u/17535?v=3","gravatar_id":"","url":"https://api.github.com/users/zkat","html_url":"https://github.com/zkat","followers_url":"https://api.github.com/users/zkat/followers","following_url":"https://api.github.com/users/zkat/following{/other_user}","gists_url":"https://api.github.com/users/zkat/gists{/gist_id}","starred_url":"https://api.github.com/users/zkat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zkat/subscriptions","organizations_url":"https://api.github.com/users/zkat/orgs","repos_url":"https://api.github.com/users/zkat/repos","events_url":"https://api.github.com/users/zkat/events{/privacy}","received_events_url":"https://api.github.com/users/zkat/received_events","type":"User","site_admin":false},"body":"This is a WIP `npm search` rewrite branch, based off the work done by @aredridel over at #10615.\r\n\r\nThe entire search process is now streamed, end-to-end. This branch brings in a couple of stream support libraries to help things along, too.\r\n\r\nIt should be able to fix pretty much all the heap-blowup crashers in npm.\r\n\r\nFixes: #12619\r\nFixes: #13247\r\nFixes: #12587\r\nFixes: #12439\r\nFixes: #6254\r\n\r\n![streamed-search-pretty](https://cloud.githubusercontent.com/assets/17535/18196461/6197d546-70a6-11e6-918a-2fa6104ad2b7.gif)\r\n\r\nIt's not great to still be doing a linear search every time, but it's a significant step for search to be at least usable (without crashing or waiting forever for any results).\r\n\r\n### TODO\r\n- [x] Stream results from `request`\r\n- [x] Stream results from cache\r\n- [x] Stream output into cache file\r\n- [x] Find issues related to crashing search, close duplicates, and reference master issue here.\r\n- [x] Fix `cb not called` corner case when cache is an empty object (probably when request is empty, too -- check and test both)\r\n- [x] Figure out the `cb not called` thing with `request`'s `res` grrr\r\n- [ ] Write unit tests\r\n  - [x] `createEntryStream`\r\n  - [x] `createCacheEntryStream`\r\n  - [x] `createEntryUpdateStream`\r\n  - [x] `createCacheWriteStream`\r\n  - [x] `updateIndex`\r\n  - [ ] Figure out why some tests are hanging (try upgrading the version of hock in `npm-registry-mock`\r\n- [x] Prettify output\r\n- [x] Do the thing from https://github.com/npm/npm/issues/6254\r\n- [x] Support `--json` for streamed JSON output\r\n- [x] Add message for when there's no search matches\r\n- [X] Rewrite history to not be a crapmess\r\n- [ ] Rewrite integration test for `npm search`\r\n- [ ] Comb through code and make sure none of the object streams have absurd `highWaterMark` settings.\r\n- [x] Merge and release https://github.com/npm/npm-registry-client/pull/131\r\n- [ ] Remove hacked-in copy of `npm-registry-client`\r\n- [ ] Remove hacked-in copy of `sorted-union-stream` by finding a better way to patch this behavior. (most likely, this'll involve a fork)\r\n- [ ] see if new deps can be minimized (do we need all of `mississippi`? really?)\r\n\r\n","created_at":"2016-08-23T23:32:03Z","updated_at":"2016-09-07T01:05:42Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a95e82629dae77666c121351fbf742be3f7923dd","assignee":{"login":"zkat","id":17535,"avatar_url":"https://avatars.githubusercontent.com/u/17535?v=3","gravatar_id":"","url":"https://api.github.com/users/zkat","html_url":"https://github.com/zkat","followers_url":"https://api.github.com/users/zkat/followers","following_url":"https://api.github.com/users/zkat/following{/other_user}","gists_url":"https://api.github.com/users/zkat/gists{/gist_id}","starred_url":"https://api.github.com/users/zkat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zkat/subscriptions","organizations_url":"https://api.github.com/users/zkat/orgs","repos_url":"https://api.github.com/users/zkat/repos","events_url":"https://api.github.com/users/zkat/events{/privacy}","received_events_url":"https://api.github.com/users/zkat/received_events","type":"User","site_admin":false},"assignees":[{"login":"zkat","id":17535,"avatar_url":"https://avatars.githubusercontent.com/u/17535?v=3","gravatar_id":"","url":"https://api.github.com/users/zkat","html_url":"https://github.com/zkat","followers_url":"https://api.github.com/users/zkat/followers","following_url":"https://api.github.com/users/zkat/following{/other_user}","gists_url":"https://api.github.com/users/zkat/gists{/gist_id}","starred_url":"https://api.github.com/users/zkat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zkat/subscriptions","organizations_url":"https://api.github.com/users/zkat/orgs","repos_url":"https://api.github.com/users/zkat/repos","events_url":"https://api.github.com/users/zkat/events{/privacy}","received_events_url":"https://api.github.com/users/zkat/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/npm/npm/pulls/13746/commits","review_comments_url":"https://api.github.com/repos/npm/npm/pulls/13746/comments","review_comment_url":"https://api.github.com/repos/npm/npm/pulls/comments{/number}","comments_url":"https://api.github.com/repos/npm/npm/issues/13746/comments","statuses_url":"https://api.github.com/repos/npm/npm/statuses/8d36ba24b9b56c8c8b777b403eb92d0b5afe2c78","head":{"label":"npm:zkat/search","ref":"zkat/search","sha":"8d36ba24b9b56c8c8b777b403eb92d0b5afe2c78","user":{"login":"npm","id":6078720,"avatar_url":"https://avatars.githubusercontent.com/u/6078720?v=3","gravatar_id":"","url":"https://api.github.com/users/npm","html_url":"https://github.com/npm","followers_url":"https://api.github.com/users/npm/followers","following_url":"https://api.github.com/users/npm/following{/other_user}","gists_url":"https://api.github.com/users/npm/gists{/gist_id}","starred_url":"https://api.github.com/users/npm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/npm/subscriptions","organizations_url":"https://api.github.com/users/npm/orgs","repos_url":"https://api.github.com/users/npm/repos","events_url":"https://api.github.com/users/npm/events{/privacy}","received_events_url":"https://api.github.com/users/npm/received_events","type":"Organization","site_admin":false},"repo":{"id":321278,"name":"npm","full_name":"npm/npm","owner":{"login":"npm","id":6078720,"avatar_url":"https://avatars.githubusercontent.com/u/6078720?v=3","gravatar_id":"","url":"https://api.github.com/users/npm","html_url":"https://github.com/npm","followers_url":"https://api.github.com/users/npm/followers","following_url":"https://api.github.com/users/npm/following{/other_user}","gists_url":"https://api.github.com/users/npm/gists{/gist_id}","starred_url":"https://api.github.com/users/npm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/npm/subscriptions","organizations_url":"https://api.github.com/users/npm/orgs","repos_url":"https://api.github.com/users/npm/repos","events_url":"https://api.github.com/users/npm/events{/privacy}","received_events_url":"https://api.github.com/users/npm/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/npm/npm","description":"a package manager for javascript","fork":false,"url":"https://api.github.com/repos/npm/npm","forks_url":"https://api.github.com/repos/npm/npm/forks","keys_url":"https://api.github.com/repos/npm/npm/keys{/key_id}","collaborators_url":"https://api.github.com/repos/npm/npm/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/npm/npm/teams","hooks_url":"https://api.github.com/repos/npm/npm/hooks","issue_events_url":"https://api.github.com/repos/npm/npm/issues/events{/number}","events_url":"https://api.github.com/repos/npm/npm/events","assignees_url":"https://api.github.com/repos/npm/npm/assignees{/user}","branches_url":"https://api.github.com/repos/npm/npm/branches{/branch}","tags_url":"https://api.github.com/repos/npm/npm/tags","blobs_url":"https://api.github.com/repos/npm/npm/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/npm/npm/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/npm/npm/git/refs{/sha}","trees_url":"https://api.github.com/repos/npm/npm/git/trees{/sha}","statuses_url":"https://api.github.com/repos/npm/npm/statuses/{sha}","languages_url":"https://api.github.com/repos/npm/npm/languages","stargazers_url":"https://api.github.com/repos/npm/npm/stargazers","contributors_url":"https://api.github.com/repos/npm/npm/contributors","subscribers_url":"https://api.github.com/repos/npm/npm/subscribers","subscription_url":"https://api.github.com/repos/npm/npm/subscription","commits_url":"https://api.github.com/repos/npm/npm/commits{/sha}","git_commits_url":"https://api.github.com/repos/npm/npm/git/commits{/sha}","comments_url":"https://api.github.com/repos/npm/npm/comments{/number}","issue_comment_url":"https://api.github.com/repos/npm/npm/issues/comments{/number}","contents_url":"https://api.github.com/repos/npm/npm/contents/{+path}","compare_url":"https://api.github.com/repos/npm/npm/compare/{base}...{head}","merges_url":"https://api.github.com/repos/npm/npm/merges","archive_url":"https://api.github.com/repos/npm/npm/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/npm/npm/downloads","issues_url":"https://api.github.com/repos/npm/npm/issues{/number}","pulls_url":"https://api.github.com/repos/npm/npm/pulls{/number}","milestones_url":"https://api.github.com/repos/npm/npm/milestones{/number}","notifications_url":"https://api.github.com/repos/npm/npm/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/npm/npm/labels{/name}","releases_url":"https://api.github.com/repos/npm/npm/releases{/id}","deployments_url":"https://api.github.com/repos/npm/npm/deployments","created_at":"2009-09-29T17:21:24Z","updated_at":"2016-09-07T00:17:46Z","pushed_at":"2016-09-06T09:40:17Z","git_url":"git://github.com/npm/npm.git","ssh_url":"git@github.com:npm/npm.git","clone_url":"https://github.com/npm/npm.git","svn_url":"https://github.com/npm/npm","homepage":"http://www.npmjs.com/","size":50079,"stargazers_count":10220,"watchers_count":10220,"language":"JavaScript","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":2111,"mirror_url":null,"open_issues_count":2414,"forks":2111,"open_issues":2414,"watchers":10220,"default_branch":"master"}},"base":{"label":"npm:release-next","ref":"release-next","sha":"debb05e8f64b4730f452a2e775fa91ad667adac6","user":{"login":"npm","id":6078720,"avatar_url":"https://avatars.githubusercontent.com/u/6078720?v=3","gravatar_id":"","url":"https://api.github.com/users/npm","html_url":"https://github.com/npm","followers_url":"https://api.github.com/users/npm/followers","following_url":"https://api.github.com/users/npm/following{/other_user}","gists_url":"https://api.github.com/users/npm/gists{/gist_id}","starred_url":"https://api.github.com/users/npm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/npm/subscriptions","organizations_url":"https://api.github.com/users/npm/orgs","repos_url":"https://api.github.com/users/npm/repos","events_url":"https://api.github.com/users/npm/events{/privacy}","received_events_url":"https://api.github.com/users/npm/received_events","type":"Organization","site_admin":false},"repo":{"id":321278,"name":"npm","full_name":"npm/npm","owner":{"login":"npm","id":6078720,"avatar_url":"https://avatars.githubusercontent.com/u/6078720?v=3","gravatar_id":"","url":"https://api.github.com/users/npm","html_url":"https://github.com/npm","followers_url":"https://api.github.com/users/npm/followers","following_url":"https://api.github.com/users/npm/following{/other_user}","gists_url":"https://api.github.com/users/npm/gists{/gist_id}","starred_url":"https://api.github.com/users/npm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/npm/subscriptions","organizations_url":"https://api.github.com/users/npm/orgs","repos_url":"https://api.github.com/users/npm/repos","events_url":"https://api.github.com/users/npm/events{/privacy}","received_events_url":"https://api.github.com/users/npm/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/npm/npm","description":"a package manager for javascript","fork":false,"url":"https://api.github.com/repos/npm/npm","forks_url":"https://api.github.com/repos/npm/npm/forks","keys_url":"https://api.github.com/repos/npm/npm/keys{/key_id}","collaborators_url":"https://api.github.com/repos/npm/npm/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/npm/npm/teams","hooks_url":"https://api.github.com/repos/npm/npm/hooks","issue_events_url":"https://api.github.com/repos/npm/npm/issues/events{/number}","events_url":"https://api.github.com/repos/npm/npm/events","assignees_url":"https://api.github.com/repos/npm/npm/assignees{/user}","branches_url":"https://api.github.com/repos/npm/npm/branches{/branch}","tags_url":"https://api.github.com/repos/npm/npm/tags","blobs_url":"https://api.github.com/repos/npm/npm/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/npm/npm/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/npm/npm/git/refs{/sha}","trees_url":"https://api.github.com/repos/npm/npm/git/trees{/sha}","statuses_url":"https://api.github.com/repos/npm/npm/statuses/{sha}","languages_url":"https://api.github.com/repos/npm/npm/languages","stargazers_url":"https://api.github.com/repos/npm/npm/stargazers","contributors_url":"https://api.github.com/repos/npm/npm/contributors","subscribers_url":"https://api.github.com/repos/npm/npm/subscribers","subscription_url":"https://api.github.com/repos/npm/npm/subscription","commits_url":"https://api.github.com/repos/npm/npm/commits{/sha}","git_commits_url":"https://api.github.com/repos/npm/npm/git/commits{/sha}","comments_url":"https://api.github.com/repos/npm/npm/comments{/number}","issue_comment_url":"https://api.github.com/repos/npm/npm/issues/comments{/number}","contents_url":"https://api.github.com/repos/npm/npm/contents/{+path}","compare_url":"https://api.github.com/repos/npm/npm/compare/{base}...{head}","merges_url":"https://api.github.com/repos/npm/npm/merges","archive_url":"https://api.github.com/repos/npm/npm/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/npm/npm/downloads","issues_url":"https://api.github.com/repos/npm/npm/issues{/number}","pulls_url":"https://api.github.com/repos/npm/npm/pulls{/number}","milestones_url":"https://api.github.com/repos/npm/npm/milestones{/number}","notifications_url":"https://api.github.com/repos/npm/npm/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/npm/npm/labels{/name}","releases_url":"https://api.github.com/repos/npm/npm/releases{/id}","deployments_url":"https://api.github.com/repos/npm/npm/deployments","created_at":"2009-09-29T17:21:24Z","updated_at":"2016-09-07T00:17:46Z","pushed_at":"2016-09-06T09:40:17Z","git_url":"git://github.com/npm/npm.git","ssh_url":"git@github.com:npm/npm.git","clone_url":"https://github.com/npm/npm.git","svn_url":"https://github.com/npm/npm","homepage":"http://www.npmjs.com/","size":50079,"stargazers_count":10220,"watchers_count":10220,"language":"JavaScript","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":2111,"mirror_url":null,"open_issues_count":2414,"forks":2111,"open_issues":2414,"watchers":10220,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/npm/npm/pulls/13746"},"html":{"href":"https://github.com/npm/npm/pull/13746"},"issue":{"href":"https://api.github.com/repos/npm/npm/issues/13746"},"comments":{"href":"https://api.github.com/repos/npm/npm/issues/13746/comments"},"review_comments":{"href":"https://api.github.com/repos/npm/npm/pulls/13746/comments"},"review_comment":{"href":"https://api.github.com/repos/npm/npm/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/npm/npm/pulls/13746/commits"},"statuses":{"href":"https://api.github.com/repos/npm/npm/statuses/8d36ba24b9b56c8c8b777b403eb92d0b5afe2c78"}}}}
{"actor":{"display_login":"iarna"}}