{"action":"opened","number":27706,"pull_request":{"url":"https://api.github.com/repos/cockroachdb/cockroach/pulls/27706","id":202278965,"node_id":"MDExOlB1bGxSZXF1ZXN0MjAyMjc4OTY1","html_url":"https://github.com/cockroachdb/cockroach/pull/27706","diff_url":"https://github.com/cockroachdb/cockroach/pull/27706.diff","patch_url":"https://github.com/cockroachdb/cockroach/pull/27706.patch","issue_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/27706","number":27706,"state":"open","locked":false,"title":"distsql: refactor hashJoiner to implement RowSource","user":{"login":"asubiotto","id":10560359,"node_id":"MDQ6VXNlcjEwNTYwMzU5","avatar_url":"https://avatars2.githubusercontent.com/u/10560359?v=4","gravatar_id":"","url":"https://api.github.com/users/asubiotto","html_url":"https://github.com/asubiotto","followers_url":"https://api.github.com/users/asubiotto/followers","following_url":"https://api.github.com/users/asubiotto/following{/other_user}","gists_url":"https://api.github.com/users/asubiotto/gists{/gist_id}","starred_url":"https://api.github.com/users/asubiotto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asubiotto/subscriptions","organizations_url":"https://api.github.com/users/asubiotto/orgs","repos_url":"https://api.github.com/users/asubiotto/repos","events_url":"https://api.github.com/users/asubiotto/events{/privacy}","received_events_url":"https://api.github.com/users/asubiotto/received_events","type":"User","site_admin":false},"body":"Closes #26843\r\n\r\nThis change includes some additional improvements:\r\n    - Construct a hashRowContainer early. We would previously consume a\r\n      whole side in the buffer phase then construct a hashRowContainer,\r\n      meaning we would have to re-encode datums to construct buckets.\r\n      If we know we will store a certain side and have not yet fully\r\n      consumed that side, we can avoid this re-encoding of all remaining\r\n      rows by directly inserting into a hashRowContainer.\r\n    - Remove unnecessary copies. During a check for nulls in equality\r\n      columns, we would previously copy a row if it had no nulls to be\r\n      used later. This copy is unnecessary as rows are already copied by\r\n      the underlying row container.\r\n    - Fix a potential memory error when building the sored side.\r\n\r\nRelease note: None\r\n\r\nMicrobenchmarks show a significant improvement in allocations but nothing really in latency (expected since the microbenchmark doesn't use `RowChannel`s):\r\n```\r\nBENCHES=BenchmarkHashJoiner 'TESTFLAGS=-count 10 -benchmem'\r\nname                                 old time/op    new time/op    delta\r\nHashJoiner/spill=true/rows=256-8       4.72ms ±59%    4.52ms ±62%     ~     (p=0.796 n=10+10)\r\nHashJoiner/spill=true/rows=4096-8      44.1ms ±53%    44.3ms ±50%     ~     (p=1.000 n=10+10)\r\nHashJoiner/spill=true/rows=65536-8      262ms ± 9%     242ms ± 8%   -7.83%  (p=0.011 n=9+9)\r\nHashJoiner/spill=false/rows=0-8        6.08µs ±15%    3.19µs ± 4%  -47.58%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=4-8        9.41µs ± 9%    9.67µs ± 8%     ~     (p=0.218 n=10+10)\r\nHashJoiner/spill=false/rows=16-8       19.2µs ± 4%    19.3µs ± 4%     ~     (p=0.739 n=10+10)\r\nHashJoiner/spill=false/rows=256-8       218µs ± 2%     217µs ± 2%     ~     (p=0.631 n=10+10)\r\nHashJoiner/spill=false/rows=4096-8     3.49ms ± 1%    3.51ms ± 2%     ~     (p=0.247 n=10+10)\r\nHashJoiner/spill=false/rows=65536-8    66.5ms ± 6%    67.5ms ± 2%     ~     (p=0.190 n=10+10)\r\n\r\nname                                 old speed      new speed      delta\r\nHashJoiner/spill=true/rows=256-8      847kB/s ±49%   911kB/s ±59%     ~     (p=0.779 n=9+9)\r\nHashJoiner/spill=true/rows=4096-8    1.68MB/s ±87%  1.63MB/s ±80%     ~     (p=0.988 n=10+10)\r\nHashJoiner/spill=true/rows=65536-8   4.01MB/s ± 9%  4.35MB/s ± 7%   +8.31%  (p=0.011 n=9+9)\r\nHashJoiner/spill=false/rows=4-8      6.82MB/s ± 8%  6.64MB/s ± 8%     ~     (p=0.225 n=10+10)\r\nHashJoiner/spill=false/rows=16-8     13.3MB/s ± 4%  13.2MB/s ± 4%     ~     (p=0.697 n=10+10)\r\nHashJoiner/spill=false/rows=256-8    18.8MB/s ± 2%  18.8MB/s ± 2%     ~     (p=0.591 n=10+10)\r\nHashJoiner/spill=false/rows=4096-8   18.8MB/s ± 1%  18.7MB/s ± 2%     ~     (p=0.256 n=10+10)\r\nHashJoiner/spill=false/rows=65536-8  15.8MB/s ± 6%  15.5MB/s ± 2%     ~     (p=0.190 n=10+10)\r\n\r\nname                                 old alloc/op   new alloc/op   delta\r\nHashJoiner/spill=true/rows=256-8       45.4kB ± 0%    22.6kB ± 0%  -50.25%  (p=0.000 n=9+7)\r\nHashJoiner/spill=true/rows=4096-8       790kB ± 0%     399kB ± 0%  -49.57%  (p=0.000 n=10+9)\r\nHashJoiner/spill=true/rows=65536-8     12.7MB ± 0%     6.5MB ± 0%  -49.34%  (p=0.000 n=9+9)\r\nHashJoiner/spill=false/rows=0-8        4.69kB ± 0%    4.90kB ± 0%   +4.44%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=4-8        8.05kB ± 0%    8.28kB ± 0%   +2.80%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=16-8       11.2kB ± 0%    10.7kB ± 0%   -4.85%  (p=0.000 n=8+7)\r\nHashJoiner/spill=false/rows=256-8      92.5kB ± 0%    68.9kB ± 0%  -25.51%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=4096-8     1.39MB ± 0%    1.00MB ± 0%  -28.21%  (p=0.000 n=9+10)\r\nHashJoiner/spill=false/rows=65536-8    22.3MB ± 0%    16.0MB ± 0%  -28.26%  (p=0.000 n=10+10)\r\n\r\nname                                 old allocs/op  new allocs/op  delta\r\nHashJoiner/spill=true/rows=256-8          850 ± 0%       826 ± 0%   -2.82%  (p=0.000 n=10+10)\r\nHashJoiner/spill=true/rows=4096-8       12.9k ± 0%     12.4k ± 0%   -3.91%  (p=0.000 n=10+10)\r\nHashJoiner/spill=true/rows=65536-8       206k ± 0%      198k ± 0%   -3.97%  (p=0.000 n=10+9)\r\nHashJoiner/spill=false/rows=0-8          18.0 ± 0%      21.0 ± 0%  +16.67%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=4-8          35.0 ± 0%      39.0 ± 0%  +11.43%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=16-8         62.0 ± 0%      65.0 ± 0%   +4.84%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=256-8         593 ± 0%       566 ± 0%   -4.55%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=4096-8      8.89k ± 0%     8.38k ± 0%   -5.70%  (p=0.000 n=10+10)\r\nHashJoiner/spill=false/rows=65536-8      142k ± 0%      134k ± 0%   -5.78%  (p=0.000 n=10+10)\r\n```\r\n\r\nLarge-scale tests with a 4-node cluster don't really show an improvement which is kind of surprising because the query I was testing with was a `hashJoin` followed by an `aggregator` on each node. This change should remove a `RowChannel` between the `hashJoiner` and `aggregator` in this case. Might look more into this.","created_at":"2018-07-18T14:47:39Z","updated_at":"2018-07-18T14:47:39Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/cockroachdb/cockroach/pulls/27706/commits","review_comments_url":"https://api.github.com/repos/cockroachdb/cockroach/pulls/27706/comments","review_comment_url":"https://api.github.com/repos/cockroachdb/cockroach/pulls/comments{/number}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/27706/comments","statuses_url":"https://api.github.com/repos/cockroachdb/cockroach/statuses/3a9df839d70cd922b6591b84083348760e4bfd3d","head":{"label":"asubiotto:hjrs","ref":"hjrs","sha":"3a9df839d70cd922b6591b84083348760e4bfd3d","user":{"login":"asubiotto","id":10560359,"node_id":"MDQ6VXNlcjEwNTYwMzU5","avatar_url":"https://avatars2.githubusercontent.com/u/10560359?v=4","gravatar_id":"","url":"https://api.github.com/users/asubiotto","html_url":"https://github.com/asubiotto","followers_url":"https://api.github.com/users/asubiotto/followers","following_url":"https://api.github.com/users/asubiotto/following{/other_user}","gists_url":"https://api.github.com/users/asubiotto/gists{/gist_id}","starred_url":"https://api.github.com/users/asubiotto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asubiotto/subscriptions","organizations_url":"https://api.github.com/users/asubiotto/orgs","repos_url":"https://api.github.com/users/asubiotto/repos","events_url":"https://api.github.com/users/asubiotto/events{/privacy}","received_events_url":"https://api.github.com/users/asubiotto/received_events","type":"User","site_admin":false},"repo":{"id":53750588,"node_id":"MDEwOlJlcG9zaXRvcnk1Mzc1MDU4OA==","name":"cockroach","full_name":"asubiotto/cockroach","owner":{"login":"asubiotto","id":10560359,"node_id":"MDQ6VXNlcjEwNTYwMzU5","avatar_url":"https://avatars2.githubusercontent.com/u/10560359?v=4","gravatar_id":"","url":"https://api.github.com/users/asubiotto","html_url":"https://github.com/asubiotto","followers_url":"https://api.github.com/users/asubiotto/followers","following_url":"https://api.github.com/users/asubiotto/following{/other_user}","gists_url":"https://api.github.com/users/asubiotto/gists{/gist_id}","starred_url":"https://api.github.com/users/asubiotto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asubiotto/subscriptions","organizations_url":"https://api.github.com/users/asubiotto/orgs","repos_url":"https://api.github.com/users/asubiotto/repos","events_url":"https://api.github.com/users/asubiotto/events{/privacy}","received_events_url":"https://api.github.com/users/asubiotto/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/asubiotto/cockroach","description":"A Scalable, Survivable, Strongly-Consistent SQL Database","fork":true,"url":"https://api.github.com/repos/asubiotto/cockroach","forks_url":"https://api.github.com/repos/asubiotto/cockroach/forks","keys_url":"https://api.github.com/repos/asubiotto/cockroach/keys{/key_id}","collaborators_url":"https://api.github.com/repos/asubiotto/cockroach/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/asubiotto/cockroach/teams","hooks_url":"https://api.github.com/repos/asubiotto/cockroach/hooks","issue_events_url":"https://api.github.com/repos/asubiotto/cockroach/issues/events{/number}","events_url":"https://api.github.com/repos/asubiotto/cockroach/events","assignees_url":"https://api.github.com/repos/asubiotto/cockroach/assignees{/user}","branches_url":"https://api.github.com/repos/asubiotto/cockroach/branches{/branch}","tags_url":"https://api.github.com/repos/asubiotto/cockroach/tags","blobs_url":"https://api.github.com/repos/asubiotto/cockroach/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/asubiotto/cockroach/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/asubiotto/cockroach/git/refs{/sha}","trees_url":"https://api.github.com/repos/asubiotto/cockroach/git/trees{/sha}","statuses_url":"https://api.github.com/repos/asubiotto/cockroach/statuses/{sha}","languages_url":"https://api.github.com/repos/asubiotto/cockroach/languages","stargazers_url":"https://api.github.com/repos/asubiotto/cockroach/stargazers","contributors_url":"https://api.github.com/repos/asubiotto/cockroach/contributors","subscribers_url":"https://api.github.com/repos/asubiotto/cockroach/subscribers","subscription_url":"https://api.github.com/repos/asubiotto/cockroach/subscription","commits_url":"https://api.github.com/repos/asubiotto/cockroach/commits{/sha}","git_commits_url":"https://api.github.com/repos/asubiotto/cockroach/git/commits{/sha}","comments_url":"https://api.github.com/repos/asubiotto/cockroach/comments{/number}","issue_comment_url":"https://api.github.com/repos/asubiotto/cockroach/issues/comments{/number}","contents_url":"https://api.github.com/repos/asubiotto/cockroach/contents/{+path}","compare_url":"https://api.github.com/repos/asubiotto/cockroach/compare/{base}...{head}","merges_url":"https://api.github.com/repos/asubiotto/cockroach/merges","archive_url":"https://api.github.com/repos/asubiotto/cockroach/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/asubiotto/cockroach/downloads","issues_url":"https://api.github.com/repos/asubiotto/cockroach/issues{/number}","pulls_url":"https://api.github.com/repos/asubiotto/cockroach/pulls{/number}","milestones_url":"https://api.github.com/repos/asubiotto/cockroach/milestones{/number}","notifications_url":"https://api.github.com/repos/asubiotto/cockroach/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/asubiotto/cockroach/labels{/name}","releases_url":"https://api.github.com/repos/asubiotto/cockroach/releases{/id}","deployments_url":"https://api.github.com/repos/asubiotto/cockroach/deployments","created_at":"2016-03-12T19:54:46Z","updated_at":"2018-06-21T20:21:54Z","pushed_at":"2018-07-18T14:34:25Z","git_url":"git://github.com/asubiotto/cockroach.git","ssh_url":"git@github.com:asubiotto/cockroach.git","clone_url":"https://github.com/asubiotto/cockroach.git","svn_url":"https://github.com/asubiotto/cockroach","homepage":"http://www.cockroachlabs.com","size":519332,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"cockroachdb:master","ref":"master","sha":"6a9584b1bc546a46194103972f97677431bcfe98","user":{"login":"cockroachdb","id":6748139,"node_id":"MDEyOk9yZ2FuaXphdGlvbjY3NDgxMzk=","avatar_url":"https://avatars3.githubusercontent.com/u/6748139?v=4","gravatar_id":"","url":"https://api.github.com/users/cockroachdb","html_url":"https://github.com/cockroachdb","followers_url":"https://api.github.com/users/cockroachdb/followers","following_url":"https://api.github.com/users/cockroachdb/following{/other_user}","gists_url":"https://api.github.com/users/cockroachdb/gists{/gist_id}","starred_url":"https://api.github.com/users/cockroachdb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cockroachdb/subscriptions","organizations_url":"https://api.github.com/users/cockroachdb/orgs","repos_url":"https://api.github.com/users/cockroachdb/repos","events_url":"https://api.github.com/users/cockroachdb/events{/privacy}","received_events_url":"https://api.github.com/users/cockroachdb/received_events","type":"Organization","site_admin":false},"repo":{"id":16563587,"node_id":"MDEwOlJlcG9zaXRvcnkxNjU2MzU4Nw==","name":"cockroach","full_name":"cockroachdb/cockroach","owner":{"login":"cockroachdb","id":6748139,"node_id":"MDEyOk9yZ2FuaXphdGlvbjY3NDgxMzk=","avatar_url":"https://avatars3.githubusercontent.com/u/6748139?v=4","gravatar_id":"","url":"https://api.github.com/users/cockroachdb","html_url":"https://github.com/cockroachdb","followers_url":"https://api.github.com/users/cockroachdb/followers","following_url":"https://api.github.com/users/cockroachdb/following{/other_user}","gists_url":"https://api.github.com/users/cockroachdb/gists{/gist_id}","starred_url":"https://api.github.com/users/cockroachdb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cockroachdb/subscriptions","organizations_url":"https://api.github.com/users/cockroachdb/orgs","repos_url":"https://api.github.com/users/cockroachdb/repos","events_url":"https://api.github.com/users/cockroachdb/events{/privacy}","received_events_url":"https://api.github.com/users/cockroachdb/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/cockroachdb/cockroach","description":"CockroachDB - the open source, cloud-native SQL database.","fork":false,"url":"https://api.github.com/repos/cockroachdb/cockroach","forks_url":"https://api.github.com/repos/cockroachdb/cockroach/forks","keys_url":"https://api.github.com/repos/cockroachdb/cockroach/keys{/key_id}","collaborators_url":"https://api.github.com/repos/cockroachdb/cockroach/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/cockroachdb/cockroach/teams","hooks_url":"https://api.github.com/repos/cockroachdb/cockroach/hooks","issue_events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/events{/number}","events_url":"https://api.github.com/repos/cockroachdb/cockroach/events","assignees_url":"https://api.github.com/repos/cockroachdb/cockroach/assignees{/user}","branches_url":"https://api.github.com/repos/cockroachdb/cockroach/branches{/branch}","tags_url":"https://api.github.com/repos/cockroachdb/cockroach/tags","blobs_url":"https://api.github.com/repos/cockroachdb/cockroach/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/cockroachdb/cockroach/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/cockroachdb/cockroach/git/refs{/sha}","trees_url":"https://api.github.com/repos/cockroachdb/cockroach/git/trees{/sha}","statuses_url":"https://api.github.com/repos/cockroachdb/cockroach/statuses/{sha}","languages_url":"https://api.github.com/repos/cockroachdb/cockroach/languages","stargazers_url":"https://api.github.com/repos/cockroachdb/cockroach/stargazers","contributors_url":"https://api.github.com/repos/cockroachdb/cockroach/contributors","subscribers_url":"https://api.github.com/repos/cockroachdb/cockroach/subscribers","subscription_url":"https://api.github.com/repos/cockroachdb/cockroach/subscription","commits_url":"https://api.github.com/repos/cockroachdb/cockroach/commits{/sha}","git_commits_url":"https://api.github.com/repos/cockroachdb/cockroach/git/commits{/sha}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/comments{/number}","issue_comment_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/comments{/number}","contents_url":"https://api.github.com/repos/cockroachdb/cockroach/contents/{+path}","compare_url":"https://api.github.com/repos/cockroachdb/cockroach/compare/{base}...{head}","merges_url":"https://api.github.com/repos/cockroachdb/cockroach/merges","archive_url":"https://api.github.com/repos/cockroachdb/cockroach/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/cockroachdb/cockroach/downloads","issues_url":"https://api.github.com/repos/cockroachdb/cockroach/issues{/number}","pulls_url":"https://api.github.com/repos/cockroachdb/cockroach/pulls{/number}","milestones_url":"https://api.github.com/repos/cockroachdb/cockroach/milestones{/number}","notifications_url":"https://api.github.com/repos/cockroachdb/cockroach/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/labels{/name}","releases_url":"https://api.github.com/repos/cockroachdb/cockroach/releases{/id}","deployments_url":"https://api.github.com/repos/cockroachdb/cockroach/deployments","created_at":"2014-02-06T00:18:47Z","updated_at":"2018-07-18T14:45:15Z","pushed_at":"2018-07-18T14:46:03Z","git_url":"git://github.com/cockroachdb/cockroach.git","ssh_url":"git@github.com:cockroachdb/cockroach.git","clone_url":"https://github.com/cockroachdb/cockroach.git","svn_url":"https://github.com/cockroachdb/cockroach","homepage":"https://www.cockroachlabs.com","size":1023525,"stargazers_count":13830,"watchers_count":13830,"language":"Go","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1444,"mirror_url":null,"archived":false,"open_issues_count":1921,"license":{"key":"other","name":"Other","spdx_id":null,"url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":1444,"open_issues":1921,"watchers":13830,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/cockroachdb/cockroach/pulls/27706"},"html":{"href":"https://github.com/cockroachdb/cockroach/pull/27706"},"issue":{"href":"https://api.github.com/repos/cockroachdb/cockroach/issues/27706"},"comments":{"href":"https://api.github.com/repos/cockroachdb/cockroach/issues/27706/comments"},"review_comments":{"href":"https://api.github.com/repos/cockroachdb/cockroach/pulls/27706/comments"},"review_comment":{"href":"https://api.github.com/repos/cockroachdb/cockroach/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/cockroachdb/cockroach/pulls/27706/commits"},"statuses":{"href":"https://api.github.com/repos/cockroachdb/cockroach/statuses/3a9df839d70cd922b6591b84083348760e4bfd3d"}},"author_association":"CONTRIBUTOR","merged":false,"mergeable":null,"rebaseable":null,"mergeable_state":"unknown","merged_by":null,"comments":0,"review_comments":0,"maintainer_can_modify":true,"commits":1,"additions":589,"deletions":544,"changed_files":2}}
{"actor":{"display_login":"asubiotto"}}