{"action":"created","comment":{"url":"https://api.github.com/repos/attic-labs/noms/pulls/comments/93768348","pull_request_review_id":14298678,"id":93768348,"diff_hunk":"@@ -313,16 +332,21 @@ func resBodyReader(res *http.Response) (reader io.ReadCloser) {\n }\n \n func (bhcs *httpBatchStore) SchedulePut(c chunks.Chunk, refHeight uint64, hints types.Hints) {\n-\tif !bhcs.unwrittenPuts.Insert(c, refHeight) {\n+\ttryCache := func(c chunks.Chunk) bool {\n+\t\tbhcs.cacheMu.RLock()\n+\t\tdefer bhcs.cacheMu.RUnlock()\n+\t\treturn bhcs.unwrittenPuts.Insert(c)\n+\t}\n+\tif !tryCache(c) {\n \t\treturn\n \t}\n \n \tbhcs.requestWg.Add(1)\n-\tbhcs.writeQueue <- writeRequest{c.Hash(), hints, false}\n+\tbhcs.writeQueue <- writeRequest{hints, false}","path":"go/datas/http_batch_store.go","position":120,"original_position":120,"commit_id":"527e3e82a288bac5afa428b1a1c367c596b1cfa8","original_commit_id":"527e3e82a288bac5afa428b1a1c367c596b1cfa8","user":{"login":"rafael-atticlabs","id":12848669,"avatar_url":"https://avatars.githubusercontent.com/u/12848669?v=3","gravatar_id":"","url":"https://api.github.com/users/rafael-atticlabs","html_url":"https://github.com/rafael-atticlabs","followers_url":"https://api.github.com/users/rafael-atticlabs/followers","following_url":"https://api.github.com/users/rafael-atticlabs/following{/other_user}","gists_url":"https://api.github.com/users/rafael-atticlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/rafael-atticlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rafael-atticlabs/subscriptions","organizations_url":"https://api.github.com/users/rafael-atticlabs/orgs","repos_url":"https://api.github.com/users/rafael-atticlabs/repos","events_url":"https://api.github.com/users/rafael-atticlabs/events{/privacy}","received_events_url":"https://api.github.com/users/rafael-atticlabs/received_events","type":"User","site_admin":false},"body":"I'm wondering whether the machinery around the write queue (the channel, the persistent goroutine that are servicing the write-queue and the flush queue) are just unneeded complexity now. this stuff is a hold over from the before time when we didn't have any validation and thus were writing in larger batches, but we haven't done that in a while.\r\n\r\ni think it's possible now to just have `SchedulePut` Put into the nbs (not care about whether the chunk was contained), and add its hints to the hint set, all inside the mutex, and have `Flush` just send everything over.\r\n\r\ni'm probably forgetting about some critical thing, right?\r\n\r\n\r\n","created_at":"2016-12-23T14:01:23Z","updated_at":"2016-12-23T14:15:25Z","html_url":"https://github.com/attic-labs/noms/pull/2983#discussion_r93768348","pull_request_url":"https://api.github.com/repos/attic-labs/noms/pulls/2983","_links":{"self":{"href":"https://api.github.com/repos/attic-labs/noms/pulls/comments/93768348"},"html":{"href":"https://github.com/attic-labs/noms/pull/2983#discussion_r93768348"},"pull_request":{"href":"https://api.github.com/repos/attic-labs/noms/pulls/2983"}}},"pull_request":{"url":"https://api.github.com/repos/attic-labs/noms/pulls/2983","id":99190280,"html_url":"https://github.com/attic-labs/noms/pull/2983","diff_url":"https://github.com/attic-labs/noms/pull/2983.diff","patch_url":"https://github.com/attic-labs/noms/pull/2983.patch","issue_url":"https://api.github.com/repos/attic-labs/noms/issues/2983","number":2983,"state":"open","locked":false,"title":"Modify httpBatchStore so that writing values maintains some locality","user":{"login":"cmasone-attic","id":13224090,"avatar_url":"https://avatars.githubusercontent.com/u/13224090?v=3","gravatar_id":"","url":"https://api.github.com/users/cmasone-attic","html_url":"https://github.com/cmasone-attic","followers_url":"https://api.github.com/users/cmasone-attic/followers","following_url":"https://api.github.com/users/cmasone-attic/following{/other_user}","gists_url":"https://api.github.com/users/cmasone-attic/gists{/gist_id}","starred_url":"https://api.github.com/users/cmasone-attic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmasone-attic/subscriptions","organizations_url":"https://api.github.com/users/cmasone-attic/orgs","repos_url":"https://api.github.com/users/cmasone-attic/repos","events_url":"https://api.github.com/users/cmasone-attic/events{/privacy}","received_events_url":"https://api.github.com/users/cmasone-attic/received_events","type":"User","site_admin":false},"body":"NBS benefits from related chunks being near one another. Initially,\r\nlet's use write-order as a proxy for \"related\".\r\n\r\nThis patch contains a pretty heinous hack to allow sync to continue\r\nputting chunks into httpBatchStore top-down without breaking\r\nserver-side validation. Work to fix this is tracked in #2982\r\n\r\nThis patch fixes #2968, at least for now","created_at":"2016-12-23T00:30:48Z","updated_at":"2016-12-23T14:15:25Z","closed_at":null,"merged_at":null,"merge_commit_sha":"e17973decf998285ffc877b31f7265504c3d1d63","assignee":{"login":"rafael-atticlabs","id":12848669,"avatar_url":"https://avatars.githubusercontent.com/u/12848669?v=3","gravatar_id":"","url":"https://api.github.com/users/rafael-atticlabs","html_url":"https://github.com/rafael-atticlabs","followers_url":"https://api.github.com/users/rafael-atticlabs/followers","following_url":"https://api.github.com/users/rafael-atticlabs/following{/other_user}","gists_url":"https://api.github.com/users/rafael-atticlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/rafael-atticlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rafael-atticlabs/subscriptions","organizations_url":"https://api.github.com/users/rafael-atticlabs/orgs","repos_url":"https://api.github.com/users/rafael-atticlabs/repos","events_url":"https://api.github.com/users/rafael-atticlabs/events{/privacy}","received_events_url":"https://api.github.com/users/rafael-atticlabs/received_events","type":"User","site_admin":false},"assignees":[{"login":"rafael-atticlabs","id":12848669,"avatar_url":"https://avatars.githubusercontent.com/u/12848669?v=3","gravatar_id":"","url":"https://api.github.com/users/rafael-atticlabs","html_url":"https://github.com/rafael-atticlabs","followers_url":"https://api.github.com/users/rafael-atticlabs/followers","following_url":"https://api.github.com/users/rafael-atticlabs/following{/other_user}","gists_url":"https://api.github.com/users/rafael-atticlabs/gists{/gist_id}","starred_url":"https://api.github.com/users/rafael-atticlabs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rafael-atticlabs/subscriptions","organizations_url":"https://api.github.com/users/rafael-atticlabs/orgs","repos_url":"https://api.github.com/users/rafael-atticlabs/repos","events_url":"https://api.github.com/users/rafael-atticlabs/events{/privacy}","received_events_url":"https://api.github.com/users/rafael-atticlabs/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/attic-labs/noms/pulls/2983/commits","review_comments_url":"https://api.github.com/repos/attic-labs/noms/pulls/2983/comments","review_comment_url":"https://api.github.com/repos/attic-labs/noms/pulls/comments{/number}","comments_url":"https://api.github.com/repos/attic-labs/noms/issues/2983/comments","statuses_url":"https://api.github.com/repos/attic-labs/noms/statuses/527e3e82a288bac5afa428b1a1c367c596b1cfa8","head":{"label":"cmasone-attic:issue2968","ref":"issue2968","sha":"527e3e82a288bac5afa428b1a1c367c596b1cfa8","user":{"login":"cmasone-attic","id":13224090,"avatar_url":"https://avatars.githubusercontent.com/u/13224090?v=3","gravatar_id":"","url":"https://api.github.com/users/cmasone-attic","html_url":"https://github.com/cmasone-attic","followers_url":"https://api.github.com/users/cmasone-attic/followers","following_url":"https://api.github.com/users/cmasone-attic/following{/other_user}","gists_url":"https://api.github.com/users/cmasone-attic/gists{/gist_id}","starred_url":"https://api.github.com/users/cmasone-attic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmasone-attic/subscriptions","organizations_url":"https://api.github.com/users/cmasone-attic/orgs","repos_url":"https://api.github.com/users/cmasone-attic/repos","events_url":"https://api.github.com/users/cmasone-attic/events{/privacy}","received_events_url":"https://api.github.com/users/cmasone-attic/received_events","type":"User","site_admin":false},"repo":{"id":64776588,"name":"noms","full_name":"cmasone-attic/noms","owner":{"login":"cmasone-attic","id":13224090,"avatar_url":"https://avatars.githubusercontent.com/u/13224090?v=3","gravatar_id":"","url":"https://api.github.com/users/cmasone-attic","html_url":"https://github.com/cmasone-attic","followers_url":"https://api.github.com/users/cmasone-attic/followers","following_url":"https://api.github.com/users/cmasone-attic/following{/other_user}","gists_url":"https://api.github.com/users/cmasone-attic/gists{/gist_id}","starred_url":"https://api.github.com/users/cmasone-attic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmasone-attic/subscriptions","organizations_url":"https://api.github.com/users/cmasone-attic/orgs","repos_url":"https://api.github.com/users/cmasone-attic/repos","events_url":"https://api.github.com/users/cmasone-attic/events{/privacy}","received_events_url":"https://api.github.com/users/cmasone-attic/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/cmasone-attic/noms","description":"The versioned, forkable, syncable database","fork":true,"url":"https://api.github.com/repos/cmasone-attic/noms","forks_url":"https://api.github.com/repos/cmasone-attic/noms/forks","keys_url":"https://api.github.com/repos/cmasone-attic/noms/keys{/key_id}","collaborators_url":"https://api.github.com/repos/cmasone-attic/noms/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/cmasone-attic/noms/teams","hooks_url":"https://api.github.com/repos/cmasone-attic/noms/hooks","issue_events_url":"https://api.github.com/repos/cmasone-attic/noms/issues/events{/number}","events_url":"https://api.github.com/repos/cmasone-attic/noms/events","assignees_url":"https://api.github.com/repos/cmasone-attic/noms/assignees{/user}","branches_url":"https://api.github.com/repos/cmasone-attic/noms/branches{/branch}","tags_url":"https://api.github.com/repos/cmasone-attic/noms/tags","blobs_url":"https://api.github.com/repos/cmasone-attic/noms/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/cmasone-attic/noms/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/cmasone-attic/noms/git/refs{/sha}","trees_url":"https://api.github.com/repos/cmasone-attic/noms/git/trees{/sha}","statuses_url":"https://api.github.com/repos/cmasone-attic/noms/statuses/{sha}","languages_url":"https://api.github.com/repos/cmasone-attic/noms/languages","stargazers_url":"https://api.github.com/repos/cmasone-attic/noms/stargazers","contributors_url":"https://api.github.com/repos/cmasone-attic/noms/contributors","subscribers_url":"https://api.github.com/repos/cmasone-attic/noms/subscribers","subscription_url":"https://api.github.com/repos/cmasone-attic/noms/subscription","commits_url":"https://api.github.com/repos/cmasone-attic/noms/commits{/sha}","git_commits_url":"https://api.github.com/repos/cmasone-attic/noms/git/commits{/sha}","comments_url":"https://api.github.com/repos/cmasone-attic/noms/comments{/number}","issue_comment_url":"https://api.github.com/repos/cmasone-attic/noms/issues/comments{/number}","contents_url":"https://api.github.com/repos/cmasone-attic/noms/contents/{+path}","compare_url":"https://api.github.com/repos/cmasone-attic/noms/compare/{base}...{head}","merges_url":"https://api.github.com/repos/cmasone-attic/noms/merges","archive_url":"https://api.github.com/repos/cmasone-attic/noms/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/cmasone-attic/noms/downloads","issues_url":"https://api.github.com/repos/cmasone-attic/noms/issues{/number}","pulls_url":"https://api.github.com/repos/cmasone-attic/noms/pulls{/number}","milestones_url":"https://api.github.com/repos/cmasone-attic/noms/milestones{/number}","notifications_url":"https://api.github.com/repos/cmasone-attic/noms/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/cmasone-attic/noms/labels{/name}","releases_url":"https://api.github.com/repos/cmasone-attic/noms/releases{/id}","deployments_url":"https://api.github.com/repos/cmasone-attic/noms/deployments","created_at":"2016-08-02T17:13:26Z","updated_at":"2016-08-02T17:13:27Z","pushed_at":"2016-12-23T00:27:46Z","git_url":"git://github.com/cmasone-attic/noms.git","ssh_url":"git@github.com:cmasone-attic/noms.git","clone_url":"https://github.com/cmasone-attic/noms.git","svn_url":"https://github.com/cmasone-attic/noms","homepage":"","size":28969,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"attic-labs:master","ref":"master","sha":"ed1e386e2436f1c810682a087e4bbe313795d9d3","user":{"login":"attic-labs","id":12534679,"avatar_url":"https://avatars.githubusercontent.com/u/12534679?v=3","gravatar_id":"","url":"https://api.github.com/users/attic-labs","html_url":"https://github.com/attic-labs","followers_url":"https://api.github.com/users/attic-labs/followers","following_url":"https://api.github.com/users/attic-labs/following{/other_user}","gists_url":"https://api.github.com/users/attic-labs/gists{/gist_id}","starred_url":"https://api.github.com/users/attic-labs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/attic-labs/subscriptions","organizations_url":"https://api.github.com/users/attic-labs/orgs","repos_url":"https://api.github.com/users/attic-labs/repos","events_url":"https://api.github.com/users/attic-labs/events{/privacy}","received_events_url":"https://api.github.com/users/attic-labs/received_events","type":"Organization","site_admin":false},"repo":{"id":36778364,"name":"noms","full_name":"attic-labs/noms","owner":{"login":"attic-labs","id":12534679,"avatar_url":"https://avatars.githubusercontent.com/u/12534679?v=3","gravatar_id":"","url":"https://api.github.com/users/attic-labs","html_url":"https://github.com/attic-labs","followers_url":"https://api.github.com/users/attic-labs/followers","following_url":"https://api.github.com/users/attic-labs/following{/other_user}","gists_url":"https://api.github.com/users/attic-labs/gists{/gist_id}","starred_url":"https://api.github.com/users/attic-labs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/attic-labs/subscriptions","organizations_url":"https://api.github.com/users/attic-labs/orgs","repos_url":"https://api.github.com/users/attic-labs/repos","events_url":"https://api.github.com/users/attic-labs/events{/privacy}","received_events_url":"https://api.github.com/users/attic-labs/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/attic-labs/noms","description":"The versioned, forkable, syncable database","fork":false,"url":"https://api.github.com/repos/attic-labs/noms","forks_url":"https://api.github.com/repos/attic-labs/noms/forks","keys_url":"https://api.github.com/repos/attic-labs/noms/keys{/key_id}","collaborators_url":"https://api.github.com/repos/attic-labs/noms/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/attic-labs/noms/teams","hooks_url":"https://api.github.com/repos/attic-labs/noms/hooks","issue_events_url":"https://api.github.com/repos/attic-labs/noms/issues/events{/number}","events_url":"https://api.github.com/repos/attic-labs/noms/events","assignees_url":"https://api.github.com/repos/attic-labs/noms/assignees{/user}","branches_url":"https://api.github.com/repos/attic-labs/noms/branches{/branch}","tags_url":"https://api.github.com/repos/attic-labs/noms/tags","blobs_url":"https://api.github.com/repos/attic-labs/noms/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/attic-labs/noms/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/attic-labs/noms/git/refs{/sha}","trees_url":"https://api.github.com/repos/attic-labs/noms/git/trees{/sha}","statuses_url":"https://api.github.com/repos/attic-labs/noms/statuses/{sha}","languages_url":"https://api.github.com/repos/attic-labs/noms/languages","stargazers_url":"https://api.github.com/repos/attic-labs/noms/stargazers","contributors_url":"https://api.github.com/repos/attic-labs/noms/contributors","subscribers_url":"https://api.github.com/repos/attic-labs/noms/subscribers","subscription_url":"https://api.github.com/repos/attic-labs/noms/subscription","commits_url":"https://api.github.com/repos/attic-labs/noms/commits{/sha}","git_commits_url":"https://api.github.com/repos/attic-labs/noms/git/commits{/sha}","comments_url":"https://api.github.com/repos/attic-labs/noms/comments{/number}","issue_comment_url":"https://api.github.com/repos/attic-labs/noms/issues/comments{/number}","contents_url":"https://api.github.com/repos/attic-labs/noms/contents/{+path}","compare_url":"https://api.github.com/repos/attic-labs/noms/compare/{base}...{head}","merges_url":"https://api.github.com/repos/attic-labs/noms/merges","archive_url":"https://api.github.com/repos/attic-labs/noms/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/attic-labs/noms/downloads","issues_url":"https://api.github.com/repos/attic-labs/noms/issues{/number}","pulls_url":"https://api.github.com/repos/attic-labs/noms/pulls{/number}","milestones_url":"https://api.github.com/repos/attic-labs/noms/milestones{/number}","notifications_url":"https://api.github.com/repos/attic-labs/noms/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/attic-labs/noms/labels{/name}","releases_url":"https://api.github.com/repos/attic-labs/noms/releases{/id}","deployments_url":"https://api.github.com/repos/attic-labs/noms/deployments","created_at":"2015-06-03T03:44:49Z","updated_at":"2016-12-23T11:33:45Z","pushed_at":"2016-12-23T08:20:30Z","git_url":"git://github.com/attic-labs/noms.git","ssh_url":"git@github.com:attic-labs/noms.git","clone_url":"https://github.com/attic-labs/noms.git","svn_url":"https://github.com/attic-labs/noms","homepage":"","size":28962,"stargazers_count":5921,"watchers_count":5921,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":173,"mirror_url":null,"open_issues_count":263,"forks":173,"open_issues":263,"watchers":5921,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/attic-labs/noms/pulls/2983"},"html":{"href":"https://github.com/attic-labs/noms/pull/2983"},"issue":{"href":"https://api.github.com/repos/attic-labs/noms/issues/2983"},"comments":{"href":"https://api.github.com/repos/attic-labs/noms/issues/2983/comments"},"review_comments":{"href":"https://api.github.com/repos/attic-labs/noms/pulls/2983/comments"},"review_comment":{"href":"https://api.github.com/repos/attic-labs/noms/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/attic-labs/noms/pulls/2983/commits"},"statuses":{"href":"https://api.github.com/repos/attic-labs/noms/statuses/527e3e82a288bac5afa428b1a1c367c596b1cfa8"}}}}
{"actor":{"display_login":"rafael-atticlabs"}}