{"action":"created","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/26152","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/26152/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/26152/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/26152/events","html_url":"https://github.com/cockroachdb/cockroach/issues/26152","id":327181913,"number":26152,"title":"github.com/cockroachdb/cockroach/pkg/storage: TestRemoveRangeWithoutGC failed under stress","user":{"login":"cockroach-teamcity","id":20559710,"avatar_url":"https://avatars0.githubusercontent.com/u/20559710?v=4","gravatar_id":"","url":"https://api.github.com/users/cockroach-teamcity","html_url":"https://github.com/cockroach-teamcity","followers_url":"https://api.github.com/users/cockroach-teamcity/followers","following_url":"https://api.github.com/users/cockroach-teamcity/following{/other_user}","gists_url":"https://api.github.com/users/cockroach-teamcity/gists{/gist_id}","starred_url":"https://api.github.com/users/cockroach-teamcity/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cockroach-teamcity/subscriptions","organizations_url":"https://api.github.com/users/cockroach-teamcity/orgs","repos_url":"https://api.github.com/users/cockroach-teamcity/repos","events_url":"https://api.github.com/users/cockroach-teamcity/events{/privacy}","received_events_url":"https://api.github.com/users/cockroach-teamcity/received_events","type":"User","site_admin":false},"labels":[{"id":191587409,"url":"https://api.github.com/repos/cockroachdb/cockroach/labels/C-test-failure","name":"C-test-failure","color":"c5def5","default":false},{"id":335217860,"url":"https://api.github.com/repos/cockroachdb/cockroach/labels/O-robot","name":"O-robot","color":"666ad1","default":false}],"state":"open","locked":false,"assignee":{"login":"bdarnell","id":160562,"avatar_url":"https://avatars1.githubusercontent.com/u/160562?v=4","gravatar_id":"","url":"https://api.github.com/users/bdarnell","html_url":"https://github.com/bdarnell","followers_url":"https://api.github.com/users/bdarnell/followers","following_url":"https://api.github.com/users/bdarnell/following{/other_user}","gists_url":"https://api.github.com/users/bdarnell/gists{/gist_id}","starred_url":"https://api.github.com/users/bdarnell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bdarnell/subscriptions","organizations_url":"https://api.github.com/users/bdarnell/orgs","repos_url":"https://api.github.com/users/bdarnell/repos","events_url":"https://api.github.com/users/bdarnell/events{/privacy}","received_events_url":"https://api.github.com/users/bdarnell/received_events","type":"User","site_admin":false},"assignees":[{"login":"bdarnell","id":160562,"avatar_url":"https://avatars1.githubusercontent.com/u/160562?v=4","gravatar_id":"","url":"https://api.github.com/users/bdarnell","html_url":"https://github.com/bdarnell","followers_url":"https://api.github.com/users/bdarnell/followers","following_url":"https://api.github.com/users/bdarnell/following{/other_user}","gists_url":"https://api.github.com/users/bdarnell/gists{/gist_id}","starred_url":"https://api.github.com/users/bdarnell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bdarnell/subscriptions","organizations_url":"https://api.github.com/users/bdarnell/orgs","repos_url":"https://api.github.com/users/bdarnell/repos","events_url":"https://api.github.com/users/bdarnell/events{/privacy}","received_events_url":"https://api.github.com/users/bdarnell/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/13","html_url":"https://github.com/cockroachdb/cockroach/milestone/13","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/13/labels","id":2745639,"number":13,"title":"2.1","description":"","creator":{"login":"knz","id":642886,"avatar_url":"https://avatars3.githubusercontent.com/u/642886?v=4","gravatar_id":"","url":"https://api.github.com/users/knz","html_url":"https://github.com/knz","followers_url":"https://api.github.com/users/knz/followers","following_url":"https://api.github.com/users/knz/following{/other_user}","gists_url":"https://api.github.com/users/knz/gists{/gist_id}","starred_url":"https://api.github.com/users/knz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/knz/subscriptions","organizations_url":"https://api.github.com/users/knz/orgs","repos_url":"https://api.github.com/users/knz/repos","events_url":"https://api.github.com/users/knz/events{/privacy}","received_events_url":"https://api.github.com/users/knz/received_events","type":"User","site_admin":false},"open_issues":582,"closed_issues":479,"state":"open","created_at":"2017-09-05T15:30:49Z","updated_at":"2018-05-29T06:12:33Z","due_on":null,"closed_at":null},"comments":0,"created_at":"2018-05-29T06:12:32Z","updated_at":"2018-05-29T10:09:22Z","closed_at":null,"author_association":"MEMBER","body":"SHA: https://github.com/cockroachdb/cockroach/commits/915a266778aab15c98fc7e18639583fbe6532399\n\nParameters:\n```\nTAGS=\nGOFLAGS=\n```\n\nFailed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=683987&tab=buildLog\n\n```\nI180529 06:11:05.726454 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:05.726488 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:05.922077 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:05.922107 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:06.039732 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:06.039754 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:06.176797 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:06.177378 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:06.324543 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:06.324571 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:06.372419 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:06.372452 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:06.429558 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:06.429585 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:06.627661 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:06.627693 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:06.822689 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:06.822724 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:07.005167 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:07.005196 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:07.078176 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:07.078218 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:07.274045 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:07.274080 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:07.453959 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:07.453989 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:07.528500 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:07.528534 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:07.537027 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:07.537054 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:07.724386 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:07.724423 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:07.949861 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:07.949885 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:07.978854 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:07.978898 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:08.174727 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:08.174767 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:08.342976 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:08.342999 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:08.429206 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:08.429325 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:08.624913 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:08.624956 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:08.625210 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:08.625242 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:08.879706 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:08.879795 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:08.968782 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:08.968804 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:09.078011 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:09.078046 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:09.330698 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:09.330847 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:09.455390 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:09.455412 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nW180529 06:11:09.514176 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:09.514206 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:09.528328 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:09.528363 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:09.781224 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:09.781262 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:09.978870 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:09.978910 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:09.988224 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:09.988249 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:10.231729 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:10.231774 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:10.429226 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:10.429267 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:10.565539 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:10.565562 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nW180529 06:11:10.591470 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:10.591490 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:10.682059 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:10.682092 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:10.879573 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:10.879614 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:10.953366 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:10.953398 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:11.132418 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:11.132451 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:11.329953 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:11.330001 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:11.442411 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:11.442438 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nW180529 06:11:11.558929 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:11.558951 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:11.582740 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:11.582784 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:11.780274 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:11.780305 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:11.870340 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:11.870364 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:12.033038 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:12.033069 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:12.230574 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:12.230605 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:12.298726 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:12.298756 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:12.483409 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:12.483446 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:12.601130 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:12.601204 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:12.680898 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:12.680933 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:12.816428 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:12.816474 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:12.933858 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:12.933898 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:13.131253 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:13.131302 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:13.384170 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:13.384205 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:13.385166 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:13.385190 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nW180529 06:11:13.553898 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:13.553925 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:13.581676 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:13.581710 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:13.834672 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:13.834706 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:13.892385 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:13.892409 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:14.031999 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:14.032033 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:14.284980 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:14.285018 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:14.445862 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:14.445885 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nW180529 06:11:14.469461 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:14.469483 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:14.482313 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:14.482345 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:14.735298 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:14.735341 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:14.932592 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:14.932622 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:15.013945 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:15.013974 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:15.185685 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:15.185721 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:15.370241 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:15.370272 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:15.382886 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:15.382918 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:15.516566 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:15.516586 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:15.636476 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:15.636510 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:15.833191 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:15.833225 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:16.088340 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:16.088373 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:16.088661 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:16.088677 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:16.283546 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:16.283585 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:16.386526 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:16.386550 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:16.538674 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:16.538711 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:16.544486 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:16.544511 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:16.733878 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:16.733912 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:16.989035 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:16.989074 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:17.041896 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:17.041927 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:17.184197 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:17.184249 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:17.439358 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:17.439394 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:17.503538 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:17.503560 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nW180529 06:11:17.523342 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:17.523363 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:17.634523 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:17.634553 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:17.889682 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:17.889713 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:17.908214 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:17.908239 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:18.084922 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:18.084960 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:18.340003 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:18.340037 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:18.432456 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:18.432479 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:18.535258 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:18.535296 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:18.671239 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:18.671261 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nW180529 06:11:18.765471 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:18.765495 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:18.790296 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:18.790332 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:18.985603 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:18.985643 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:19.240639 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:19.240669 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:19.436000 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:19.436036 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:19.528000 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:19.528024 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:19.690984 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:19.691021 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:19.761873 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:19.761891 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nW180529 06:11:19.862683 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:19.862706 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:19.886478 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:19.886514 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:20.141374 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:20.141419 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:20.336813 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:20.336847 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:20.459601 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:20.459622 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:20.591683 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:20.591713 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:20.756104 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:20.756128 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:20.787465 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:20.787498 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:20.964510 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:20.964532 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:21.042008 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:21.042039 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:21.237768 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:21.237825 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:21.461078 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:21.461103 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:21.492505 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:21.492546 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:21.688185 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:21.688231 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:21.777627 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:21.777661 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nW180529 06:11:21.896572 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:21.896597 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:21.942868 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:21.942900 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:22.138504 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:22.138536 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:22.393178 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:22.393214 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:22.522905 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:22.522928 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:22.588860 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:22.588893 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:22.843505 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:22.843541 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:22.869834 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:22.869854 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nW180529 06:11:22.964102 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:22.964129 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:23.040705 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:23.040741 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:23.293873 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:23.293910 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:23.379057 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:23.379096 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:23.491052 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:23.491081 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:23.745096 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:23.745127 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:23.885504 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:23.885533 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nW180529 06:11:23.917405 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:23.917432 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:23.941357 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:23.941393 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:24.195391 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:24.195424 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:24.379365 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:24.379386 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:24.391670 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:24.391700 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:24.645666 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:24.645698 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:24.841965 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:24.842087 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:24.891947 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:24.891969 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nW180529 06:11:24.903554 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:24.903576 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:25.095939 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:25.095976 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:25.230045 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:25.230065 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:25.292528 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:25.292558 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:25.546245 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:25.546278 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:25.742828 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:25.742863 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:25.870947 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:25.870974 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nW180529 06:11:25.888976 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:25.889029 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:25.996656 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:25.996695 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:26.193153 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:26.193185 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:26.375638 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:26.375660 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:26.446977 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:26.447022 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:26.643495 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:26.643532 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:26.897292 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:26.897329 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:26.913963 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:26.914000 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nW180529 06:11:27.027713 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:27.027788 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:27.093828 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:27.093864 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:27.263055 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:27.263082 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:27.347551 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:27.347585 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:27.544205 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:27.544237 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:27.797848 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:27.797884 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:27.839556 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:27.839588 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nW180529 06:11:27.891460 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:27.891488 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nI180529 06:11:27.994513 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:27.994567 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:28.248132 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:28.248163 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:28.329381 82834 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:28.329405 82834 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip first range descriptor: r1 was not found\nI180529 06:11:28.444889 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:28.444941 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:28.698440 82685 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:28.698475 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nW180529 06:11:28.806577 82836 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:28.806600 82836 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip node liveness: r1 was not found\nW180529 06:11:28.856389 82835 storage/replica.go:5677  [s1,r1/?:/M{in-ax}] could not acquire lease for range gossip: r1 was not found\nW180529 06:11:28.856413 82835 storage/store.go:1546  [s1,r1/?:/M{in-ax}] could not gossip system config: r1 was not found\nI180529 06:11:28.895216 82840 storage/node_liveness.go:773  [hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (context finished during distsender send: context deadline exceeded)\nW180529 06:11:28.895248 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: context deadline exceeded\nI180529 06:11:28.975688 82842 storage/client_test.go:1237  [replicaGC,s1,r1/?:/M{in-ax}] test clock advanced to: 864009.000000134,0\nE180529 06:11:28.975706 82174 storage/queue.go:778  [replicaGC,s1,r1/?:/M{in-ax}] result is ambiguous (context finished during distsender send: context deadline exceeded)\nI180529 06:11:28.975843 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n266    [async] storage.multiTestContextKVTransport: calling next replica\n131    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.975870 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n265    [async] storage.multiTestContextKVTransport: calling next replica\n131    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.975896 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n264    [async] storage.multiTestContextKVTransport: calling next replica\n131    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.975928 84333 util/stop/stopper.go:471  quiescing; tasks left:\n133    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.975956 84333 util/stop/stopper.go:471  quiescing; tasks left:\n132    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.975979 84333 util/stop/stopper.go:471  quiescing; tasks left:\n131    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.976012 84333 util/stop/stopper.go:471  quiescing; tasks left:\n130    [async] kv.TxnCoordSender: aborting txn\nW180529 06:11:28.976062 82840 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: failed to send RPC: sending to all 1 replicas failed; last error: {(err: node unavailable; try another peer) <nil>}\nI180529 06:11:28.976111 84333 util/stop/stopper.go:471  quiescing; tasks left:\n129    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.976128 84333 util/stop/stopper.go:471  quiescing; tasks left:\n128    [async] kv.TxnCoordSender: aborting txn\nW180529 06:11:28.976209 82685 storage/node_liveness.go:465  [hb] failed node liveness heartbeat: failed to send RPC: sending to all 1 replicas failed; last error: {(err: node unavailable; try another peer) <nil>}\nI180529 06:11:28.976237 84333 util/stop/stopper.go:471  quiescing; tasks left:\n127    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.976251 84333 util/stop/stopper.go:471  quiescing; tasks left:\n126    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.976315 84333 util/stop/stopper.go:471  quiescing; tasks left:\n125    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.976371 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n244    [async] storage.multiTestContextKVTransport: calling next replica\n123    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976384 84333 util/stop/stopper.go:471  quiescing; tasks left:\n124    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.976395 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n244    [async] storage.multiTestContextKVTransport: calling next replica\n122    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976416 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n244    [async] storage.multiTestContextKVTransport: calling next replica\n121    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976436 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n242    [async] storage.multiTestContextKVTransport: calling next replica\n121    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976453 84333 util/stop/stopper.go:471  quiescing; tasks left:\n123    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.976464 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n241    [async] storage.multiTestContextKVTransport: calling next replica\n121    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976480 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n240    [async] storage.multiTestContextKVTransport: calling next replica\n121    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976498 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n239    [async] storage.multiTestContextKVTransport: calling next replica\n121    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976513 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n239    [async] storage.multiTestContextKVTransport: calling next replica\n120    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976529 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n238    [async] storage.multiTestContextKVTransport: calling next replica\n120    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976548 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n237    [async] storage.multiTestContextKVTransport: calling next replica\n120    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976562 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n237    [async] storage.multiTestContextKVTransport: calling next replica\n119    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976584 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n236    [async] storage.multiTestContextKVTransport: calling next replica\n118    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976602 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n235    [async] storage.multiTestContextKVTransport: calling next replica\n118    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976618 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n234    [async] storage.multiTestContextKVTransport: calling next replica\n118    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976634 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n233    [async] storage.multiTestContextKVTransport: calling next replica\n118    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976648 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n232    [async] storage.multiTestContextKVTransport: calling next replica\n118    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976675 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n231    [async] storage.multiTestContextKVTransport: calling next replica\n117    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976690 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n229    [async] storage.multiTestContextKVTransport: calling next replica\n117    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976719 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n229    [async] storage.multiTestContextKVTransport: calling next replica\n116    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976737 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n227    [async] storage.multiTestContextKVTransport: calling next replica\n116    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nI180529 06:11:28.976841 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      storage.Replica: acquiring lease to gossip\n227    [async] storage.multiTestContextKVTransport: calling next replica\n115    [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.pendingLeaseRequest: requesting lease\nW180529 06:11:28.978800 82678 storage/store.go:1546  [s2,r1/2:/M{in-ax}] could not gossip first range descriptor: [NotLeaseHolderError] r1: replica (n2,s2):2 not lease holder; lease holder unknown\nI180529 06:11:28.979006 82751 storage/client_test.go:1237  test clock advanced to: 864010.800000136,0\nW180529 06:11:28.979024 82679 storage/store.go:1546  [s2,r1/2:/M{in-ax}] could not gossip system config: [NotLeaseHolderError] r1: replica (n2,s2):2 not lease holder; lease holder unknown\nW180529 06:11:28.979082 82680 storage/store.go:1546  [s2,r1/2:/M{in-ax}] could not gossip node liveness: [NotLeaseHolderError] r1: replica (n2,s2):2 not lease holder; lease holder unknown\nI180529 06:11:28.979135 82752 storage/client_test.go:1237  test clock advanced to: 864012.600000138,0\nI180529 06:11:28.976466 84333 util/stop/stopper.go:471  quiescing; tasks left:\n122    [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.979499 84334 util/stop/stopper.go:471  quiescing; tasks left:\n9      [async] kv.TxnCoordSender: aborting txn\n4      [async] storage.multiTestContextKVTransport: calling next replica\nI180529 06:11:28.979542 84334 util/stop/stopper.go:471  quiescing; tasks left:\n6      [async] kv.TxnCoordSender: aborting txn\n2      [async] storage.multiTestContextKVTransport: calling next replica\nI180529 06:11:28.979563 84334 util/stop/stopper.go:471  quiescing; tasks left:\n5      [async] kv.TxnCoordSender: aborting txn\n2      [async] storage.multiTestContextKVTransport: calling next replica\nI180529 06:11:28.979577 84334 util/stop/stopper.go:471  quiescing; tasks left:\n5      [async] kv.TxnCoordSender: aborting txn\n1      [async] storage.multiTestContextKVTransport: calling next replica\nI180529 06:11:28.979594 84334 util/stop/stopper.go:471  quiescing; tasks left:\n4      [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.979617 84334 util/stop/stopper.go:471  quiescing; tasks left:\n3      [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.979642 84334 util/stop/stopper.go:471  quiescing; tasks left:\n2      [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.979664 84334 util/stop/stopper.go:471  quiescing; tasks left:\n1      [async] kv.TxnCoordSender: aborting txn\nI180529 06:11:28.979833 84333 util/stop/stopper.go:471  quiescing; tasks left:\n1      [async] kv.TxnCoordSender: aborting txn\nW180529 06:11:28.980477 82480 gossip/gossip.go:1293  [n2] no incoming or outgoing connections\nI180529 06:11:28.981663 84332 storage/engine/rocksdb.go:699  closing rocksdb instance at \"/go/src/github.com/cockroachdb/cockroach/artifacts/TestRemoveRangeWithoutGC302648156\"\nI180529 06:11:28.981953 84332 storage/engine/rocksdb.go:699  closing rocksdb instance at \"/go/src/github.com/cockroachdb/cockroach/artifacts/TestRemoveRangeWithoutGC966188299\"\n\tclient_raft_test.go:3311: expected replica to be missing; got <nil>\n```"},"comment":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/comments/392724673","html_url":"https://github.com/cockroachdb/cockroach/issues/26152#issuecomment-392724673","issue_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/26152","id":392724673,"user":{"login":"tschottdorf","id":5076964,"avatar_url":"https://avatars1.githubusercontent.com/u/5076964?v=4","gravatar_id":"","url":"https://api.github.com/users/tschottdorf","html_url":"https://github.com/tschottdorf","followers_url":"https://api.github.com/users/tschottdorf/followers","following_url":"https://api.github.com/users/tschottdorf/following{/other_user}","gists_url":"https://api.github.com/users/tschottdorf/gists{/gist_id}","starred_url":"https://api.github.com/users/tschottdorf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tschottdorf/subscriptions","organizations_url":"https://api.github.com/users/tschottdorf/orgs","repos_url":"https://api.github.com/users/tschottdorf/repos","events_url":"https://api.github.com/users/tschottdorf/events{/privacy}","received_events_url":"https://api.github.com/users/tschottdorf/received_events","type":"User","site_admin":false},"created_at":"2018-05-29T10:09:22Z","updated_at":"2018-05-29T10:09:22Z","author_association":"MEMBER","body":"This log looks pretty unhappy (death spiral?) but the test then fails cleanly with:\r\n\r\nhttps://github.com/cockroachdb/cockroach/blob/915a266778aab15c98fc7e18639583fbe6532399/pkg/storage/client_raft_test.go#L3311\r\n\r\n```go\r\nif _, err := mtc.stores[0].GetReplica(rangeID); !testutils.IsError(err, \"r[0-9]+ was not found\") {\r\n\t\tt.Fatalf(\"expected replica to be missing; got %v\", err)\r\n}\r\n```\r\n\r\nIt forces a GC queue run before that check. I assume that this run didn't manage to GC the replica (the command does not return an error)."}}
{"actor":{"display_login":"tschottdorf"}}