{"action":"opened","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/11627","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/11627/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/11627/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/11627/events","html_url":"https://github.com/cockroachdb/cockroach/issues/11627","id":191797774,"number":11627,"title":"github.com/cockroachdb/cockroach/pkg/storage: TestReplicateQueueRebalance failed under stress","user":{"login":"cockroach-teamcity","id":20559710,"avatar_url":"https://avatars.githubusercontent.com/u/20559710?v=3","gravatar_id":"","url":"https://api.github.com/users/cockroach-teamcity","html_url":"https://github.com/cockroach-teamcity","followers_url":"https://api.github.com/users/cockroach-teamcity/followers","following_url":"https://api.github.com/users/cockroach-teamcity/following{/other_user}","gists_url":"https://api.github.com/users/cockroach-teamcity/gists{/gist_id}","starred_url":"https://api.github.com/users/cockroach-teamcity/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cockroach-teamcity/subscriptions","organizations_url":"https://api.github.com/users/cockroach-teamcity/orgs","repos_url":"https://api.github.com/users/cockroach-teamcity/repos","events_url":"https://api.github.com/users/cockroach-teamcity/events{/privacy}","received_events_url":"https://api.github.com/users/cockroach-teamcity/received_events","type":"User","site_admin":false},"labels":[{"id":335217860,"url":"https://api.github.com/repos/cockroachdb/cockroach/labels/Robot","name":"Robot","color":"5319e7","default":false},{"id":191587409,"url":"https://api.github.com/repos/cockroachdb/cockroach/labels/test-failure","name":"test-failure","color":"e11d21","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-11-26T10:01:05Z","updated_at":"2016-11-26T10:01:05Z","closed_at":null,"body":"SHA: https://github.com/cockroachdb/cockroach/commits/e049ec0385d5f6b31b98586903c125e33563d3af\n\nParameters:\n```\nCOCKROACH_PROPOSER_EVALUATED_KV=false\nTAGS=stress\nGOFLAGS=-race\n```\n\nStress build found a failed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=68119&tab=buildLog\n\n```\nI161126 10:00:13.233071 46843 gossip/gossip.go:244  [n?] initial resolvers: []\nW161126 10:00:13.233184 46843 gossip/gossip.go:1120  [n?] no resolvers found; use --join to specify a connected node\nW161126 10:00:13.320178 46843 server/status/runtime.go:116  Could not parse build timestamp: parsing time \"\" as \"2006/01/02 15:04:05\": cannot parse \"\" as \"2006\"\nI161126 10:00:13.321858 46843 storage/engine/rocksdb.go:340  opening in memory rocksdb instance\nI161126 10:00:13.322727 46843 server/config.go:443  1 storage engine initialized\nI161126 10:00:13.324041 46843 server/node.go:419  [n?] store [n0,s0] not bootstrapped\nI161126 10:00:13.348319 46899 storage/replica_proposal.go:351  [s1,r1/1:/M{in-ax}] new range lease replica {1 1 1} 1970-01-01 00:00:00 +0000 UTC 411154h0m22.345125077s following replica {0 0 0} 1970-01-01 00:00:00 +0000 UTC 0s [physicalTime=2016-11-26 10:00:13.34818208 +0000 UTC]\nI161126 10:00:13.352878 46843 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks\nI161126 10:00:13.352991 46843 util/stop/stopper.go:468  quiescing; tasks left:\n1      storage/replica_range_lease.go:157\nI161126 10:00:13.354544 46843 server/node.go:348  [n?] **** cluster 2b0e8089-797b-4bff-b602-1cfa37a69012 has been created\nI161126 10:00:13.354601 46843 server/node.go:349  [n?] **** add additional nodes by specifying --join=127.0.0.1:50498\nI161126 10:00:13.356006 46843 base/node_id.go:62  [n1] NodeID set to 1\nI161126 10:00:13.361331 46843 storage/store.go:1201  [n1] [n1,s1]: failed initial metrics computation: [n1,s1]: system config not yet available\nI161126 10:00:13.361440 46843 server/node.go:432  [n1] initialized store [n1,s1]: {Capacity:536870912 Available:536870912 RangeCount:0 LeaseCount:0}\nI161126 10:00:13.361580 46843 server/node.go:317  [n1] node ID 1 initialized\nI161126 10:00:13.370912 46918 storage/split_queue.go:103  [n1,split,s1,r1/1:/M{in-ax}] splitting at keys [/Table/11/0 /Table/12/0 /Table/13/0 /Table/14/0]\nI161126 10:00:13.382586 46843 gossip/gossip.go:286  [n1] NodeDescriptor set to node_id:1 address:<network_field:\"tcp\" address_field:\"127.0.0.1:50498\" > attrs:<> locality:<> \nI161126 10:00:13.383188 46843 storage/stores.go:296  [n1] read 0 node addresses from persistent storage\nI161126 10:00:13.383572 46843 server/node.go:562  [n1] connecting to gossip network to verify cluster ID...\nI161126 10:00:13.383665 46843 server/node.go:582  [n1] node connected via gossip and verified as part of cluster \"2b0e8089-797b-4bff-b602-1cfa37a69012\"\nI161126 10:00:13.430740 46918 storage/replica_command.go:2377  [n1,split,s1,r1/1:/M{in-ax}] initiating a split of this range at key /Table/11 [r2]\nI161126 10:00:13.476602 46843 server/node.go:367  [n1] node=1: started with [[]=] engine(s) and attributes []\nI161126 10:00:13.483805 46843 server/server.go:630  [n1] starting https server at 127.0.0.1:53451\nI161126 10:00:13.483877 46843 server/server.go:631  [n1] starting grpc/postgres server at 127.0.0.1:50498\nI161126 10:00:13.483917 46843 server/server.go:632  [n1] advertising CockroachDB node at 127.0.0.1:50498\nE161126 10:00:13.538174 46919 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:13.575656 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:13.580270 46918 storage/queue.go:576  [n1,split,s1,r1/1:/{Min-Table/11}] unable to split [n1,s1,r1/1:/{Min-Table/11}] at key \"/Table/12/0\": key range /Table/12/0-/Table/12/0 outside of bounds of range /Min-/Max\nI161126 10:00:13.635923 46963 sql/event_log.go:95  [n1] Event: \"node_join\", target: 1, info: {Descriptor:{NodeID:1 Address:{NetworkField:tcp AddressField:127.0.0.1:50498} Attrs: Locality:} ClusterID:2b0e8089-797b-4bff-b602-1cfa37a69012 StartedAt:1480154413383697718}\nE161126 10:00:13.642096 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nI161126 10:00:13.648168 46918 storage/split_queue.go:103  [n1,split,s1,r2/1:/{Table/11-Max}] splitting at keys [/Table/12/0 /Table/13/0 /Table/14/0]\nI161126 10:00:13.656603 46918 storage/replica_command.go:2377  [n1,split,s1,r2/1:/{Table/11-Max}] initiating a split of this range at key /Table/12 [r3]\nE161126 10:00:13.694978 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:13.803242 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:13.900583 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:13.933721 46919 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:13.944036 46918 storage/queue.go:576  [n1,split,s1,r2/1:/Table/1{1-2}] unable to split [n1,s1,r2/1:/Table/1{1-2}] at key \"/Table/13/0\": key range /Table/13/0-/Table/13/0 outside of bounds of range /Table/11-/Max\nI161126 10:00:13.954921 46918 storage/split_queue.go:103  [n1,split,s1,r3/1:/{Table/12-Max}] splitting at keys [/Table/13/0 /Table/14/0]\nI161126 10:00:13.956327 46918 storage/replica_command.go:2377  [n1,split,s1,r3/1:/{Table/12-Max}] initiating a split of this range at key /Table/13 [r4]\nE161126 10:00:14.003863 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.008983 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.092425 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.093165 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.245270 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.258337 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.297465 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.298404 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.364852 46919 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.382376 46918 storage/queue.go:576  [n1,split,s1,r3/1:/Table/1{2-3}] unable to split [n1,s1,r3/1:/Table/1{2-3}] at key \"/Table/14/0\": key range /Table/14/0-/Table/14/0 outside of bounds of range /Table/12-/Max\nI161126 10:00:14.383232 46918 storage/split_queue.go:103  [n1,split,s1,r4/1:/{Table/13-Max}] splitting at keys [/Table/14/0]\nI161126 10:00:14.383482 46918 storage/replica_command.go:2377  [n1,split,s1,r4/1:/{Table/13-Max}] initiating a split of this range at key /Table/14 [r5]\nE161126 10:00:14.398844 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.399887 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.405233 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.521886 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.522521 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.523083 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.591759 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.592456 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.593015 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.736452 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.763856 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.778106 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.803587 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.824470 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.825382 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.827443 46919 storage/queue.go:587  [n1,replicate,s1,r4/1:/Table/1{3-4}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.830694 46919 storage/queue.go:587  [n1,replicate,s1,r5/1:/{Table/14-Max}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.891606 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.892295 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.892850 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.893439 46957 storage/queue.go:587  [n1,replicate,s1,r4/1:/Table/1{3-4}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:14.894035 46957 storage/queue.go:587  [n1,replicate,s1,r5/1:/{Table/14-Max}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nI161126 10:00:14.915322 46843 gossip/gossip.go:244  [n?] initial resolvers: [127.0.0.1:50498]\nW161126 10:00:14.915441 46843 gossip/gossip.go:1122  [n?] no incoming or outgoing connections\nE161126 10:00:15.001544 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.007901 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.008587 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.009169 46957 storage/queue.go:587  [n1,replicate,s1,r4/1:/Table/1{3-4}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.009740 46957 storage/queue.go:587  [n1,replicate,s1,r5/1:/{Table/14-Max}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nW161126 10:00:15.048370 46843 server/status/runtime.go:116  Could not parse build timestamp: parsing time \"\" as \"2006/01/02 15:04:05\": cannot parse \"\" as \"2006\"\nI161126 10:00:15.062025 46843 storage/engine/rocksdb.go:340  opening in memory rocksdb instance\nI161126 10:00:15.063013 46843 server/config.go:443  1 storage engine initialized\nI161126 10:00:15.064328 46843 server/node.go:419  [n?] store [n0,s0] not bootstrapped\nI161126 10:00:15.064418 46843 storage/stores.go:296  [n?] read 0 node addresses from persistent storage\nI161126 10:00:15.064500 46843 server/node.go:562  [n?] connecting to gossip network to verify cluster ID...\nE161126 10:00:15.091634 46957 storage/queue.go:587  [n1,replicate,s1,r4/1:/Table/1{3-4}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.092403 46957 storage/queue.go:587  [n1,replicate,s1,r5/1:/{Table/14-Max}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.093032 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.093619 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.094202 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nI161126 10:00:15.180053 47162 gossip/server.go:285  [n1] received gossip from unknown node\nI161126 10:00:15.180424 47177 gossip/client.go:125  [n?] started gossip client to 127.0.0.1:50498\nI161126 10:00:15.186598 46843 server/node.go:582  [n?] node connected via gossip and verified as part of cluster \"2b0e8089-797b-4bff-b602-1cfa37a69012\"\nI161126 10:00:15.187072 47216 storage/stores.go:312  [n?] wrote 1 node addresses to persistent storage\nE161126 10:00:15.220791 46957 storage/queue.go:587  [n1,replicate,s1,r5/1:/{Table/14-Max}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.228203 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.231072 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.231736 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.232353 46957 storage/queue.go:587  [n1,replicate,s1,r4/1:/Table/1{3-4}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nI161126 10:00:15.235503 46843 kv/dist_sender.go:302  [n?] unable to determine this node's attributes for replica selection; node is most likely bootstrapping\nI161126 10:00:15.244744 46843 server/node.go:310  [n?] new node allocated ID 2\nI161126 10:00:15.245005 46843 base/node_id.go:62  [n2] NodeID set to 2\nI161126 10:00:15.245198 46843 gossip/gossip.go:286  [n2] NodeDescriptor set to node_id:2 address:<network_field:\"tcp\" address_field:\"127.0.0.1:57452\" > attrs:<> locality:<> \nI161126 10:00:15.245965 46843 server/node.go:367  [n2] node=2: started with [[]=] engine(s) and attributes []\nI161126 10:00:15.246522 46843 server/server.go:630  [n2] starting https server at 127.0.0.1:46454\nI161126 10:00:15.246565 46843 server/server.go:631  [n2] starting grpc/postgres server at 127.0.0.1:57452\nI161126 10:00:15.246603 46843 server/server.go:632  [n2] advertising CockroachDB node at 127.0.0.1:57452\nI161126 10:00:15.250465 47117 storage/stores.go:312  [n1] wrote 1 node addresses to persistent storage\nE161126 10:00:15.303903 46957 storage/queue.go:587  [n1,replicate,s1,r1/1:/{Min-Table/11}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.305978 46957 storage/queue.go:587  [n1,replicate,s1,r2/1:/Table/1{1-2}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nE161126 10:00:15.314418 46957 storage/queue.go:587  [n1,replicate,s1,r3/1:/Table/1{2-3}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nI161126 10:00:15.345098 47167 server/node.go:543  [n2] bootstrapped store [n2,s2]\nE161126 10:00:15.354643 46957 storage/queue.go:587  [n1,replicate,s1,r4/1:/Table/1{3-4}] purgatory: 0 of 1 store with an attribute matching []; likely not enough nodes in cluster\nI161126 10:00:15.439284 46957 storage/replica_raftstorage.go:459  [n1,replicate,s1,r5/1:/{Table/14-Max}] generated preemptive snapshot 215594c5 at index 11\nI161126 10:00:15.521032 46843 gossip/gossip.go:244  [n?] initial resolvers: [127.0.0.1:50498]\nW161126 10:00:15.521178 46843 gossip/gossip.go:1122  [n?] no incoming or outgoing connections\nI161126 10:00:15.525922 47266 sql/event_log.go:95  [n2] Event: \"node_join\", target: 2, info: {Descriptor:{NodeID:2 Address:{NetworkField:tcp AddressField:127.0.0.1:57452} Attrs: Locality:} ClusterID:2b0e8089-797b-4bff-b602-1cfa37a69012 StartedAt:1480154415245679526}\nW161126 10:00:15.600196 46843 server/status/runtime.go:116  Could not parse build timestamp: parsing time \"\" as \"2006/01/02 15:04:05\": cannot parse \"\" as \"2006\"\nI161126 10:00:15.651057 46843 storage/engine/rocksdb.go:340  opening in memory rocksdb instance\nI161126 10:00:15.653340 46843 server/config.go:443  1 storage engine initialized\nI161126 10:00:15.654594 46843 server/node.go:419  [n?] store [n0,s0] not bootstrapped\nI161126 10:00:15.654687 46843 storage/stores.go:296  [n?] read 0 node addresses from persistent storage\nI161126 10:00:15.654779 46843 server/node.go:562  [n?] connecting to gossip network to verify cluster ID...\nI161126 10:00:15.677717 46957 storage/store.go:3157  [n1,replicate,s1,r5/1:/{Table/14-Max}] streamed snapshot: kv pairs: 9, log entries: 1\nI161126 10:00:15.678965 47397 storage/replica_raftstorage.go:621  [n2,s2,r5/?:{-}] applying preemptive snapshot at index 11 (id=215594c5, encoded size=508, 1 rocksdb batches, 1 log entries)\nI161126 10:00:15.680134 47397 storage/replica_raftstorage.go:629  [n2,s2,r5/?:/{Table/14-Max}] applied preemptive snapshot in 1ms [clear=0ms batch=0ms entries=0ms commit=0ms]\nI161126 10:00:15.691911 46957 storage/replica_command.go:3273  [n1,replicate,s1,r5/1:/{Table/14-Max}] change replicas: read existing descriptor range_id:5 start_key:\"\\226\" end_key:\"\\377\\377\" replicas:<node_id:1 store_id:1 replica_id:1 > next_replica_id:2 \nI161126 10:00:15.888429 47200 gossip/server.go:285  [n1] received gossip from unknown node\nI161126 10:00:15.889002 47260 gossip/client.go:125  [n?] started gossip client to 127.0.0.1:50498\nI161126 10:00:15.895841 47265 storage/replica.go:2190  [n1,s1,r5/1:/{Table/14-Max}] proposing ADD_REPLICA {NodeID:2 StoreID:2 ReplicaID:2}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:2 StoreID:2 ReplicaID:2}]\nI161126 10:00:15.906594 46957 storage/replica_raftstorage.go:459  [n1,replicate,s1,r1/1:/{Min-Table/11}] generated preemptive snapshot 1767bd82 at index 135\nI161126 10:00:15.916565 47446 storage/stores.go:312  [n?] wrote 1 node addresses to persistent storage\nI161126 10:00:15.917419 47446 storage/stores.go:312  [n?] wrote 2 node addresses to persistent storage\nI161126 10:00:15.917685 46843 server/node.go:582  [n?] node connected via gossip and verified as part of cluster \"2b0e8089-797b-4bff-b602-1cfa37a69012\"\nI161126 10:00:15.942379 46843 kv/dist_sender.go:302  [n?] unable to determine this node's attributes for replica selection; node is most likely bootstrapping\nI161126 10:00:15.959778 47511 storage/raft_transport.go:437  [n2] raft transport stream to node 1 established\nI161126 10:00:15.989637 46843 server/node.go:310  [n?] new node allocated ID 3\nI161126 10:00:15.989879 46843 base/node_id.go:62  [n3] NodeID set to 3\nI161126 10:00:15.990086 46843 gossip/gossip.go:286  [n3] NodeDescriptor set to node_id:3 address:<network_field:\"tcp\" address_field:\"127.0.0.1:43931\" > attrs:<> locality:<> \nI161126 10:00:15.998481 46843 server/node.go:367  [n3] node=3: started with [[]=] engine(s) and attributes []\nI161126 10:00:16.005635 46843 server/server.go:630  [n3] starting https server at 127.0.0.1:43221\nI161126 10:00:16.005705 46843 server/server.go:631  [n3] starting grpc/postgres server at 127.0.0.1:43931\nI161126 10:00:16.005743 46843 server/server.go:632  [n3] advertising CockroachDB node at 127.0.0.1:43931\nI161126 10:00:16.017771 46957 storage/store.go:3157  [n1,replicate,s1,r1/1:/{Min-Table/11}] streamed snapshot: kv pairs: 590, log entries: 21\nI161126 10:00:16.053096 47463 storage/replica_raftstorage.go:621  [n2,s2,r1/?:{-}] applying preemptive snapshot at index 135 (id=1767bd82, encoded size=99976, 1 rocksdb batches, 21 log entries)\nI161126 10:00:16.066274 47463 storage/replica_raftstorage.go:629  [n2,s2,r1/?:/{Min-Table/11}] applied preemptive snapshot in 10ms [clear=0ms batch=2ms entries=1ms commit=3ms]\nI161126 10:00:16.077952 47515 storage/stores.go:312  [n1] wrote 2 node addresses to persistent storage\nI161126 10:00:16.090704 47470 storage/stores.go:312  [n2] wrote 2 node addresses to persistent storage\nI161126 10:00:16.096465 46957 storage/replica_command.go:3273  [n1,replicate,s1,r1/1:/{Min-Table/11}] change replicas: read existing descriptor range_id:1 start_key:\"\" end_key:\"\\223\" replicas:<node_id:1 store_id:1 replica_id:1 > next_replica_id:2 \nI161126 10:00:16.232949 46843 gossip/gossip.go:244  [n?] initial resolvers: [127.0.0.1:50498]\nW161126 10:00:16.233079 46843 gossip/gossip.go:1122  [n?] no incoming or outgoing connections\nI161126 10:00:16.259076 47602 storage/replica.go:2190  [n1,s1,r1/1:/{Min-Table/11}] proposing ADD_REPLICA {NodeID:2 StoreID:2 ReplicaID:2}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:2 StoreID:2 ReplicaID:2}]\nI161126 10:00:16.271354 47385 server/node.go:543  [n3] bootstrapped store [n3,s3]\nI161126 10:00:16.442774 46957 storage/replica_raftstorage.go:459  [n1,replicate,s1,r2/1:/Table/1{1-2}] generated preemptive snapshot 87c15ae7 at index 25\nW161126 10:00:16.448069 46843 server/status/runtime.go:116  Could not parse build timestamp: parsing time \"\" as \"2006/01/02 15:04:05\": cannot parse \"\" as \"2006\"\nI161126 10:00:16.476172 46843 storage/engine/rocksdb.go:340  opening in memory rocksdb instance\nI161126 10:00:16.477057 46843 server/config.go:443  1 storage engine initialized\nI161126 10:00:16.478320 46843 server/node.go:419  [n?] store [n0,s0] not bootstrapped\nI161126 10:00:16.478392 46843 storage/stores.go:296  [n?] read 0 node addresses from persistent storage\nI161126 10:00:16.478487 46843 server/node.go:562  [n?] connecting to gossip network to verify cluster ID...\nI161126 10:00:16.515167 47388 sql/event_log.go:95  [n3] Event: \"node_join\", target: 3, info: {Descriptor:{NodeID:3 Address:{NetworkField:tcp AddressField:127.0.0.1:43931} Attrs: Locality:} ClusterID:2b0e8089-797b-4bff-b602-1cfa37a69012 StartedAt:1480154415998125437}\nI161126 10:00:16.856193 47658 gossip/client.go:125  [n?] started gossip client to 127.0.0.1:50498\nI161126 10:00:16.858226 47732 gossip/server.go:285  [n1] received gossip from unknown node\nI161126 10:00:16.864041 47734 storage/stores.go:312  [n?] wrote 1 node addresses to persistent storage\nI161126 10:00:16.864293 47734 storage/stores.go:312  [n?] wrote 2 node addresses to persistent storage\nI161126 10:00:16.864506 47734 storage/stores.go:312  [n?] wrote 3 node addresses to persistent storage\nI161126 10:00:16.866205 46843 server/node.go:582  [n?] node connected via gossip and verified as part of cluster \"2b0e8089-797b-4bff-b602-1cfa37a69012\"\nI161126 10:00:16.873966 46957 storage/store.go:3157  [n1,replicate,s1,r2/1:/Table/1{1-2}] streamed snapshot: kv pairs: 10, log entries: 15\nI161126 10:00:16.876104 47747 storage/replica_raftstorage.go:621  [n3,s3,r2/?:{-}] applying preemptive snapshot at index 25 (id=87c15ae7, encoded size=7631, 1 rocksdb batches, 15 log entries)\nI161126 10:00:16.878295 47747 storage/replica_raftstorage.go:629  [n3,s3,r2/?:/Table/1{1-2}] applied preemptive snapshot in 2ms [clear=0ms batch=0ms entries=1ms commit=0ms]\nI161126 10:00:16.882837 46957 storage/replica_command.go:3273  [n1,replicate,s1,r2/1:/Table/1{1-2}] change replicas: read existing descriptor range_id:2 start_key:\"\\223\" end_key:\"\\224\" replicas:<node_id:1 store_id:1 replica_id:1 > next_replica_id:2 \nI161126 10:00:16.913710 46843 kv/dist_sender.go:302  [n?] unable to determine this node's attributes for replica selection; node is most likely bootstrapping\nI161126 10:00:16.958700 46843 server/node.go:310  [n?] new node allocated ID 4\nI161126 10:00:16.958937 46843 base/node_id.go:62  [n4] NodeID set to 4\nI161126 10:00:16.959137 46843 gossip/gossip.go:286  [n4] NodeDescriptor set to node_id:4 address:<network_field:\"tcp\" address_field:\"127.0.0.1:34085\" > attrs:<> locality:<> \nI161126 10:00:16.980901 46843 server/node.go:367  [n4] node=4: started with [[]=] engine(s) and attributes []\nI161126 10:00:16.981610 46843 server/server.go:630  [n4] starting https server at 127.0.0.1:38526\nI161126 10:00:16.981660 46843 server/server.go:631  [n4] starting grpc/postgres server at 127.0.0.1:34085\nI161126 10:00:16.981699 46843 server/server.go:632  [n4] advertising CockroachDB node at 127.0.0.1:34085\nI161126 10:00:17.001521 47723 storage/stores.go:312  [n1] wrote 3 node addresses to persistent storage\nI161126 10:00:17.014369 47830 storage/stores.go:312  [n3] wrote 3 node addresses to persistent storage\nI161126 10:00:17.088436 47711 storage/stores.go:312  [n2] wrote 3 node addresses to persistent storage\nI161126 10:00:17.209395 46843 gossip/gossip.go:244  [n?] initial resolvers: [127.0.0.1:50498]\nW161126 10:00:17.225908 46843 gossip/gossip.go:1122  [n?] no incoming or outgoing connections\nI161126 10:00:17.235428 47906 storage/replica.go:2190  [n1,s1,r2/1:/Table/1{1-2}] proposing ADD_REPLICA {NodeID:3 StoreID:3 ReplicaID:2}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:3 StoreID:3 ReplicaID:2}]\nI161126 10:00:17.279463 47813 server/node.go:543  [n4] bootstrapped store [n4,s4]\nI161126 10:00:17.322613 46957 storage/replica_command.go:3273  [n1,replicate,s1,r2/1:/Table/1{1-2}] change replicas: read existing descriptor range_id:2 start_key:\"\\223\" end_key:\"\\224\" replicas:<node_id:1 store_id:1 replica_id:1 > next_replica_id:2 \nW161126 10:00:17.390985 46843 server/status/runtime.go:116  Could not parse build timestamp: parsing time \"\" as \"2006/01/02 15:04:05\": cannot parse \"\" as \"2006\"\nI161126 10:00:17.392857 46843 storage/engine/rocksdb.go:340  opening in memory rocksdb instance\nI161126 10:00:17.393753 46843 server/config.go:443  1 storage engine initialized\nI161126 10:00:17.415713 46843 server/node.go:419  [n?] store [n0,s0] not bootstrapped\nI161126 10:00:17.434241 46843 storage/stores.go:296  [n?] read 0 node addresses from persistent storage\nI161126 10:00:17.434412 46843 server/node.go:562  [n?] connecting to gossip network to verify cluster ID...\nI161126 10:00:17.434464 47816 sql/event_log.go:95  [n4] Event: \"node_join\", target: 4, info: {Descriptor:{NodeID:4 Address:{NetworkField:tcp AddressField:127.0.0.1:34085} Attrs: Locality:} ClusterID:2b0e8089-797b-4bff-b602-1cfa37a69012 StartedAt:1480154416980538994}\nI161126 10:00:17.571793 47884 storage/replica.go:2190  [n1,s1,r2/1:/Table/1{1-2}] proposing ADD_REPLICA {NodeID:3 StoreID:3 ReplicaID:2}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:3 StoreID:3 ReplicaID:2}]\nI161126 10:00:17.577038 46957 storage/replica_raftstorage.go:459  [n1,replicate,s1,r3/1:/Table/1{2-3}] generated preemptive snapshot a3308ef0 at index 29\nI161126 10:00:17.615310 46957 storage/store.go:3157  [n1,replicate,s1,r3/1:/Table/1{2-3}] streamed snapshot: kv pairs: 30, log entries: 19\nI161126 10:00:17.619581 47959 storage/replica_raftstorage.go:621  [n3,s3,r3/?:{-}] applying preemptive snapshot at index 29 (id=a3308ef0, encoded size=12143, 1 rocksdb batches, 19 log entries)\nI161126 10:00:17.621442 47959 storage/replica_raftstorage.go:629  [n3,s3,r3/?:/Table/1{2-3}] applied preemptive snapshot in 2ms [clear=0ms batch=0ms entries=1ms commit=0ms]\nI161126 10:00:17.668913 46957 storage/replica_command.go:3273  [n1,replicate,s1,r3/1:/Table/1{2-3}] change replicas: read existing descriptor range_id:3 start_key:\"\\224\" end_key:\"\\225\" replicas:<node_id:1 store_id:1 replica_id:1 > next_replica_id:2 \nI161126 10:00:17.703836 48072 storage/raft_transport.go:437  [n3] raft transport stream to node 1 established\nI161126 10:00:17.749387 47975 gossip/client.go:125  [n?] started gossip client to 127.0.0.1:50498\nI161126 10:00:17.761861 48080 gossip/server.go:285  [n1] received gossip from unknown node\nI161126 10:00:17.789102 48044 storage/stores.go:312  [n?] wrote 1 node addresses to persistent storage\nI161126 10:00:17.795155 48046 storage/stores.go:312  [n?] wrote 2 node addresses to persistent storage\nI161126 10:00:17.795429 48046 storage/stores.go:312  [n?] wrote 3 node addresses to persistent storage\nI161126 10:00:17.797267 46843 server/node.go:582  [n?] node connected via gossip and verified as part of cluster \"2b0e8089-797b-4bff-b602-1cfa37a69012\"\nI161126 10:00:17.808202 48046 storage/stores.go:312  [n?] wrote 4 node addresses to persistent storage\nI161126 10:00:17.843313 46843 kv/dist_sender.go:302  [n?] unable to determine this node's attributes for replica selection; node is most likely bootstrapping\nI161126 10:00:17.905644 48138 storage/replica.go:2190  [n1,s1,r3/1:/Table/1{2-3}] proposing ADD_REPLICA {NodeID:3 StoreID:3 ReplicaID:2}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:3 StoreID:3 ReplicaID:2}]\nI161126 10:00:17.949767 46843 server/node.go:310  [n?] new node allocated ID 5\nI161126 10:00:17.950123 46843 base/node_id.go:62  [n5] NodeID set to 5\nI161126 10:00:17.950316 46843 gossip/gossip.go:286  [n5] NodeDescriptor set to node_id:5 address:<network_field:\"tcp\" address_field:\"127.0.0.1:46689\" > attrs:<> locality:<> \nI161126 10:00:17.951066 46843 server/node.go:367  [n5] node=5: started with [[]=] engine(s) and attributes []\nI161126 10:00:17.951742 46843 server/server.go:630  [n5] starting https server at 127.0.0.1:60253\nI161126 10:00:17.951786 46843 server/server.go:631  [n5] starting grpc/postgres server at 127.0.0.1:46689\nI161126 10:00:17.951822 46843 server/server.go:632  [n5] advertising CockroachDB node at 127.0.0.1:46689\nI161126 10:00:17.956786 48080 gossip/server.go:263  [n1] refusing gossip from node 5 (max 3 conns); forwarding to 2 ({tcp 127.0.0.1:57452})\nI161126 10:00:17.957770 48080 gossip/server.go:263  [n1] refusing gossip from node 5 (max 3 conns); forwarding to 3 ({tcp 127.0.0.1:43931})\nI161126 10:00:17.958423 48112 storage/stores.go:312  [n1] wrote 4 node addresses to persistent storage\nI161126 10:00:18.010293 48113 storage/stores.go:312  [n2] wrote 4 node addresses to persistent storage\nI161126 10:00:18.012857 47975 gossip/client.go:130  [n5] closing client to node 1 (127.0.0.1:50498): received forward from node 1 to 2 (127.0.0.1:57452)\nI161126 10:00:18.017577 48211 storage/stores.go:312  [n4] wrote 4 node addresses to persistent storage\nI161126 10:00:18.098988 48242 storage/stores.go:312  [n3] wrote 4 node addresses to persistent storage\nI161126 10:00:18.112301 46957 storage/replica_raftstorage.go:459  [n1,replicate,s1,r4/1:/Table/1{3-4}] generated preemptive snapshot 5be22d33 at index 29\nW161126 10:00:18.207524 46876 vendor/github.com/coreos/etcd/raft/raft.go:794  [n1,s1,r3/1:/Table/1{2-3}] 1 stepped down to follower since quorum is not active\nI161126 10:00:18.306074 48141 gossip/client.go:125  [n5] started gossip client to 127.0.0.1:57452\nI161126 10:00:18.501991 48198 server/node.go:543  [n5] bootstrapped store [n5,s5]\nI161126 10:00:18.693610 46957 storage/store.go:3157  [n1,replicate,s1,r4/1:/Table/1{3-4}] streamed snapshot: kv pairs: 54, log entries: 19\nI161126 10:00:18.732975 48421 storage/replica_raftstorage.go:621  [n4,s4,r4/?:{-}] applying preemptive snapshot at index 29 (id=5be22d33, encoded size=15500, 1 rocksdb batches, 19 log entries)\nI161126 10:00:18.734924 48421 storage/replica_raftstorage.go:629  [n4,s4,r4/?:/Table/1{3-4}] applied preemptive snapshot in 2ms [clear=0ms batch=0ms entries=1ms commit=0ms]\nI161126 10:00:18.776990 46957 storage/replica_command.go:3273  [n1,replicate,s1,r4/1:/Table/1{3-4}] change replicas: read existing descriptor range_id:4 start_key:\"\\225\" end_key:\"\\226\" replicas:<node_id:1 store_id:1 replica_id:1 > next_replica_id:2 \nI161126 10:00:19.008750 48522 storage/replica.go:2190  [n1,s1,r4/1:/Table/1{3-4}] proposing ADD_REPLICA {NodeID:4 StoreID:4 ReplicaID:2}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:4 StoreID:4 ReplicaID:2}]\nI161126 10:00:19.051734 46957 storage/queue.go:639  [n1,replicate] purgatory is now empty\nI161126 10:00:19.053291 46919 storage/replica_raftstorage.go:459  [n1,replicate,s1,r5/1:/{Table/14-Max}] generated preemptive snapshot a024fc04 at index 15\nI161126 10:00:19.059189 46919 storage/store.go:3157  [n1,replicate,s1,r5/1:/{Table/14-Max}] streamed snapshot: kv pairs: 10, log entries: 5\nI161126 10:00:19.060498 48414 storage/replica_raftstorage.go:621  [n4,s4,r5/?:{-}] applying preemptive snapshot at index 15 (id=a024fc04, encoded size=1772, 1 rocksdb batches, 5 log entries)\nI161126 10:00:19.061706 48414 storage/replica_raftstorage.go:629  [n4,s4,r5/?:/{Table/14-Max}] applied preemptive snapshot in 1ms [clear=0ms batch=0ms entries=0ms commit=0ms]\nI161126 10:00:19.066961 48626 storage/raft_transport.go:437  [n4] raft transport stream to node 1 established\nI161126 10:00:19.081439 46919 storage/replica_command.go:3273  [n1,replicate,s1,r5/1:/{Table/14-Max}] change replicas: read existing descriptor range_id:5 start_key:\"\\226\" end_key:\"\\377\\377\" replicas:<node_id:1 store_id:1 replica_id:1 > replicas:<node_id:2 store_id:2 replica_id:2 > next_replica_id:3 \nI161126 10:00:19.189630 48642 storage/replica.go:2190  [n1,s1,r5/1:/{Table/14-Max}] proposing ADD_REPLICA {NodeID:4 StoreID:4 ReplicaID:3}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:2 StoreID:2 ReplicaID:2} {NodeID:4 StoreID:4 ReplicaID:3}]\nI161126 10:00:19.224112 46919 storage/replica_raftstorage.go:459  [n1,replicate,s1,r4/1:/Table/1{3-4}] generated preemptive snapshot 975ca489 at index 34\nI161126 10:00:19.228436 46919 storage/store.go:3157  [n1,replicate,s1,r4/1:/Table/1{3-4}] streamed snapshot: kv pairs: 70, log entries: 24\nI161126 10:00:19.239767 48646 storage/replica_raftstorage.go:621  [n3,s3,r4/?:{-}] applying preemptive snapshot at index 34 (id=975ca489, encoded size=19470, 1 rocksdb batches, 24 log entries)\nI161126 10:00:19.242017 48646 storage/replica_raftstorage.go:629  [n3,s3,r4/?:/Table/1{3-4}] applied preemptive snapshot in 2ms [clear=0ms batch=0ms entries=1ms commit=0ms]\nI161126 10:00:19.345055 46919 storage/replica_command.go:3273  [n1,replicate,s1,r4/1:/Table/1{3-4}] change replicas: read existing descriptor range_id:4 start_key:\"\\225\" end_key:\"\\226\" replicas:<node_id:1 store_id:1 replica_id:1 > replicas:<node_id:4 store_id:4 replica_id:2 > next_replica_id:3 \nI161126 10:00:19.631581 48732 storage/replica.go:2190  [n1,s1,r4/1:/Table/1{3-4}] proposing ADD_REPLICA {NodeID:3 StoreID:3 ReplicaID:3}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:4 StoreID:4 ReplicaID:2} {NodeID:3 StoreID:3 ReplicaID:3}]\nI161126 10:00:19.647815 46919 storage/replica_raftstorage.go:459  [n1,replicate,s1,r2/1:/Table/1{1-2}] generated preemptive snapshot 813a927d at index 30\nI161126 10:00:20.062908 46919 storage/store.go:3157  [n1,replicate,s1,r2/1:/Table/1{1-2}] streamed snapshot: kv pairs: 11, log entries: 20\nI161126 10:00:20.065686 48845 storage/replica_raftstorage.go:621  [n5,s5,r2/?:{-}] applying preemptive snapshot at index 30 (id=813a927d, encoded size=9565, 1 rocksdb batches, 20 log entries)\nI161126 10:00:20.067497 48845 storage/replica_raftstorage.go:629  [n5,s5,r2/?:/Table/1{1-2}] applied preemptive snapshot in 2ms [clear=0ms batch=0ms entries=1ms commit=0ms]\nI161126 10:00:20.073661 46919 storage/replica_command.go:3273  [n1,replicate,s1,r2/1:/Table/1{1-2}] change replicas: read existing descriptor range_id:2 start_key:\"\\223\" end_key:\"\\224\" replicas:<node_id:1 store_id:1 replica_id:1 > replicas:<node_id:3 store_id:3 replica_id:2 > next_replica_id:3 \nI161126 10:00:20.210106 48799 storage/replica.go:2190  [n1,s1,r2/1:/Table/1{1-2}] proposing ADD_REPLICA {NodeID:5 StoreID:5 ReplicaID:3}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:3 StoreID:3 ReplicaID:2} {NodeID:5 StoreID:5 ReplicaID:3}]\nI161126 10:00:20.248062 46919 storage/replica_raftstorage.go:459  [n1,replicate,s1,r1/1:/{Min-Table/11}] generated preemptive snapshot 301904a0 at index 256\nI161126 10:00:20.376282 48977 storage/raft_transport.go:437  [n5] raft transport stream to node 1 established\nI161126 10:00:20.397915 46919 storage/store.go:3157  [n1,replicate,s1,r1/1:/{Min-Table/11}] streamed snapshot: kv pairs: 1186, log entries: 22\nI161126 10:00:20.399361 49026 storage/replica_raftstorage.go:621  [n5,s5,r1/?:{-}] applying preemptive snapshot at index 256 (id=301904a0, encoded size=183192, 1 rocksdb batches, 22 log entries)\nI161126 10:00:20.415592 49026 storage/replica_raftstorage.go:629  [n5,s5,r1/?:/{Min-Table/11}] applied preemptive snapshot in 9ms [clear=3ms batch=2ms entries=2ms commit=1ms]\nI161126 10:00:20.468360 46919 storage/replica_command.go:3273  [n1,replicate,s1,r1/1:/{Min-Table/11}] change replicas: read existing descriptor range_id:1 start_key:\"\" end_key:\"\\223\" replicas:<node_id:1 store_id:1 replica_id:1 > replicas:<node_id:2 store_id:2 replica_id:2 > next_replica_id:3 \nI161126 10:00:20.638572 49008 storage/replica.go:2190  [n1,s1,r1/1:/{Min-Table/11}] proposing ADD_REPLICA {NodeID:5 StoreID:5 ReplicaID:3}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:2 StoreID:2 ReplicaID:2} {NodeID:5 StoreID:5 ReplicaID:3}]\nI161126 10:00:20.770529 46919 storage/replica_raftstorage.go:459  [n1,replicate,s1,r3/1:/Table/1{2-3}] generated preemptive snapshot 6c6ca2e9 at index 31\nI161126 10:00:20.775457 46919 storage/store.go:3157  [n1,replicate,s1,r3/1:/Table/1{2-3}] streamed snapshot: kv pairs: 32, log entries: 21\nI161126 10:00:20.843002 49218 storage/replica_raftstorage.go:621  [n2,s2,r3/?:{-}] applying preemptive snapshot at index 31 (id=6c6ca2e9, encoded size=13323, 1 rocksdb batches, 21 log entries)\nI161126 10:00:20.847554 49218 storage/replica_raftstorage.go:629  [n2,s2,r3/?:/Table/1{2-3}] applied preemptive snapshot in 4ms [clear=0ms batch=0ms entries=3ms commit=0ms]\nI161126 10:00:20.873754 46919 storage/replica_command.go:3273  [n1,replicate,s1,r3/1:/Table/1{2-3}] change replicas: read existing descriptor range_id:3 start_key:\"\\224\" end_key:\"\\225\" replicas:<node_id:1 store_id:1 replica_id:1 > replicas:<node_id:3 store_id:3 replica_id:2 > next_replica_id:3 \nI161126 10:00:20.903507 49241 storage/replica_command.go:2377  [n1,s1,r5/1:/{Table/14-Max}] initiating a split of this range at key /Table/50 [r6]\nI161126 10:00:21.346906 49328 storage/replica_command.go:2377  [n1,s1,r6/1:/{Table/50-Max}] initiating a split of this range at key /Table/51 [r7]\nI161126 10:00:21.936367 49487 storage/replica_command.go:2377  [n1,s1,r7/1:/{Table/51-Max}] initiating a split of this range at key /Table/52 [r8]\nI161126 10:00:22.180964 48201 sql/event_log.go:95  [n5] Event: \"node_join\", target: 5, info: {Descriptor:{NodeID:5 Address:{NetworkField:tcp AddressField:127.0.0.1:46689} Attrs: Locality:} ClusterID:2b0e8089-797b-4bff-b602-1cfa37a69012 StartedAt:1480154417950772637}\nI161126 10:00:22.483604 49724 storage/replica.go:2190  [n1,s1,r3/1:/Table/1{2-3}] proposing ADD_REPLICA {NodeID:2 StoreID:2 ReplicaID:3}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:3 StoreID:3 ReplicaID:2} {NodeID:2 StoreID:2 ReplicaID:3}]\nI161126 10:00:22.743539 46919 storage/replica_raftstorage.go:459  [n1,replicate,s1,r3/1:/Table/1{2-3}] generated preemptive snapshot a9054d2b at index 40\nI161126 10:00:22.781103 46919 storage/store.go:3157  [n1,replicate,s1,r3/1:/Table/1{2-3}] streamed snapshot: kv pairs: 38, log entries: 30\nI161126 10:00:22.845856 49906 storage/replica_raftstorage.go:621  [n5,s5,r3/?:{-}] applying preemptive snapshot at index 40 (id=a9054d2b, encoded size=16686, 1 rocksdb batches, 30 log entries)\nI161126 10:00:22.904494 49906 storage/replica_raftstorage.go:629  [n5,s5,r3/?:/Table/1{2-3}] applied preemptive snapshot in 42ms [clear=39ms batch=0ms entries=1ms commit=0ms]\nI161126 10:00:22.908975 46919 storage/replica_command.go:3273  [n1,replicate,s1,r3/1:/Table/1{2-3}] change replicas: read existing descriptor range_id:3 start_key:\"\\224\" end_key:\"\\225\" replicas:<node_id:1 store_id:1 replica_id:1 > replicas:<node_id:3 store_id:3 replica_id:2 > replicas:<node_id:2 store_id:2 replica_id:3 > next_replica_id:4 \nI161126 10:00:23.172951 49915 storage/replica_command.go:2377  [n1,s1,r8/1:/{Table/52-Max}] initiating a split of this range at key /Table/53 [r9]\nI161126 10:00:23.623191 50042 storage/replica.go:2190  [n1,s1,r3/1:/Table/1{2-3}] proposing ADD_REPLICA {NodeID:5 StoreID:5 ReplicaID:4}: [{NodeID:1 StoreID:1 ReplicaID:1} {NodeID:3 StoreID:3 ReplicaID:2} {NodeID:2 StoreID:2 ReplicaID:3} {NodeID:5 StoreID:5 ReplicaID:4}]\nI161126 10:00:23.798536 46843 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks\nI161126 10:00:23.818913 46919 storage/replica_command.go:3273  [n1,replicate,s1,r3/1:/Table/1{2-3}] change replicas: read existing descriptor range_id:3 start_key:\"\\224\" end_key:\"\\225\" replicas:<node_id:1 store_id:1 replica_id:1 > replicas:<node_id:3 store_id:3 replica_id:2 > replicas:<node_id:2 store_id:2 replica_id:3 > replicas:<node_id:5 store_id:5 replica_id:4 > next_replica_id:5 \nI161126 10:00:23.830215 50199 util/stop/stopper.go:468  quiescing; tasks left:\n2      gossip/infostore.go:301\nW161126 10:00:23.861133 50216 storage/replica.go:1928  [n1,s1,r1/1:/{Min-Table/11}] shutdown cancellation after 0.0s of attempting command ResolveIntent [/Meta2/Table/53,/Min), ResolveIntent [/Meta2/Max,/Min), Noop [/Min,/Min), Noop [/Min,/Min), Noop [/Min,/Min), Noop [/Min,/Min), Noop [/Min,/Min), Noop [/Min,/Min)\nW161126 10:00:23.861220 47404 gossip/infostore.go:303  [n3] node unavailable; try another peer\nW161126 10:00:23.861478 47404 gossip/infostore.go:303  [n3] node unavailable; try another peer\nW161126 10:00:23.861803 47404 gossip/infostore.go:303  [n3] node unavailable; try another peer\nW161126 10:00:23.861933 48977 storage/raft_transport.go:443  [n5] raft transport stream to node 1 failed: rpc error: code = 13 desc = transport is closing\nW161126 10:00:23.862542 47387 gossip/infostore.go:303  [n3] node unavailable; try another peer\nW161126 10:00:23.863144 48200 gossip/infostore.go:303  [n5] node unavailable; try another peer\nI161126 10:00:23.863340 50197 util/stop/stopper.go:468  quiescing; tasks left:\n5      server/node.go:830\n2      storage/queue.go:478\n2      storage/intent_resolver.go:380\n2      kv/dist_sender.go:796\nW161126 10:00:23.863706 50184 storage/replica.go:1928  [n1,replicate,s1,r3/1:/Table/1{2-3}] shutdown cancellation after 0.0s of attempting command [txn: bc1869e3], BeginTransaction [/Local/Range/\"\\x94\"/RangeDescriptor,/Min), ConditionalPut [/Local/Range/\"\\x94\"/RangeDescriptor,/Min)\nW161126 10:00:23.864125 48626 storage/raft_transport.go:443  [n4] raft transport stream to node 1 failed: rpc error: code = 13 desc = transport is closing\nW161126 10:00:23.864335 48997 storage/raft_transport.go:443  [n1] raft transport stream to node 5 failed: EOF\nW161126 10:00:23.864527 48573 storage/raft_transport.go:443  [n1] raft transport stream to node 4 failed: EOF\nW161126 10:00:23.864830 47511 storage/raft_transport.go:443  [n2] raft transport stream to node 1 failed: rpc error: code = 13 desc = transport is closing\nW161126 10:00:23.865271 48072 storage/raft_transport.go:443  [n3] raft transport stream to node 1 failed: rpc error: code = 13 desc = transport is closing\nW161126 10:00:23.865443 47429 storage/raft_transport.go:443  [n1] raft transport stream to node 2 failed: EOF\nW161126 10:00:23.865655 48067 storage/raft_transport.go:443  [n1] raft transport stream to node 3 failed: EOF\nW161126 10:00:23.865836 50061 storage/replica.go:1928  [n1,timeSeriesMaintenance,s1,r1/1:/{Min-Table/11}] shutdown cancellation after 0.1s of attempting command DeleteRange [/System/tsd/insufficient bytes to decode uvarint value,/System/tsd/cr.store.mutex.raftnanos-p99//10s/2016-10-27T10:00:00Z)\nW161126 10:00:23.866301 50168 storage/replica.go:1928  [n1,s1,r1/1:/{Min-Table/11}] shutdown cancellation after 0.1s of attempting command ResolveIntent [/Meta2/Table/13,/Min), Noop [/Min,/Min), Noop [/Min,/Min), Noop [/Min,/Min), Noop [/Min,/Min), Noop [/Min,/Min)\nW161126 10:00:23.867520 47169 gossip/infostore.go:303  [n2] node unavailable; try another peer\nW161126 10:00:23.870019 50215 storage/replica.go:1928  [n1,s1,r4/1:/Table/1{3-4}] shutdown cancellation after 0.0s of attempting command Noop [/Min,/Min), Noop [/Min,/Min), ResolveIntent [/Table/13/1/2016-11-26T10:00:23.173215Z/196883327795593216/0,/Min), ResolveIntent [/Table/13/1/2016-11-26T10:00:23.173215Z/196883327795593216/2/1,/Min), ResolveIntent [/Table/13/1/2016-11-26T10:00:23.173215Z/196883327795593216/3/1,/Min), ResolveIntent [/Table/13/1/2016-11-26T10:00:23.173215Z/196883327795593216/4/1,/Min), ResolveIntent [/Table/13/1/2016-11-26T10:00:23.173215Z/196883327795593216/5/1,/Min), ResolveIntent [/Table/13/1/2016-11-26T10:00:23.173215Z/196883327795593216/6/1,/Min)\nI161126 10:00:23.870382 50197 util/stop/stopper.go:468  quiescing; tasks left:\n4      server/node.go:830\n2      storage/queue.go:478\n2      storage/intent_resolver.go:380\n2      kv/dist_sender.go:796\nI161126 10:00:23.875397 50197 util/stop/stopper.go:468  quiescing; tasks left:\n3      server/node.go:830\n2      storage/queue.go:478\n2      storage/intent_resolver.go:380\n2      kv/dist_sender.go:796\nI161126 10:00:23.875667 50197 util/stop/stopper.go:468  quiescing; tasks left:\n2      storage/queue.go:478\n2      storage/intent_resolver.go:380\n2      server/node.go:830\n2      kv/dist_sender.go:796\nI161126 10:00:23.875888 50197 util/stop/stopper.go:468  quiescing; tasks left:\n2      storage/queue.go:478\n2      storage/intent_resolver.go:380\n2      server/node.go:830\n1      kv/dist_sender.go:796\nW161126 10:00:23.876223 50181 storage/intent_resolver.go:334  [n1,s1,r8/1:/Table/5{2-3}]: failed to resolve intents: result is ambiguous (server shutdown)\nE161126 10:00:23.882530 46919 storage/queue.go:576  [n1,replicate,s1,r3/1:/Table/1{2-3}] change replicas of range 3 failed: result is ambiguous (server shutdown)\nW161126 10:00:23.882625 50180 storage/intent_resolver.go:334  [n1,s1,r3/1:/Table/1{2-3}]: failed to resolve intents: result is ambiguous (server shutdown)\nI161126 10:00:23.882735 50197 util/stop/stopper.go:468  quiescing; tasks left:\n2      storage/queue.go:478\n2      storage/intent_resolver.go:380\n1      server/node.go:830\nE161126 10:00:23.884396 46922 storage/queue.go:576  [n1,timeSeriesMaintenance,s1,r1/1:/{Min-Table/11}] result is ambiguous (server shutdown)\nI161126 10:00:23.884642 50197 util/stop/stopper.go:468  quiescing; tasks left:\n2      storage/intent_resolver.go:380\n1      storage/queue.go:478\nI161126 10:00:23.884852 50197 util/stop/stopper.go:468  quiescing; tasks left:\n1      storage/queue.go:478\n1      storage/intent_resolver.go:380\nI161126 10:00:23.885076 46843 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks\nW161126 10:00:23.907897 47815 gossip/infostore.go:303  [n4] node unavailable; try another peer\nW161126 10:00:23.910960 47387 gossip/infostore.go:303  [n3] node unavailable; try another peer\nW161126 10:00:23.924749 48200 gossip/infostore.go:303  [n5] node unavailable; try another peer\nW161126 10:00:23.949936 47169 gossip/infostore.go:303  [n2] node unavailable; try another peer\nW161126 10:00:23.986799 47815 gossip/infostore.go:303  [n4] node unavailable; try another peer\nW161126 10:00:23.999624 47387 gossip/infostore.go:303  [n3] node unavailable; try another peer\nW161126 10:00:24.023756 48200 gossip/infostore.go:303  [n5] node unavailable; try another peer\nI161126 10:00:24.036302 46947 kv/transport_race.go:71  transport race promotion: ran 124 iterations on up to 639 requests\nI161126 10:00:24.037333 46843 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks\nI161126 10:00:24.046116 46843 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks\nI161126 10:00:24.050574 46843 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks\nI161126 10:00:24.055099 46843 util/stop/stopper.go:396  stop has been called, stopping or quiescing all running tasks\n\treplicate_queue_test.go:67: /Table/53/0: split unexpected error: storage/replica_command.go:2362: range is already split at key /Table/53\n```"}}
{"actor":{"display_login":"cockroach-teamcity"}}