{"action":"created","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/28947","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/28947/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/28947/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/28947/events","html_url":"https://github.com/cockroachdb/cockroach/issues/28947","id":352882851,"node_id":"MDU6SXNzdWUzNTI4ODI4NTE=","number":28947,"title":"storage: very frequent near identical lease acquisition log messages","user":{"login":"tschottdorf","id":5076964,"node_id":"MDQ6VXNlcjUwNzY5NjQ=","avatar_url":"https://avatars1.githubusercontent.com/u/5076964?v=4","gravatar_id":"","url":"https://api.github.com/users/tschottdorf","html_url":"https://github.com/tschottdorf","followers_url":"https://api.github.com/users/tschottdorf/followers","following_url":"https://api.github.com/users/tschottdorf/following{/other_user}","gists_url":"https://api.github.com/users/tschottdorf/gists{/gist_id}","starred_url":"https://api.github.com/users/tschottdorf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tschottdorf/subscriptions","organizations_url":"https://api.github.com/users/tschottdorf/orgs","repos_url":"https://api.github.com/users/tschottdorf/repos","events_url":"https://api.github.com/users/tschottdorf/events{/privacy}","received_events_url":"https://api.github.com/users/tschottdorf/received_events","type":"User","site_admin":false},"labels":[{"id":912737912,"node_id":"MDU6TGFiZWw5MTI3Mzc5MTI=","url":"https://api.github.com/repos/cockroachdb/cockroach/labels/A-core-kv","name":"A-core-kv","color":"c2e0c6","default":false},{"id":78928345,"node_id":"MDU6TGFiZWw3ODkyODM0NQ==","url":"https://api.github.com/repos/cockroachdb/cockroach/labels/C-bug","name":"C-bug","color":"c5def5","default":false}],"state":"open","locked":false,"assignee":{"login":"nvanbenschoten","id":5438456,"node_id":"MDQ6VXNlcjU0Mzg0NTY=","avatar_url":"https://avatars3.githubusercontent.com/u/5438456?v=4","gravatar_id":"","url":"https://api.github.com/users/nvanbenschoten","html_url":"https://github.com/nvanbenschoten","followers_url":"https://api.github.com/users/nvanbenschoten/followers","following_url":"https://api.github.com/users/nvanbenschoten/following{/other_user}","gists_url":"https://api.github.com/users/nvanbenschoten/gists{/gist_id}","starred_url":"https://api.github.com/users/nvanbenschoten/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nvanbenschoten/subscriptions","organizations_url":"https://api.github.com/users/nvanbenschoten/orgs","repos_url":"https://api.github.com/users/nvanbenschoten/repos","events_url":"https://api.github.com/users/nvanbenschoten/events{/privacy}","received_events_url":"https://api.github.com/users/nvanbenschoten/received_events","type":"User","site_admin":false},"assignees":[{"login":"nvanbenschoten","id":5438456,"node_id":"MDQ6VXNlcjU0Mzg0NTY=","avatar_url":"https://avatars3.githubusercontent.com/u/5438456?v=4","gravatar_id":"","url":"https://api.github.com/users/nvanbenschoten","html_url":"https://github.com/nvanbenschoten","followers_url":"https://api.github.com/users/nvanbenschoten/followers","following_url":"https://api.github.com/users/nvanbenschoten/following{/other_user}","gists_url":"https://api.github.com/users/nvanbenschoten/gists{/gist_id}","starred_url":"https://api.github.com/users/nvanbenschoten/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nvanbenschoten/subscriptions","organizations_url":"https://api.github.com/users/nvanbenschoten/orgs","repos_url":"https://api.github.com/users/nvanbenschoten/repos","events_url":"https://api.github.com/users/nvanbenschoten/events{/privacy}","received_events_url":"https://api.github.com/users/nvanbenschoten/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/13","html_url":"https://github.com/cockroachdb/cockroach/milestone/13","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/13/labels","id":2745639,"node_id":"MDk6TWlsZXN0b25lMjc0NTYzOQ==","number":13,"title":"2.1","description":"","creator":{"login":"knz","id":642886,"node_id":"MDQ6VXNlcjY0Mjg4Ng==","avatar_url":"https://avatars3.githubusercontent.com/u/642886?v=4","gravatar_id":"","url":"https://api.github.com/users/knz","html_url":"https://github.com/knz","followers_url":"https://api.github.com/users/knz/followers","following_url":"https://api.github.com/users/knz/following{/other_user}","gists_url":"https://api.github.com/users/knz/gists{/gist_id}","starred_url":"https://api.github.com/users/knz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/knz/subscriptions","organizations_url":"https://api.github.com/users/knz/orgs","repos_url":"https://api.github.com/users/knz/repos","events_url":"https://api.github.com/users/knz/events{/privacy}","received_events_url":"https://api.github.com/users/knz/received_events","type":"User","site_admin":false},"open_issues":385,"closed_issues":1749,"state":"open","created_at":"2017-09-05T15:30:49Z","updated_at":"2018-09-29T18:49:43Z","due_on":null,"closed_at":null},"comments":8,"created_at":"2018-08-22T09:57:28Z","updated_at":"2018-09-29T18:53:31Z","closed_at":null,"author_association":"MEMBER","body":"Seen in the logs while debugging https://github.com/cockroachdb/cockroach/issues/28918.\r\n\r\nThere are 19 lease proposal messages for n2/r604 in a single second. The messages all have an identical lease, but different proposed timestamps. It's unclear to me why that should happen, so we likely have something to fix. And even in the unlikely case in which this is expected, the log is extremely spammy and needs to be throttled. \r\n\r\n```\r\n180821 13:58:11.423098 137 storage/replica_proposal.go:214  [n3,s3,r604/2:/Table/54/{2/945/1…-3}] new range lease repl=(n3,s3):2 seq=5 start=1534855664.670639431,0 epo=1 pro=1534859890.693085948,0 following repl=(n3,s3):2 seq=5 start=1534855664.670639431,0 epo=1 pro=1534859888.543800373,0\r\n180821 14:27:27.092828 120 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861646.829038018,0 following repl=(n3,s3):2 seq=5 start=1534855664.670639431,0 epo=1 pro=1534859890.693085948,0\r\n180821 14:27:40.704952 177 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.689998132,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861646.829038018,0\r\n180821 14:27:40.717496 177 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.705193324,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.689998132,0\r\n180821 14:27:40.736353 126 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.717712497,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.705193324,0\r\n180821 14:27:40.751478 123 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.736836520,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.717712497,0\r\n180821 14:27:40.763590 173 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.751758824,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.736836520,0\r\n180821 14:27:40.777126 168 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.763842466,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.751758824,0\r\n180821 14:27:40.791095 167 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.777457277,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.763842466,0\r\n180821 14:27:40.808234 171 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.791419487,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.777457277,0\r\n180821 14:27:40.822601 168 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.808601097,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.791419487,0\r\n180821 14:27:40.837601 122 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.822825197,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.808601097,0\r\n180821 14:27:40.864301 129 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.837791590,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.822825197,0\r\n180821 14:27:40.885089 126 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.865708675,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.837791590,0\r\n180821 14:27:40.901847 129 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.885296935,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.865708675,0\r\n180821 14:27:40.915856 173 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.902606623,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.885296935,0\r\n180821 14:27:40.929760 173 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.916097139,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.902606623,0\r\n180821 14:27:40.942607 122 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.930587554,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.916097139,0\r\n180821 14:27:40.959644 128 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.942824492,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.930587554,0\r\n180821 14:27:40.973122 178 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.960146739,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.942824492,0\r\n180821 14:27:40.988157 178 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.974917324,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.960146739,0\r\n180821 14:27:41.001183 179 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.988554372,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.974917324,0\r\n180821 14:27:41.015964 179 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.001552995,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861660.988554372,0\r\n180821 14:27:41.031030 178 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.016203856,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.001552995,0\r\n180821 14:27:41.047547 126 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.031225361,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.016203856,0\r\n180821 14:27:41.061015 128 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.047872992,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.031225361,0\r\n180821 14:27:41.074587 179 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.061302724,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.047872992,0\r\n180821 14:27:41.087465 174 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.074901102,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.061302724,0\r\n180821 14:27:41.102171 177 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.088295754,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.074901102,0\r\n180821 14:27:41.119045 177 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.102585015,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.088295754,0\r\n180821 14:27:41.135279 128 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.119410020,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.102585015,0\r\n180821 14:27:41.170818 122 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.135530480,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.119410020,0\r\n180821 14:27:41.188962 128 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.171110550,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.135530480,0\r\n180821 14:27:41.801610 176 storage/replica_proposal.go:214  [n2,s2,r604/3:/Table/54/{2/945/1…-3}] new range lease repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.189245834,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.171110550,0\r\n180821 20:41:48.882340 165 storage/replica_proposal.go:214  [n3,s3,r604/2:/Table/54/{2/945/1…-3}] new range lease repl=(n3,s3):2 seq=7 start=1534884105.354893619,0 epo=4 pro=1534884105.354897712,0 following repl=(n2,s2):3 seq=6 start=1534861646.829035154,0 epo=2 pro=1534861661.189245834,0\r\n```"},"comment":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/comments/425667740","html_url":"https://github.com/cockroachdb/cockroach/issues/28947#issuecomment-425667740","issue_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/28947","id":425667740,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNTY2Nzc0MA==","user":{"login":"tschottdorf","id":5076964,"node_id":"MDQ6VXNlcjUwNzY5NjQ=","avatar_url":"https://avatars1.githubusercontent.com/u/5076964?v=4","gravatar_id":"","url":"https://api.github.com/users/tschottdorf","html_url":"https://github.com/tschottdorf","followers_url":"https://api.github.com/users/tschottdorf/followers","following_url":"https://api.github.com/users/tschottdorf/following{/other_user}","gists_url":"https://api.github.com/users/tschottdorf/gists{/gist_id}","starred_url":"https://api.github.com/users/tschottdorf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tschottdorf/subscriptions","organizations_url":"https://api.github.com/users/tschottdorf/orgs","repos_url":"https://api.github.com/users/tschottdorf/repos","events_url":"https://api.github.com/users/tschottdorf/events{/privacy}","received_events_url":"https://api.github.com/users/tschottdorf/received_events","type":"User","site_admin":false},"created_at":"2018-09-29T18:53:31Z","updated_at":"2018-09-29T18:53:31Z","author_association":"MEMBER","body":"@nvanbenschoten could I put this on your plate? Basically just keep an eye out for the lease acquisition log spam when you run tpcc the next time (which I assume you do all the time). There's hopefully nothing to fix, but perhaps we want to cut down on the number of these messages."}}
{"actor":{"display_login":"tschottdorf"}}