{"action":"created","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31667","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31667/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31667/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31667/events","html_url":"https://github.com/cockroachdb/cockroach/issues/31667","id":372176694,"node_id":"MDU6SXNzdWUzNzIxNzY2OTQ=","number":31667,"title":"sql: TestCancelDistSQLQuery failed under stress","user":{"login":"cockroach-teamcity","id":20559710,"node_id":"MDQ6VXNlcjIwNTU5NzEw","avatar_url":"https://avatars0.githubusercontent.com/u/20559710?v=4","gravatar_id":"","url":"https://api.github.com/users/cockroach-teamcity","html_url":"https://github.com/cockroach-teamcity","followers_url":"https://api.github.com/users/cockroach-teamcity/followers","following_url":"https://api.github.com/users/cockroach-teamcity/following{/other_user}","gists_url":"https://api.github.com/users/cockroach-teamcity/gists{/gist_id}","starred_url":"https://api.github.com/users/cockroach-teamcity/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cockroach-teamcity/subscriptions","organizations_url":"https://api.github.com/users/cockroach-teamcity/orgs","repos_url":"https://api.github.com/users/cockroach-teamcity/repos","events_url":"https://api.github.com/users/cockroach-teamcity/events{/privacy}","received_events_url":"https://api.github.com/users/cockroach-teamcity/received_events","type":"User","site_admin":false},"labels":[{"id":191587409,"node_id":"MDU6TGFiZWwxOTE1ODc0MDk=","url":"https://api.github.com/repos/cockroachdb/cockroach/labels/C-test-failure","name":"C-test-failure","color":"c5def5","default":false},{"id":335217860,"node_id":"MDU6TGFiZWwzMzUyMTc4NjA=","url":"https://api.github.com/repos/cockroachdb/cockroach/labels/O-robot","name":"O-robot","color":"666ad1","default":false}],"state":"open","locked":false,"assignee":{"login":"benesch","id":882976,"node_id":"MDQ6VXNlcjg4Mjk3Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/882976?v=4","gravatar_id":"","url":"https://api.github.com/users/benesch","html_url":"https://github.com/benesch","followers_url":"https://api.github.com/users/benesch/followers","following_url":"https://api.github.com/users/benesch/following{/other_user}","gists_url":"https://api.github.com/users/benesch/gists{/gist_id}","starred_url":"https://api.github.com/users/benesch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benesch/subscriptions","organizations_url":"https://api.github.com/users/benesch/orgs","repos_url":"https://api.github.com/users/benesch/repos","events_url":"https://api.github.com/users/benesch/events{/privacy}","received_events_url":"https://api.github.com/users/benesch/received_events","type":"User","site_admin":false},"assignees":[{"login":"benesch","id":882976,"node_id":"MDQ6VXNlcjg4Mjk3Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/882976?v=4","gravatar_id":"","url":"https://api.github.com/users/benesch","html_url":"https://github.com/benesch","followers_url":"https://api.github.com/users/benesch/followers","following_url":"https://api.github.com/users/benesch/following{/other_user}","gists_url":"https://api.github.com/users/benesch/gists{/gist_id}","starred_url":"https://api.github.com/users/benesch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benesch/subscriptions","organizations_url":"https://api.github.com/users/benesch/orgs","repos_url":"https://api.github.com/users/benesch/repos","events_url":"https://api.github.com/users/benesch/events{/privacy}","received_events_url":"https://api.github.com/users/benesch/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/13","html_url":"https://github.com/cockroachdb/cockroach/milestone/13","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/13/labels","id":2745639,"node_id":"MDk6TWlsZXN0b25lMjc0NTYzOQ==","number":13,"title":"2.1","description":"","creator":{"login":"knz","id":642886,"node_id":"MDQ6VXNlcjY0Mjg4Ng==","avatar_url":"https://avatars3.githubusercontent.com/u/642886?v=4","gravatar_id":"","url":"https://api.github.com/users/knz","html_url":"https://github.com/knz","followers_url":"https://api.github.com/users/knz/followers","following_url":"https://api.github.com/users/knz/following{/other_user}","gists_url":"https://api.github.com/users/knz/gists{/gist_id}","starred_url":"https://api.github.com/users/knz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/knz/subscriptions","organizations_url":"https://api.github.com/users/knz/orgs","repos_url":"https://api.github.com/users/knz/repos","events_url":"https://api.github.com/users/knz/events{/privacy}","received_events_url":"https://api.github.com/users/knz/received_events","type":"User","site_admin":false},"open_issues":62,"closed_issues":1855,"state":"open","created_at":"2017-09-05T15:30:49Z","updated_at":"2018-10-21T21:51:49Z","due_on":null,"closed_at":null},"comments":0,"created_at":"2018-10-20T05:46:17Z","updated_at":"2018-10-21T22:13:23Z","closed_at":null,"author_association":"MEMBER","body":"SHA: https://github.com/cockroachdb/cockroach/commits/7ce0cba090528e7c50bd8fe708d536362f5d766a\n\nParameters:\n```\nTAGS=\nGOFLAGS=\n```\n\nTo repro, try:\n\n```\n# Don't forget to check out a clean suitable branch and experiment with the\n# stress invocation until the desired results present themselves. For example,\n# using stressrace instead of stress and passing the '-p' stressflag which\n# controls concurrency.\n./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh\ncd ~/go/src/github.com/cockroachdb/cockroach && \\\nmake stress TESTS=TestCancelDistSQLQuery PKG=github.com/cockroachdb/cockroach/pkg/sql TESTTIMEOUT=5m STRESSFLAGS='-stderr=false -maxtime 20m -timeout 10m'\n```\n\nFailed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=977304&tab=buildLog\n\n```\nI181020 05:44:48.386396 94410 server/server.go:1525  [n2] starting https server at 127.0.0.1:33211 (use: 127.0.0.1:33211)\nI181020 05:44:48.386410 94410 server/server.go:1527  [n2] starting grpc/postgres server at 127.0.0.1:36283\nI181020 05:44:48.386420 94410 server/server.go:1528  [n2] advertising CockroachDB node at 127.0.0.1:36283\nI181020 05:44:48.389210 94410 server/server.go:1581  [n2] done ensuring all necessary migrations have run\nI181020 05:44:48.389253 94410 server/server.go:1584  [n2] serving sql connections\nI181020 05:44:48.395899 94666 sql/event_log.go:126  [n1,client=127.0.0.1:52506,user=root] Event: \"create_database\", target: 52, info: {DatabaseName:test Statement:CREATE DATABASE IF NOT EXISTS test User:root}\nI181020 05:44:48.399466 94666 sql/event_log.go:126  [n1,client=127.0.0.1:52506,user=root] Event: \"create_table\", target: 53, info: {TableName:test.public.nums Statement:CREATE TABLE test.public.nums (num INT) User:root}\nI181020 05:44:48.407427 94666 storage/replica_command.go:298  [n1,s1,r1/1:/M{in-ax}] initiating a split of this range at key /Table/53/1/50 [r2]\nI181020 05:44:48.430162 93201 util/stop/stopper.go:537  quiescing; tasks left:\n3      node.Node: batch\n1      [async] closedts-rangefeed-subscriber\nI181020 05:44:48.430218 93201 util/stop/stopper.go:537  quiescing; tasks left:\n1      node.Node: batch\n1      [async] closedts-rangefeed-subscriber\nW181020 05:44:48.430432 94734 server/node.go:942  [n2] node=2: unable to log node_join event: log-event: failed to send RPC: sending to all 1 replicas failed; last error: <nil> rpc error: code = Unavailable desc = transport is closing\nW181020 05:44:48.430530 94874 internal/client/txn.go:532  [n2,hb] failure aborting transaction: node unavailable; try another peer; abort caused by: result is ambiguous (error=rpc error: code = Unavailable desc = transport is closing [exhausted])\nI181020 05:44:48.430557 94874 storage/node_liveness.go:817  [n2,hb] retrying liveness update after storage.errRetryLiveness: result is ambiguous (error=rpc error: code = Unavailable desc = transport is closing [exhausted])\nW181020 05:44:48.430656 94874 internal/client/txn.go:532  [n2,hb] failure aborting transaction: node unavailable; try another peer; abort caused by: node unavailable; try another peer\nW181020 05:44:48.430675 94874 storage/node_liveness.go:494  [n2,hb] failed node liveness heartbeat: node unavailable; try another peer\nW181020 05:44:48.430740 94868 ts/db.go:195  [n2,ts-poll] error writing time series data: failed to send RPC: sending to all 1 replicas failed; last error: <nil> rpc error: code = Unavailable desc = transport is closing\nI181020 05:44:48.430780 94930 util/stop/stopper.go:537  quiescing; tasks left:\n1      ts.poller: poll\n1      node.Node: writing summary\n1      [async] closedts-rangefeed-subscriber\n1      [async] auto-upgrade\nW181020 05:44:48.430813 94869 server/node.go:845  [n2,summaries] error recording initial status summaries: failed to send RPC: sending to all 1 replicas failed; last error: <nil> rpc error: code = Unavailable desc = transport is closing\nI181020 05:44:48.431048 94930 util/stop/stopper.go:537  quiescing; tasks left:\n1      [async] closedts-rangefeed-subscriber\n1      [async] auto-upgrade\nI181020 05:44:48.431405 93201 util/stop/stopper.go:537  quiescing; tasks left:\n1      node.Node: batch\nI181020 05:44:48.432732 94930 util/stop/stopper.go:537  quiescing; tasks left:\n1      [async] auto-upgrade\nW181020 05:44:48.433289 94898 internal/client/txn.go:532  [n2,intExec=read-setting] failure aborting transaction: node unavailable; try another peer; abort caused by: context canceled\nW181020 05:44:48.433440 94737 internal/client/txn.go:532  [n2,intExec=show-version] failure aborting transaction: node unavailable; try another peer; abort caused by: read-setting: context canceled\nI181020 05:44:48.433513 94732 server/server_update.go:64  [n2] failed attempt to upgrade cluster version, error: show-version: context canceled\n--- FAIL: TestCancelDistSQLQuery (0.33s)\n\trun_control_test.go:245: pq: cannot up-repliate to s2; missing gossiped StoreDescriptor\n\n```"},"comment":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/comments/431708852","html_url":"https://github.com/cockroachdb/cockroach/issues/31667#issuecomment-431708852","issue_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31667","id":431708852,"node_id":"MDEyOklzc3VlQ29tbWVudDQzMTcwODg1Mg==","user":{"login":"benesch","id":882976,"node_id":"MDQ6VXNlcjg4Mjk3Ng==","avatar_url":"https://avatars1.githubusercontent.com/u/882976?v=4","gravatar_id":"","url":"https://api.github.com/users/benesch","html_url":"https://github.com/benesch","followers_url":"https://api.github.com/users/benesch/followers","following_url":"https://api.github.com/users/benesch/following{/other_user}","gists_url":"https://api.github.com/users/benesch/gists{/gist_id}","starred_url":"https://api.github.com/users/benesch/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benesch/subscriptions","organizations_url":"https://api.github.com/users/benesch/orgs","repos_url":"https://api.github.com/users/benesch/repos","events_url":"https://api.github.com/users/benesch/events{/privacy}","received_events_url":"https://api.github.com/users/benesch/received_events","type":"User","site_admin":false},"created_at":"2018-10-21T22:13:23Z","updated_at":"2018-10-21T22:13:23Z","author_association":"MEMBER","body":"Alex's fix for this wasn't on release-2.1. Opened a backport (#31686)."}}
{"actor":{"display_login":"benesch"}}