{"action":"closed","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31000","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31000/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31000/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/31000/events","html_url":"https://github.com/cockroachdb/cockroach/issues/31000","id":367096656,"node_id":"MDU6SXNzdWUzNjcwOTY2NTY=","number":31000,"title":"ccl/importccl: TestMultiNodeExportStmt failed under stress","user":{"login":"cockroach-teamcity","id":20559710,"node_id":"MDQ6VXNlcjIwNTU5NzEw","avatar_url":"https://avatars0.githubusercontent.com/u/20559710?v=4","gravatar_id":"","url":"https://api.github.com/users/cockroach-teamcity","html_url":"https://github.com/cockroach-teamcity","followers_url":"https://api.github.com/users/cockroach-teamcity/followers","following_url":"https://api.github.com/users/cockroach-teamcity/following{/other_user}","gists_url":"https://api.github.com/users/cockroach-teamcity/gists{/gist_id}","starred_url":"https://api.github.com/users/cockroach-teamcity/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cockroach-teamcity/subscriptions","organizations_url":"https://api.github.com/users/cockroach-teamcity/orgs","repos_url":"https://api.github.com/users/cockroach-teamcity/repos","events_url":"https://api.github.com/users/cockroach-teamcity/events{/privacy}","received_events_url":"https://api.github.com/users/cockroach-teamcity/received_events","type":"User","site_admin":false},"labels":[{"id":191587409,"node_id":"MDU6TGFiZWwxOTE1ODc0MDk=","url":"https://api.github.com/repos/cockroachdb/cockroach/labels/C-test-failure","name":"C-test-failure","color":"c5def5","default":false},{"id":335217860,"node_id":"MDU6TGFiZWwzMzUyMTc4NjA=","url":"https://api.github.com/repos/cockroachdb/cockroach/labels/O-robot","name":"O-robot","color":"666ad1","default":false}],"state":"closed","locked":false,"assignee":{"login":"dt","id":15615,"node_id":"MDQ6VXNlcjE1NjE1","avatar_url":"https://avatars2.githubusercontent.com/u/15615?v=4","gravatar_id":"","url":"https://api.github.com/users/dt","html_url":"https://github.com/dt","followers_url":"https://api.github.com/users/dt/followers","following_url":"https://api.github.com/users/dt/following{/other_user}","gists_url":"https://api.github.com/users/dt/gists{/gist_id}","starred_url":"https://api.github.com/users/dt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dt/subscriptions","organizations_url":"https://api.github.com/users/dt/orgs","repos_url":"https://api.github.com/users/dt/repos","events_url":"https://api.github.com/users/dt/events{/privacy}","received_events_url":"https://api.github.com/users/dt/received_events","type":"User","site_admin":false},"assignees":[{"login":"dt","id":15615,"node_id":"MDQ6VXNlcjE1NjE1","avatar_url":"https://avatars2.githubusercontent.com/u/15615?v=4","gravatar_id":"","url":"https://api.github.com/users/dt","html_url":"https://github.com/dt","followers_url":"https://api.github.com/users/dt/followers","following_url":"https://api.github.com/users/dt/following{/other_user}","gists_url":"https://api.github.com/users/dt/gists{/gist_id}","starred_url":"https://api.github.com/users/dt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dt/subscriptions","organizations_url":"https://api.github.com/users/dt/orgs","repos_url":"https://api.github.com/users/dt/repos","events_url":"https://api.github.com/users/dt/events{/privacy}","received_events_url":"https://api.github.com/users/dt/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/13","html_url":"https://github.com/cockroachdb/cockroach/milestone/13","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/13/labels","id":2745639,"node_id":"MDk6TWlsZXN0b25lMjc0NTYzOQ==","number":13,"title":"2.1","description":"","creator":{"login":"knz","id":642886,"node_id":"MDQ6VXNlcjY0Mjg4Ng==","avatar_url":"https://avatars3.githubusercontent.com/u/642886?v=4","gravatar_id":"","url":"https://api.github.com/users/knz","html_url":"https://github.com/knz","followers_url":"https://api.github.com/users/knz/followers","following_url":"https://api.github.com/users/knz/following{/other_user}","gists_url":"https://api.github.com/users/knz/gists{/gist_id}","starred_url":"https://api.github.com/users/knz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/knz/subscriptions","organizations_url":"https://api.github.com/users/knz/orgs","repos_url":"https://api.github.com/users/knz/repos","events_url":"https://api.github.com/users/knz/events{/privacy}","received_events_url":"https://api.github.com/users/knz/received_events","type":"User","site_admin":false},"open_issues":68,"closed_issues":1845,"state":"open","created_at":"2017-09-05T15:30:49Z","updated_at":"2018-10-17T18:48:14Z","due_on":null,"closed_at":null},"comments":11,"created_at":"2018-10-05T07:46:06Z","updated_at":"2018-10-17T18:48:14Z","closed_at":"2018-10-17T18:48:13Z","author_association":"MEMBER","body":"SHA: https://github.com/cockroachdb/cockroach/commits/93a54f9be34d00b755e440bd76240a19f04650b5\n\nParameters:\n```\nTAGS=\nGOFLAGS=-race\n```\n\nTo repro, try:\n\n```\n# Don't forget to check out a clean suitable branch and experiment with the\n# stress invocation until the desired results present themselves. For example,\n# using stressrace instead of stress and passing the '-p' stressflag which\n# controls concurrency.\n./scripts/gceworker.sh start && ./scripts/gceworker.sh mosh\ncd ~/go/src/github.com/cockroachdb/cockroach && \\\nmake stress TESTS=TestMultiNodeExportStmt PKG=github.com/cockroachdb/cockroach/pkg/ccl/importccl TESTTIMEOUT=5m STRESSFLAGS='-stderr=false -maxtime 20m -timeout 10m'\n```\n\nFailed test: https://teamcity.cockroachdb.com/viewLog.html?buildId=945258&tab=buildLog\n\n```\n1      [async] closedts-rangefeed-subscriber\nI181005 07:45:43.353454 73145 util/stop/stopper.go:537  quiescing; tasks left:\n1      [async] transport racer\nW181005 07:45:43.353721 70173 storage/raft_transport.go:583  [n3] while processing outgoing Raft queue to node 2: EOF:\nW181005 07:45:43.355102 69469 gossip/gossip.go:1499  [n5] no incoming or outgoing connections\nI181005 07:45:43.355665 73146 util/stop/stopper.go:537  quiescing; tasks left:\n2      [async] closedts-subscription\nW181005 07:45:43.357106 68828 ts/db.go:198  [n3,ts-poll] node unavailable; try another peer\nW181005 07:45:43.358362 70170 storage/raft_transport.go:583  [n2] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:\nW181005 07:45:43.376954 70347 storage/raft_transport.go:583  [n3] while processing outgoing Raft queue to node 5: rpc error: code = Canceled desc = grpc: the client connection is closing:\nW181005 07:45:43.377880 67956 gossip/gossip.go:1499  [n1] no incoming or outgoing connections\nI181005 07:45:43.378177 73147 util/stop/stopper.go:537  quiescing; tasks left:\n3      [async] closedts-subscription\n1      node.Node: writing summary\n1      [async] closedts-rangefeed-subscriber\nW181005 07:45:43.381534 70393 storage/raft_transport.go:583  [n5] while processing outgoing Raft queue to node 3: EOF:\nW181005 07:45:43.381780 68829 server/node.go:886  [n3,summaries] health alerts detected: {Alerts:[{StoreID:3 Category:METRICS Description:queue.replicate.process.failure Value:1}]}\nW181005 07:45:43.387249 68829 server/node.go:857  [n3,summaries] error recording status summaries: node unavailable; try another peer\nI181005 07:45:43.387898 73147 util/stop/stopper.go:537  quiescing; tasks left:\n3      [async] closedts-subscription\n1      [async] closedts-rangefeed-subscriber\nI181005 07:45:43.388995 73149 util/stop/stopper.go:537  quiescing; tasks left:\n2      [async] closedts-subscription\n1      [async] closedts-rangefeed-subscriber\nI181005 07:45:43.390683 73149 util/stop/stopper.go:537  quiescing; tasks left:\n1      [async] closedts-subscription\n1      [async] closedts-rangefeed-subscriber\nI181005 07:45:43.391427 73147 util/stop/stopper.go:537  quiescing; tasks left:\n2      [async] closedts-subscription\n1      [async] closedts-rangefeed-subscriber\nI181005 07:45:43.392005 73149 util/stop/stopper.go:537  quiescing; tasks left:\n1      [async] closedts-subscription\nI181005 07:45:43.392329 73147 util/stop/stopper.go:537  quiescing; tasks left:\n2      [async] closedts-subscription\nW181005 07:45:43.445089 73222 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n5: context canceled\nW181005 07:45:43.445418 73220 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n3: context canceled\nW181005 07:45:43.527084 73234 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n2: context canceled\nW181005 07:45:43.527495 73235 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n1: context canceled\nW181005 07:45:43.561581 73204 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n4: context canceled\nW181005 07:45:43.561963 73205 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n5: context canceled\nW181005 07:45:43.563231 73206 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n1: context canceled\nW181005 07:45:43.570634 73207 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n2: context canceled\nW181005 07:45:43.573378 73209 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n2: context canceled\nW181005 07:45:43.586866 73208 rpc/nodedialer/nodedialer.go:89  [ct-client] unable to connect to n1: context canceled\nI181005 07:45:43.600274 68137 kv/transport_race.go:91  transport race promotion: ran 51 iterations on up to 4037 requests\nI181005 07:45:43.677078 69121 server/status/runtime.go:465  [n4] runtime stats: 1.7 GiB RSS, 434 goroutines, 36 MiB/82 MiB/146 MiB GO alloc/idle/total, 81 MiB/136 MiB CGO alloc/total, 2867.0 CGO/sec, 314.4/23.1 %(u/s)time, 1.3 %gc (16x), 2.5 MiB/2.5 MiB (r/w)net\n--- FAIL: TestMultiNodeExportStmt (44.70s)\n\ttesting.go:730: race detected during execution of test\n\n```"}}
{"actor":{"display_login":"dt"}}