{"action":"opened","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/36620","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/36620/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/36620/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/36620/events","html_url":"https://github.com/cockroachdb/cockroach/issues/36620","id":430563537,"node_id":"MDU6SXNzdWU0MzA1NjM1Mzc=","number":36620,"title":"teamcity: failed test: TestChangefeedMonitoring","user":{"login":"cockroach-teamcity","id":20559710,"node_id":"MDQ6VXNlcjIwNTU5NzEw","avatar_url":"https://avatars0.githubusercontent.com/u/20559710?v=4","gravatar_id":"","url":"https://api.github.com/users/cockroach-teamcity","html_url":"https://github.com/cockroach-teamcity","followers_url":"https://api.github.com/users/cockroach-teamcity/followers","following_url":"https://api.github.com/users/cockroach-teamcity/following{/other_user}","gists_url":"https://api.github.com/users/cockroach-teamcity/gists{/gist_id}","starred_url":"https://api.github.com/users/cockroach-teamcity/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cockroach-teamcity/subscriptions","organizations_url":"https://api.github.com/users/cockroach-teamcity/orgs","repos_url":"https://api.github.com/users/cockroach-teamcity/repos","events_url":"https://api.github.com/users/cockroach-teamcity/events{/privacy}","received_events_url":"https://api.github.com/users/cockroach-teamcity/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/17","html_url":"https://github.com/cockroachdb/cockroach/milestone/17","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/17/labels","id":3492386,"node_id":"MDk6TWlsZXN0b25lMzQ5MjM4Ng==","number":17,"title":"19.1","description":"","creator":{"login":"petermattis","id":6201335,"node_id":"MDQ6VXNlcjYyMDEzMzU=","avatar_url":"https://avatars0.githubusercontent.com/u/6201335?v=4","gravatar_id":"","url":"https://api.github.com/users/petermattis","html_url":"https://github.com/petermattis","followers_url":"https://api.github.com/users/petermattis/followers","following_url":"https://api.github.com/users/petermattis/following{/other_user}","gists_url":"https://api.github.com/users/petermattis/gists{/gist_id}","starred_url":"https://api.github.com/users/petermattis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/petermattis/subscriptions","organizations_url":"https://api.github.com/users/petermattis/orgs","repos_url":"https://api.github.com/users/petermattis/repos","events_url":"https://api.github.com/users/petermattis/events{/privacy}","received_events_url":"https://api.github.com/users/petermattis/received_events","type":"User","site_admin":false},"open_issues":230,"closed_issues":1839,"state":"open","created_at":"2018-07-12T18:50:47Z","updated_at":"2019-04-08T17:11:12Z","due_on":null,"closed_at":null},"comments":0,"created_at":"2019-04-08T17:11:12Z","updated_at":"2019-04-08T17:11:12Z","closed_at":null,"author_association":"MEMBER","body":"The following tests appear to have failed on master (test): TestChangefeedMonitoring, TestChangefeedMonitoring/sinkless\n\nYou may want to check [for open issues](https://github.com/cockroachdb/cockroach/issues?q=is%3Aissue+is%3Aopen+TestChangefeedMonitoring).\n\n[#1228307](https://teamcity.cockroachdb.com/viewLog.html?buildId=1228307):\n\n\n```\nTestChangefeedMonitoring\n--- FAIL: test/TestChangefeedMonitoring (47.250s)\n\n\n\n\nTestChangefeedMonitoring\n--- FAIL: test/TestChangefeedMonitoring (59.260s)\nNone\n\n\n\nTestChangefeedMonitoring/sinkless\n...te-default-db] Event: \"create_database\", target: 50, info: {DatabaseName:defaultdb Statement:CREATE DATABASE IF NOT EXISTS defaultdb User:root}\r\nI190408 16:45:41.574434 51751 sql/event_log.go:135  [n1,intExec=create-default-db] Event: \"create_database\", target: 51, info: {DatabaseName:postgres Statement:CREATE DATABASE IF NOT EXISTS postgres User:root}\r\nI190408 16:45:41.686951 51414 server/server.go:1653  [n1] done ensuring all necessary migrations have run\r\nI190408 16:45:41.687093 51414 server/server.go:1656  [n1] serving sql connections\r\nI190408 16:45:41.722651 51986 server/server_update.go:67  [n1] no need to upgrade, cluster already at the newest version\r\nI190408 16:45:41.747836 51989 sql/event_log.go:135  [n1] Event: \"node_join\", target: 1, info: {Descriptor:{NodeID:1 Address:127.0.0.1:33031 Attrs: Locality: ServerVersion:19.1 BuildTag:v19.1.0-beta.20190318-712-g5c2cc21 StartedAt:1554741940234034666 LocalityAddress:[] XXX_NoUnkeyedLiteral:{} XXX_sizecache:0} ClusterID:e993605d-5d10-416c-b675-2688779ecd08 StartedAt:1554741940234034666 LastUp:1554741940234034666}\r\nI190408 16:45:41.876038 51729 sql/event_log.go:135  [n1,client=127.0.0.1:54016,user=root] Event: \"set_cluster_setting\", target: 0, info: {SettingName:kv.rangefeed.enabled Value:true User:root}\r\nI190408 16:45:41.940361 51729 sql/event_log.go:135  [n1,client=127.0.0.1:54016,user=root] Event: \"set_cluster_setting\", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:00:00:01 User:root}\r\nI190408 16:45:41.995252 51729 sql/event_log.go:135  [n1,client=127.0.0.1:54016,user=root] Event: \"set_cluster_setting\", target: 0, info: {SettingName:changefeed.experimental_poll_interval Value:00:00:00.01 User:root}\r\nI190408 16:45:42.045198 51729 sql/event_log.go:135  [n1,client=127.0.0.1:54016,user=root] Event: \"create_database\", target: 52, info: {DatabaseName:d Statement:CREATE DATABASE d User:root}\r\nI190408 16:45:42.112960 51729 sql/event_log.go:135  [n1,client=127.0.0.1:54016,user=root] Event: \"create_table\", target: 53, info: {TableName:d.public.foo Statement:CREATE TABLE foo (a INT8 PRIMARY KEY) User:root}\r\nI190408 16:45:42.453843 51729 sql/event_log.go:135  [n1,client=127.0.0.1:54016,user=root] Event: \"set_cluster_setting\", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:00:00:00.01 User:root}\r\nI190408 16:45:42.512459 51509 gossip/gossip.go:1510  [n1] node has connected to cluster via gossip\r\nI190408 16:45:42.513042 51509 storage/stores.go:263  [n1] wrote 0 node addresses to persistent storage\r\nI190408 16:45:42.726972 52075 storage/replica_command.go:246  [n1,split,s1,r20/1:/{Table/24-Max}] initiating a split of this range at key /Table/53 [r21] (zone config)\r\nI190408 16:45:42.732851 51729 sql/event_log.go:135  [n1,client=127.0.0.1:54016,user=root] Event: \"set_cluster_setting\", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:00:00:01 User:root}\r\nI190408 16:45:42.797421 51544 storage/queue.go:435  [n1,s1,r20/1:/Table/{24-53}] rate limited in maybeadd-replicate: throttled on async limiting semaphore\r\nI190408 16:45:42.992962 51414 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] transport racer\r\n1      [async] storage.IntentResolver: cleanup txn records\r\n1      [async] intent_resolver_ir_batcher\r\n1      [async] intent_resolver_gc_batcher\r\n1      [async] closedts-rangefeed-subscriber\r\nW190408 16:45:42.995464 52238 storage/intentresolver/intent_resolver.go:820  failed to gc transaction record: could not GC completed transaction anchored at /Local/Range/Table/24/RangeDescriptor: node unavailable; try another peer\r\nI190408 16:45:42.996553 51414 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] transport racer\r\n1      [async] closedts-rangefeed-subscriber\r\nI190408 16:45:42.998580 51414 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] transport racer\r\nI190408 16:45:43.237172 51612 kv/transport_race.go:113  transport race promotion: ran 16 iterations on up to 417 requests\n\n\n\nTestChangefeedMonitoring/sinkless\n... the feed to be < 100000000 nanos behind got 38391424598\r\nI190408 16:41:13.566686 36685 ccl/changefeedccl/changefeed_test.go:1010  SucceedsSoon: waiting for the feed to be < 100000000 nanos behind got 39391712443\r\nI190408 16:41:13.953741 37013 server/status/runtime.go:500  [n1] runtime stats: 157 MiB RSS, 198 goroutines, 21 MiB/35 MiB/71 MiB GO alloc/idle/total, 17 MiB/52 MiB CGO alloc/total, 2446.9 CGO/sec, 10.6/2.3 %(u/s)time, 0.0 %gc (1x), 8.8 MiB/8.8 MiB (r/w)net\r\nI190408 16:41:14.566941 36685 ccl/changefeedccl/changefeed_test.go:1010  SucceedsSoon: waiting for the feed to be < 100000000 nanos behind got 40391966600\r\nI190408 16:41:15.567524 36685 ccl/changefeedccl/changefeed_test.go:1010  SucceedsSoon: waiting for the feed to be < 100000000 nanos behind got 41392536365\r\nI190408 16:41:16.567778 36685 ccl/changefeedccl/changefeed_test.go:1010  SucceedsSoon: waiting for the feed to be < 100000000 nanos behind got 42392776761\r\nI190408 16:41:17.568087 36685 ccl/changefeedccl/changefeed_test.go:1010  SucceedsSoon: waiting for the feed to be < 100000000 nanos behind got 43393108325\r\nI190408 16:41:18.568374 36685 ccl/changefeedccl/changefeed_test.go:1010  SucceedsSoon: waiting for the feed to be < 100000000 nanos behind got 44393406193\r\nI190408 16:41:19.569108 36685 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] stats.Refresher: maybeRefreshStats\r\n1      [async] rangefeed: output loop\r\n1      [async] intent_resolver_ir_batcher\r\n1      [async] intent_resolver_gc_batcher\r\n1      [async] closedts-rangefeed-subscriber\r\n1      [async] changefeed-poller\r\nI190408 16:41:19.571858 36685 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] rangefeed: output loop\r\n1      [async] intent_resolver_ir_batcher\r\n1      [async] intent_resolver_gc_batcher\r\n1      [async] closedts-rangefeed-subscriber\r\n1      [async] changefeed-poller\r\nI190408 16:41:19.571916 36685 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] rangefeed: output loop\r\n1      [async] intent_resolver_ir_batcher\r\n1      [async] closedts-rangefeed-subscriber\r\n1      [async] changefeed-poller\r\nI190408 16:41:19.571946 36685 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] rangefeed: output loop\r\n1      [async] closedts-rangefeed-subscriber\r\n1      [async] changefeed-poller\r\nI190408 16:41:19.572186 36685 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] closedts-rangefeed-subscriber\r\n1      [async] changefeed-poller\r\nI190408 16:41:19.575916 36685 util/stop/stopper.go:546  quiescing; tasks left:\r\n1      [async] closedts-rangefeed-subscriber\r\nW190408 16:41:19.580498 37296 internal/client/txn.go:510  [n1,client=127.0.0.1:54162,user=root] failure aborting transaction: node unavailable; try another peer; abort caused by: context canceled\r\n        changefeed_test.go:1010: condition failed to evaluate within 45s: waiting for the feed to be < 100000000 nanos behind got 44393406193\r\n            goroutine 36685 [running]:\r\n            runtime/debug.Stack(0xa7a358200, 0xc001520b40, 0x3918500)\r\n            \t/usr/local/go/src/runtime/debug/stack.go:24 +0xa7\r\n            github.com/cockroachdb/cockroach/pkg/testutils.SucceedsSoon(0x3988320, 0xc002626100, 0xc0015799a0)\r\n            \t/go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:49 +0x103\r\n            github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.TestChangefeedMonitoring.func1(0xc002626100, 0xc00118a600, 0x3920960, 0xc001752090)\r\n            \t/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/changefeed_test.go:1010 +0x90d\r\n            github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl.sinklessTest.func1(0xc002626100)\r\n            \t/go/src/github.com/cockroachdb/cockroach/pkg/ccl/changefeedccl/helpers_test.go:239 +0x5ad\r\n            testing.tRunner(0xc002626100, 0xc0003a8060)\r\n            \t/usr/local/go/src/testing/testing.go:827 +0xbf\r\n            created by testing.(*T).Run\r\n            \t/usr/local/go/src/testing/testing.go:878 +0x35c\n\n\n\n\n```\n\nPlease assign, take a look and update the issue accordingly.\n"}}
{"actor":{"display_login":"cockroach-teamcity"}}