{"action":"created","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5417","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5417/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5417/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5417/events","html_url":"https://github.com/cockroachdb/cockroach/issues/5417","id":142192698,"number":5417,"title":"circleci: failed tests (14884): TestScanBatches","user":{"login":"tschottdorf","id":5076964,"avatar_url":"https://avatars.githubusercontent.com/u/5076964?v=3","gravatar_id":"","url":"https://api.github.com/users/tschottdorf","html_url":"https://github.com/tschottdorf","followers_url":"https://api.github.com/users/tschottdorf/followers","following_url":"https://api.github.com/users/tschottdorf/following{/other_user}","gists_url":"https://api.github.com/users/tschottdorf/gists{/gist_id}","starred_url":"https://api.github.com/users/tschottdorf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tschottdorf/subscriptions","organizations_url":"https://api.github.com/users/tschottdorf/orgs","repos_url":"https://api.github.com/users/tschottdorf/repos","events_url":"https://api.github.com/users/tschottdorf/events{/privacy}","received_events_url":"https://api.github.com/users/tschottdorf/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/cockroachdb/cockroach/labels/Robot","name":"Robot","color":"5319e7"},{"url":"https://api.github.com/repos/cockroachdb/cockroach/labels/test-failure","name":"test-failure","color":"e11d21"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/1","html_url":"https://github.com/cockroachdb/cockroach/milestones/Beta","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/milestones/1/labels","id":778698,"number":1,"title":"Beta","description":"","creator":{"login":"andybons","id":365204,"avatar_url":"https://avatars.githubusercontent.com/u/365204?v=3","gravatar_id":"","url":"https://api.github.com/users/andybons","html_url":"https://github.com/andybons","followers_url":"https://api.github.com/users/andybons/followers","following_url":"https://api.github.com/users/andybons/following{/other_user}","gists_url":"https://api.github.com/users/andybons/gists{/gist_id}","starred_url":"https://api.github.com/users/andybons/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andybons/subscriptions","organizations_url":"https://api.github.com/users/andybons/orgs","repos_url":"https://api.github.com/users/andybons/repos","events_url":"https://api.github.com/users/andybons/events{/privacy}","received_events_url":"https://api.github.com/users/andybons/received_events","type":"User","site_admin":false},"open_issues":90,"closed_issues":489,"state":"open","created_at":"2014-09-04T16:42:18Z","updated_at":"2016-03-20T21:39:52Z","due_on":null,"closed_at":null},"comments":1,"created_at":"2016-03-20T18:56:05Z","updated_at":"2016-03-20T23:09:21Z","closed_at":null,"body":"The following test appears to have failed:\n\n[#14884](https://circleci.com/gh/cockroachdb/cockroach/14884):\n\n```\nI160320 18:55:29.103937 kv/range_cache.go:228  evict cached descriptor: key=/Meta2/Table/2/1/50/\"scan\"/3/1 desc=<nil>\nI160320 18:55:29.103991 kv/range_cache.go:228  evict cached descriptor: key=/Meta1/Table/2/1/50/\"scan\"/3/1 desc=<nil>\nI160320 18:55:29.104047 client/db.go:422  failed batch: failed to send RPC: too many errors encountered (1 of 1 total): server/node.go:702: node 1 stopped\nW160320 18:55:29.104127 storage/intent_resolver.go:283  [node=1,store=1,range=6] failed to resolve intents: failed to send RPC: too many errors encountered (1 of 1 total): server/node.go:702: node 1 stopped\nI160320 18:55:30.103504 /go/src/google.golang.org/grpc/clientconn.go:463  grpc: Conn.resetTransport failed to create client transport: connection error: desc = \"transport: dial tcp 127.0.0.1:60002: getsockopt: connection refused\"; Reconnecting to \"127.0.0.1:60002\"\npanic: test timed out after 1m10s\n\ngoroutine 22788 [running]:\npanic(0x1481600, 0xc8212f9fa0)\n\t/usr/local/go/src/runtime/panic.go:464 +0x3e6\ntesting.startAlarm.func1()\n\t/usr/local/go/src/testing/testing.go:725 +0x14b\ncreated by time.goFunc\n\t/usr/local/go/src/time/sleep.go:129 +0x3a\n\ngoroutine 1 [chan receive]:\ntesting.RunTests(0x1bb1108, 0x232a2e0, 0x48, 0x48, 0xbbb51295fdcb3201)\n\t/usr/local/go/src/testing/testing.go:583 +0x8d2\ntesting.(*M).Run(0xc820e5af08, 0x11)\n\t/usr/local/go/src/testing/testing.go:515 +0x81\ngithub.com/cockroachdb/cockroach/sql_test.TestMain(0xc820e5af08)\n\t/go/src/github.com/cockroachdb/cockroach/sql/main_test.go:234 +0x26\nmain.main()\n\tgithub.com/cockroachdb/cockroach/sql/_test/_testmain.go:382 +0x114\n\ngoroutine 17 [syscall, 1 minutes, locked to thread]:\nruntime.goexit()\n\t/usr/local/go/src/runtime/asm_amd64.s:1998 +0x1\n\ngoroutine 8 [chan receive]:\ngithub.com/cockroachdb/cockroach/util/log.(*loggingT).flushDaemon(0x25fdf80)\n\t/go/src/github.com/cockroachdb/cockroach/util/log/clog.go:1003 +0x64\ncreated by github.com/cockroachdb/cockroach/util/log.init.1\n\t/go/src/github.com/cockroachdb/cockroach/util/log/clog.go:604 +0x8a\n\ngoroutine 22428 [semacquire]:\nsync.runtime_Semacquire(0xc82128a024)\n\t/usr/local/go/src/runtime/sema.go:47 +0x26\nsync.(*WaitGroup).Wait(0xc82128a018)\n\t/usr/local/go/src/sync/waitgroup.go:127 +0xb4\ngithub.com/cockroachdb/cockroach/util/stop.(*Stopper).Stop(0xc82128a000)\n\t/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:302 +0x1dd\ngithub.com/cockroachdb/cockroach/server.(*Server).Stop(0xc8201100d0)\n\t/go/src/github.com/cockroachdb/cockroach/server/server.go:359 +0x28\ngithub.com/cockroachdb/cockroach/server.(*TestServer).Stop(0xc8202160e0)\n\t/go/src/github.com/cockroachdb/cockroach/server/testserver.go:318 +0x5d\n--\ntesting.tRunner(0xc820548f30, 0x232a820)\n\t/usr/local/go/src/testing/testing.go:473 +0x98\ncreated by testing.RunTests\n\t/usr/local/go/src/testing/testing.go:582 +0x892\n\ngoroutine 22399 [IO wait]:\nnet.runtime_pollWait(0x7ff8b7d61748, 0x72, 0xc820a55400)\n\t/usr/local/go/src/runtime/netpoll.go:160 +0x60\nnet.(*pollDesc).Wait(0xc820d861b0, 0x72, 0x0, 0x0)\n\t/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a\nnet.(*pollDesc).WaitRead(0xc820d861b0, 0x0, 0x0)\n\t/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36\nnet.(*netFD).Read(0xc820d86150, 0xc820a55400, 0x400, 0x400, 0x0, 0x7ff8bb741050, 0xc820010048)\n\t/usr/local/go/src/net/fd_unix.go:250 +0x23a\nnet.(*conn).Read(0xc8202d0010, 0xc820a55400, 0x400, 0x400, 0x0, 0x0, 0x0)\n\t/usr/local/go/src/net/net.go:172 +0xe4\n--\ngoogle.golang.org/grpc.(*Server).handleRawConn(0xc8203e3600, 0x7ff8bb759500, 0xc8205ec000)\n\t/go/src/google.golang.org/grpc/server.go:289 +0x4ee\ncreated by google.golang.org/grpc.(*Server).Serve\n\t/go/src/google.golang.org/grpc/server.go:261 +0x372\n\ngoroutine 22445 [select]:\ngoogle.golang.org/grpc.(*Conn).transportMonitor(0xc8211ea2a0)\n\t/go/src/google.golang.org/grpc/clientconn.go:511 +0x1d3\ngoogle.golang.org/grpc.NewConn.func1(0xc8211ea2a0)\n\t/go/src/google.golang.org/grpc/clientconn.go:322 +0x1b5\ncreated by google.golang.org/grpc.NewConn\n\t/go/src/google.golang.org/grpc/clientconn.go:323 +0x509\n\ngoroutine 22435 [chan receive]:\ngithub.com/cockroachdb/cockroach/storage/engine.(*RocksDB).Open.func1(0xc820cd6b40)\n\t/go/src/github.com/cockroachdb/cockroach/storage/engine/rocksdb.go:135 +0x3a\ncreated by github.com/cockroachdb/cockroach/storage/engine.(*RocksDB).Open\n\t/go/src/github.com/cockroachdb/cockroach/storage/engine/rocksdb.go:136 +0x7c9\n\ngoroutine 22446 [chan receive]:\ngithub.com/cockroachdb/cockroach/server.RegisterAdminHandlerFromEndpoint.func1.1(0x7ff8b7d60e90, 0xc820850c40, 0xc821216820, 0xc8205668c0, 0xf)\n\t/go/src/github.com/cockroachdb/cockroach/server/admin.pb.gw.go:177 +0x51\ncreated by github.com/cockroachdb/cockroach/server.RegisterAdminHandlerFromEndpoint.func1\n\t/go/src/github.com/cockroachdb/cockroach/server/admin.pb.gw.go:181 +0x205\n\ngoroutine 22401 [select]:\ngoogle.golang.org/grpc/transport.(*http2Client).controller(0xc820d944b0)\n\t/go/src/google.golang.org/grpc/transport/http2_client.go:831 +0x5da\ncreated by google.golang.org/grpc/transport.newHTTP2Client\n\t/go/src/google.golang.org/grpc/transport/http2_client.go:194 +0x153b\n\ngoroutine 22482 [IO wait]:\nnet.runtime_pollWait(0x7ff8b7d61ec8, 0x72, 0xc820730000)\n\t/usr/local/go/src/runtime/netpoll.go:160 +0x60\nnet.(*pollDesc).Wait(0xc820d86140, 0x72, 0x0, 0x0)\n\t/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a\nnet.(*pollDesc).WaitRead(0xc820d86140, 0x0, 0x0)\n\t/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36\nnet.(*netFD).Read(0xc820d860e0, 0xc820730000, 0x1000, 0x1000, 0x0, 0x7ff8bb741050, 0xc820010048)\n\t/usr/local/go/src/net/fd_unix.go:250 +0x23a\nnet.(*conn).Read(0xc8202d0008, 0xc820730000, 0x1000, 0x1000, 0x0, 0x0, 0x0)\n\t/usr/local/go/src/net/net.go:172 +0xe4\n--\ngoogle.golang.org/grpc/transport.(*http2Client).reader(0xc820d944b0)\n\t/go/src/google.golang.org/grpc/transport/http2_client.go:754 +0xef\ncreated by google.golang.org/grpc/transport.newHTTP2Client\n\t/go/src/google.golang.org/grpc/transport/http2_client.go:200 +0x159a\n\ngoroutine 22400 [select]:\ngoogle.golang.org/grpc/transport.(*http2Server).controller(0xc82080e090)\n\t/go/src/google.golang.org/grpc/transport/http2_server.go:610 +0x5da\ncreated by google.golang.org/grpc/transport.newHTTP2Server\n\t/go/src/google.golang.org/grpc/transport/http2_server.go:134 +0x853\n\ngoroutine 22509 [select]:\ngoogle.golang.org/grpc.(*Conn).Wait(0xc8211ead20, 0x7ff8bb759e48, 0xc82076a300, 0x0, 0x0, 0x0, 0x0)\n\t/go/src/google.golang.org/grpc/clientconn.go:552 +0x360\ngoogle.golang.org/grpc.(*unicastPicker).Pick(0xc820d96580, 0x7ff8bb759e48, 0xc82076a300, 0x0, 0x0, 0x0, 0x0)\n\t/go/src/google.golang.org/grpc/picker.go:84 +0x51\ngoogle.golang.org/grpc.Invoke(0x7ff8bb759e48, 0xc82076a300, 0x1932460, 0x1d, 0x17ab160, 0xc820802480, 0x179d8e0, 0xc820c8ecc0, 0xc820c76960, 0x0, ...)\n\t/go/src/google.golang.org/grpc/call.go:154 +0x6d0\ngithub.com/cockroachdb/cockroach/rpc.(*heartbeatClient).Ping(0xc8202d0030, 0x7ff8bb759e48, 0xc82076a300, 0xc820802480, 0x0, 0x0, 0x0, 0xc820538c90, 0x0, 0x0)\n\t/go/src/github.com/cockroachdb/cockroach/rpc/heartbeat.pb.go:115 +0xec\ngithub.com/cockroachdb/cockroach/rpc.(*Context).runHeartbeat(0xc820648b40, 0xc820c76960, 0xc82041db20, 0xf, 0x0, 0x0)\n\t/go/src/github.com/cockroachdb/cockroach/rpc/context.go:183 +0x2b5\n--\ngithub.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc82128a000, 0xc8205fa4b0)\n\t/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:139 +0x52\ncreated by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker\n\t/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:140 +0x62\n\ngoroutine 22508 [sleep]:\ntime.Sleep(0x4e033f78)\n\t/usr/local/go/src/runtime/time.go:59 +0xf9\ngoogle.golang.org/grpc.(*Conn).resetTransport(0xc8211ead20, 0x0, 0x0, 0x0)\n\t/go/src/google.golang.org/grpc/clientconn.go:461 +0x79f\ngoogle.golang.org/grpc.NewConn.func1(0xc8211ead20)\n\t/go/src/google.golang.org/grpc/clientconn.go:317 +0x2a\ncreated by google.golang.org/grpc.NewConn\n\t/go/src/google.golang.org/grpc/clientconn.go:323 +0x509\nFAIL\tgithub.com/cockroachdb/cockroach/sql\t70.056s\n=== RUN   TestEval\n--- PASS: TestEval (0.01s)\n=== RUN   TestEvalError\n--- PASS: TestEvalError (0.00s)\n=== RUN   TestEvalComparisonExprCaching\n--- PASS: TestEvalComparisonExprCaching (0.00s)\n=== RUN   TestSimilarEscape\n--- PASS: TestSimilarEscape (0.00s)\n=== RUN   TestQualifiedNameString\n--- PASS: TestQualifiedNameString (0.00s)\n\n```\nPlease assign, take a look and update the issue accordingly."},"comment":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/comments/199057096","html_url":"https://github.com/cockroachdb/cockroach/issues/5417#issuecomment-199057096","issue_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/5417","id":199057096,"user":{"login":"RaduBerinde","id":16544120,"avatar_url":"https://avatars.githubusercontent.com/u/16544120?v=3","gravatar_id":"","url":"https://api.github.com/users/RaduBerinde","html_url":"https://github.com/RaduBerinde","followers_url":"https://api.github.com/users/RaduBerinde/followers","following_url":"https://api.github.com/users/RaduBerinde/following{/other_user}","gists_url":"https://api.github.com/users/RaduBerinde/gists{/gist_id}","starred_url":"https://api.github.com/users/RaduBerinde/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RaduBerinde/subscriptions","organizations_url":"https://api.github.com/users/RaduBerinde/orgs","repos_url":"https://api.github.com/users/RaduBerinde/repos","events_url":"https://api.github.com/users/RaduBerinde/events{/privacy}","received_events_url":"https://api.github.com/users/RaduBerinde/received_events","type":"User","site_admin":false},"created_at":"2016-03-20T23:09:21Z","updated_at":"2016-03-20T23:09:21Z","body":"Dup of #5394 / #5376. Leaving open or it will just get filed again.\r\n\r\nTestScanBatches takes ~3 seconds so it's not the cause of the timeout but it's still a bit high. I verified that the scanning itself is fast (~0.5s), it's the cleanup at the end of the test that takes a few seconds."}}