{"action":"created","issue":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/8784","repository_url":"https://api.github.com/repos/cockroachdb/cockroach","labels_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/8784/labels{/name}","comments_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/8784/comments","events_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/8784/events","html_url":"https://github.com/cockroachdb/cockroach/issues/8784","id":172795949,"number":8784,"title":"stability: deadlock on processRaftMu","user":{"login":"tschottdorf","id":5076964,"avatar_url":"https://avatars.githubusercontent.com/u/5076964?v=3","gravatar_id":"","url":"https://api.github.com/users/tschottdorf","html_url":"https://github.com/tschottdorf","followers_url":"https://api.github.com/users/tschottdorf/followers","following_url":"https://api.github.com/users/tschottdorf/following{/other_user}","gists_url":"https://api.github.com/users/tschottdorf/gists{/gist_id}","starred_url":"https://api.github.com/users/tschottdorf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tschottdorf/subscriptions","organizations_url":"https://api.github.com/users/tschottdorf/orgs","repos_url":"https://api.github.com/users/tschottdorf/repos","events_url":"https://api.github.com/users/tschottdorf/events{/privacy}","received_events_url":"https://api.github.com/users/tschottdorf/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/cockroachdb/cockroach/labels/stability","name":"stability","color":"b60205"}],"state":"open","locked":false,"assignee":{"login":"a-robinson","id":7085343,"avatar_url":"https://avatars.githubusercontent.com/u/7085343?v=3","gravatar_id":"","url":"https://api.github.com/users/a-robinson","html_url":"https://github.com/a-robinson","followers_url":"https://api.github.com/users/a-robinson/followers","following_url":"https://api.github.com/users/a-robinson/following{/other_user}","gists_url":"https://api.github.com/users/a-robinson/gists{/gist_id}","starred_url":"https://api.github.com/users/a-robinson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/a-robinson/subscriptions","organizations_url":"https://api.github.com/users/a-robinson/orgs","repos_url":"https://api.github.com/users/a-robinson/repos","events_url":"https://api.github.com/users/a-robinson/events{/privacy}","received_events_url":"https://api.github.com/users/a-robinson/received_events","type":"User","site_admin":false},"assignees":[{"login":"a-robinson","id":7085343,"avatar_url":"https://avatars.githubusercontent.com/u/7085343?v=3","gravatar_id":"","url":"https://api.github.com/users/a-robinson","html_url":"https://github.com/a-robinson","followers_url":"https://api.github.com/users/a-robinson/followers","following_url":"https://api.github.com/users/a-robinson/following{/other_user}","gists_url":"https://api.github.com/users/a-robinson/gists{/gist_id}","starred_url":"https://api.github.com/users/a-robinson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/a-robinson/subscriptions","organizations_url":"https://api.github.com/users/a-robinson/orgs","repos_url":"https://api.github.com/users/a-robinson/repos","events_url":"https://api.github.com/users/a-robinson/events{/privacy}","received_events_url":"https://api.github.com/users/a-robinson/received_events","type":"User","site_admin":false},{"login":"sploiselle","id":19260003,"avatar_url":"https://avatars.githubusercontent.com/u/19260003?v=3","gravatar_id":"","url":"https://api.github.com/users/sploiselle","html_url":"https://github.com/sploiselle","followers_url":"https://api.github.com/users/sploiselle/followers","following_url":"https://api.github.com/users/sploiselle/following{/other_user}","gists_url":"https://api.github.com/users/sploiselle/gists{/gist_id}","starred_url":"https://api.github.com/users/sploiselle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sploiselle/subscriptions","organizations_url":"https://api.github.com/users/sploiselle/orgs","repos_url":"https://api.github.com/users/sploiselle/repos","events_url":"https://api.github.com/users/sploiselle/events{/privacy}","received_events_url":"https://api.github.com/users/sploiselle/received_events","type":"User","site_admin":false}],"milestone":null,"comments":37,"created_at":"2016-08-23T20:17:55Z","updated_at":"2016-08-29T15:36:28Z","closed_at":null,"body":"@sploiselle has a 4-node GCE cluster running beta-20160728 which has locked up, with the following in the goroutine dump:\r\n\r\n```\r\ngoroutine 218864 [semacquire, 47 minutes]:\r\nsync.runtime_Semacquire(0xc820188214)\r\n\t/usr/local/go/src/runtime/sema.go:47 +0x26\r\nsync.(*Mutex).Lock(0xc820188210)\r\n\t/usr/local/go/src/sync/mutex.go:83 +0x1c4\r\ngithub.com/cockroachdb/cockroach/storage.(*Store).handleRaftMessage(0xc820188000, 0xc8214845a0, 0x0, 0x0)\r\n\t/go/src/github.com/cockroachdb/cockroach/storage/store.go:2347 +0x7b\r\ngithub.com/cockroachdb/cockroach/storage.(*Store).(github.com/cockroachdb/cockroach/storage.handleRaftMessage)-fm(0xc8214845a0, 0x0, 0x0)\r\n\t/go/src/github.com/cockroachdb/cockroach/storage/store.go:1124 +0x38\r\ngithub.com/cockroachdb/cockroach/storage.(*RaftTransport).RaftMessage.func1.1.1(0x7f6bfca18830, 0xc8202221a0, 0xc8202f73b0, 0x0, 0x0)\r\n\t/go/src/github.com/cockroachdb/cockroach/storage/raft_transport.go:141 +0x2ef\r\ngithub.com/cockroachdb/cockroach/storage.(*RaftTransport).RaftMessage.func1.1()\r\n\t/go/src/github.com/cockroachdb/cockroach/storage/raft_transport.go:145 +0x48\r\ngithub.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker.func1(0xc8202e0d80, 0xc8216bd440)\r\n\t/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:187 +0x7d\r\ncreated by github.com/cockroachdb/cockroach/util/stop.(*Stopper).RunWorker\r\n\t/go/src/github.com/cockroachdb/cockroach/util/stop/stopper.go:188 +0x62\r\n```\r\n\r\nI'm investigating possible causes."},"comment":{"url":"https://api.github.com/repos/cockroachdb/cockroach/issues/comments/243161077","html_url":"https://github.com/cockroachdb/cockroach/issues/8784#issuecomment-243161077","issue_url":"https://api.github.com/repos/cockroachdb/cockroach/issues/8784","id":243161077,"user":{"login":"tschottdorf","id":5076964,"avatar_url":"https://avatars.githubusercontent.com/u/5076964?v=3","gravatar_id":"","url":"https://api.github.com/users/tschottdorf","html_url":"https://github.com/tschottdorf","followers_url":"https://api.github.com/users/tschottdorf/followers","following_url":"https://api.github.com/users/tschottdorf/following{/other_user}","gists_url":"https://api.github.com/users/tschottdorf/gists{/gist_id}","starred_url":"https://api.github.com/users/tschottdorf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tschottdorf/subscriptions","organizations_url":"https://api.github.com/users/tschottdorf/orgs","repos_url":"https://api.github.com/users/tschottdorf/repos","events_url":"https://api.github.com/users/tschottdorf/events{/privacy}","received_events_url":"https://api.github.com/users/tschottdorf/received_events","type":"User","site_admin":false},"created_at":"2016-08-29T15:36:28Z","updated_at":"2016-08-29T15:36:28Z","body":"Interesting, how do we botch `db_opts.num_cpu`? "}}
{"actor":{"display_login":"tschottdorf"}}