{"action":"created","issue":{"url":"https://api.github.com/repos/scylladb/scylla/issues/721","labels_url":"https://api.github.com/repos/scylladb/scylla/issues/721/labels{/name}","comments_url":"https://api.github.com/repos/scylladb/scylla/issues/721/comments","events_url":"https://api.github.com/repos/scylladb/scylla/issues/721/events","html_url":"https://github.com/scylladb/scylla/issues/721","id":124082656,"number":721,"title":"Node's bootstrap process is too sensitive to networking problems ","user":{"login":"vladzcloudius","id":6103981,"avatar_url":"https://avatars.githubusercontent.com/u/6103981?v=3","gravatar_id":"","url":"https://api.github.com/users/vladzcloudius","html_url":"https://github.com/vladzcloudius","followers_url":"https://api.github.com/users/vladzcloudius/followers","following_url":"https://api.github.com/users/vladzcloudius/following{/other_user}","gists_url":"https://api.github.com/users/vladzcloudius/gists{/gist_id}","starred_url":"https://api.github.com/users/vladzcloudius/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vladzcloudius/subscriptions","organizations_url":"https://api.github.com/users/vladzcloudius/orgs","repos_url":"https://api.github.com/users/vladzcloudius/repos","events_url":"https://api.github.com/users/vladzcloudius/events{/privacy}","received_events_url":"https://api.github.com/users/vladzcloudius/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/scylladb/scylla/labels/bug","name":"bug","color":"eb6420"},{"url":"https://api.github.com/repos/scylladb/scylla/labels/high","name":"high","color":"d4c5f9"}],"state":"open","locked":false,"assignee":{"login":"asias","id":320076,"avatar_url":"https://avatars.githubusercontent.com/u/320076?v=3","gravatar_id":"","url":"https://api.github.com/users/asias","html_url":"https://github.com/asias","followers_url":"https://api.github.com/users/asias/followers","following_url":"https://api.github.com/users/asias/following{/other_user}","gists_url":"https://api.github.com/users/asias/gists{/gist_id}","starred_url":"https://api.github.com/users/asias/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asias/subscriptions","organizations_url":"https://api.github.com/users/asias/orgs","repos_url":"https://api.github.com/users/asias/repos","events_url":"https://api.github.com/users/asias/events{/privacy}","received_events_url":"https://api.github.com/users/asias/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/scylladb/scylla/milestones/1","html_url":"https://github.com/scylladb/scylla/milestones/GA","labels_url":"https://api.github.com/repos/scylladb/scylla/milestones/1/labels","id":1164307,"number":1,"title":"GA","description":null,"creator":{"login":"dorlaor","id":1735237,"avatar_url":"https://avatars.githubusercontent.com/u/1735237?v=3","gravatar_id":"","url":"https://api.github.com/users/dorlaor","html_url":"https://github.com/dorlaor","followers_url":"https://api.github.com/users/dorlaor/followers","following_url":"https://api.github.com/users/dorlaor/following{/other_user}","gists_url":"https://api.github.com/users/dorlaor/gists{/gist_id}","starred_url":"https://api.github.com/users/dorlaor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dorlaor/subscriptions","organizations_url":"https://api.github.com/users/dorlaor/orgs","repos_url":"https://api.github.com/users/dorlaor/repos","events_url":"https://api.github.com/users/dorlaor/events{/privacy}","received_events_url":"https://api.github.com/users/dorlaor/received_events","type":"User","site_admin":false},"open_issues":150,"closed_issues":87,"state":"open","created_at":"2015-06-15T06:37:52Z","updated_at":"2016-01-11T08:52:31Z","due_on":null,"closed_at":null},"comments":8,"created_at":"2015-12-28T15:44:27Z","updated_at":"2016-01-11T09:01:44Z","closed_at":null,"body":"scylla HEAD: `827a4d00106415e80d9d70395db0d60a2dcc366c`\r\nseastar HEAD: `8b2171e`\r\nsetup: `jepsen AMI running on c3.8xlarge instance`\r\n\r\nNodes seem to be too sensitive to networking problems during bootstrap phase. Tests like `lein test :only cassandra.collections.map-test/cql-map-isolate-node-bootstrap` and `lein test :only cassandra.collections.map-test/cql-map-bridge-bootstrap` always end up with at least one Node failing to bootstrap with the following message in the log:\r\n```\r\nDEBUG [shard 0] range_streamer - get_all_ranges_with_strict_sources_for ks=jepsen_keyspace\r\nExiting on unhandled exception of type 'std::runtime_error': A node required to move the data consistently is down (192.168.122.13).  If you wish to move the data from a potentially inconsistent replica, restart the node with -Dcassandra.\r\nconsistent.rangemovement=false\r\nINFO  [shard 0] compaction_manager - compaction task handler stopped due to shutdown\r\n``` \r\n\r\nIt seems that we need a similar treatment for failures in the bootstrap stage like we do in the decommission stage. "},"comment":{"url":"https://api.github.com/repos/scylladb/scylla/issues/comments/170472962","html_url":"https://github.com/scylladb/scylla/issues/721#issuecomment-170472962","issue_url":"https://api.github.com/repos/scylladb/scylla/issues/721","id":170472962,"user":{"login":"asias","id":320076,"avatar_url":"https://avatars.githubusercontent.com/u/320076?v=3","gravatar_id":"","url":"https://api.github.com/users/asias","html_url":"https://github.com/asias","followers_url":"https://api.github.com/users/asias/followers","following_url":"https://api.github.com/users/asias/following{/other_user}","gists_url":"https://api.github.com/users/asias/gists{/gist_id}","starred_url":"https://api.github.com/users/asias/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asias/subscriptions","organizations_url":"https://api.github.com/users/asias/orgs","repos_url":"https://api.github.com/users/asias/repos","events_url":"https://api.github.com/users/asias/events{/privacy}","received_events_url":"https://api.github.com/users/asias/received_events","type":"User","site_admin":false},"created_at":"2016-01-11T09:01:44Z","updated_at":"2016-01-11T09:01:44Z","body":"On Jan 11, 2016 4:51 PM, \"vladzcloudius\" <notifications@github.com> wrote:\n>\n>\n>\n> On 01/11/2016 10:29 AM, Asias He wrote:\n> > On Mon, Jan 11, 2016 at 4:27 PM, Tzach Livyatan <\nnotifications@github.com>\n> > wrote:\n> >\n> > > Or maybe u mean that \"it is related\" because we ignore it - then u may\n> > > be right... ;)\n> > >\n> > > From my reading of the code phi_convict_threshold is not ignored.\n> > >\n> >\n> > No, it is not. If you specify a non-default phi_convict_threshhold it\nwill\n> > take effects.\n>\n> And if we specify a default phi_convict_threshold (8)? Will it be\n> ignored then?\n\nNo. It will be set to 8.\n\n>\n> >\n> > —\n> > > Reply to this email directly or view it on GitHub\n> > > <https://github.com/scylladb/scylla/issues/721#issuecomment-170462537\n>.\n> > >\n> >\n> >\n> >\n> > --\n> > Asias\n>\n> >\n> > —\n> > Reply to this email directly or view it on GitHub\n> > <https://github.com/scylladb/scylla/issues/721#issuecomment-170463282>.\n>\n> >\n>\n> —\n> Reply to this email directly or view it on GitHub.\n"}}